Observational, cross-sectional and longitudinal studies showed that physical activity and sedentary behaviour are associated with adiposity-related traits, apparently in a bidirectional manner. Physical activity is also suggested to suppress the genetic risk of adiposity. Since phenotypic associations with genetic variants are not subject to reverse causation or confounding, they may be used as tools to shed light on cause and effect in this complex interdependency. We review the evidence for shared genetics of physical activity and adiposity-related traits and for gene-by-physical activity interactions on adiposity-related traits in human studies. We outline limitations, challenges and opportunities in studying and understanding of these relationships. In summary, physical activity and sedentary behaviour are genetically correlated with body mass index and fat percentage but may not be correlated with lean body mass. Mendelian randomisation analyses show that physical activity and sedentary behaviour have bidirectional relationships with adiposity. Several studies suggest that physical activity suppresses genetic risk of adiposity. No studies have yet tested whether adiposity enhances genetic predisposition to sedentariness. The complexity of the comprehensive causal model makes the assessment of the single or combined components challenging. Substantial progress in this field may need long-term intervention studies.