Prostate cancer (PCa) is the most common visceral malignancy in men with androgen deprivation therapy (ADT) the preferred therapy to suppress testosterone production and hence tumor growth. Despite its effectiveness in lowering testosterone, ADT is associated with side effects including loss of muscle mass, diminished muscle strength, decrements in physical performance, earlier fatigue and declining quality of life. This review reports a survey of the literature with a focus on changes in muscle strength, physical function and body composition, due to short-term and long-term ADT. Studies in these areas are sparse, especially well-controlled, prospective randomized trials. Cross-sectional and longitudinal data (up to 2 years) for men with PCa treated with ADT as well as patients with PCa not receiving ADT and age-matched healthy men are presented when available. Based on limited longitudinal data, the adverse effects of ADT on muscle function, physical performance and body composition occur shortly after the onset of ADT and tend to persist and worsen over time. Exercise training is a safe and effective intervention for mitigating these changes and initial guidelines for exercise program design for men with PCa have been published by the American College of Sports Medicine. Disparities in study duration, types of studies and other patient-specific variables such as time since diagnosis, cancer stage and comorbidities may all affect an understanding of the influence of ADT on health, physical performance and mortality.