Adjuvant endocrine therapy for breast cancer reduces recurrence and improves survival rates. Many patients never start treatment or discontinue prematurely. A better understanding of factors associated with endocrine therapy initiation and persistence could inform practitioners how to support patients. We analyzed data from a longitudinal study of 2,268 women diagnosed with breast cancer and reported to the Metropolitan Detroit and Los Angeles SEER cancer registries in 2005-2007. Patients were surveyed approximately both 9 months and 4 years after diagnosis. At the 4-year mark, patients were asked if they had initiated endocrine therapy, terminated therapy, or were currently taking therapy (defined as persistence). Multivariable logistic regression models examined factors associated with initiation and persistence. Of the 743 patients eligible for endocrine therapy, 80 (10.8 %) never initiated therapy, 112 (15.1 %) started therapy but discontinued prematurely, and 551 (74.2 %) continued use at the second time point. Compared with whites, Latinas (OR 2.80, 95 % CI 1.08-7.23) and black women (OR 3.63, 95 % CI 1.22-10.78) were more likely to initiate therapy. Other factors associated with initiation included worry about recurrence (OR 3.54, 95 % CI 1.31-9.56) and inadequate information about side effects (OR 0.24, 95 % CI 0.10-0.55). Factors associated with persistence included two or more medications taken weekly (OR 4.19, 95 % CI 2.28-7.68) and increased age (OR 0.98, 95 % CI 0.95-0.99). Enhanced patient education about potential side effects and the effectiveness of adjuvant endocrine therapy in improving outcomes may improve initiation and persistence rates and optimize breast cancer survival.