Background Despite past and ongoing efforts to achieve health equity in the United States, persistent disparities in socioeconomic status along with multilevel racism maintain disparate outcomes and appear to be amplified by COVID-19. Objective Measure socioeconomic factors and primary language effects on the risk of COVID-19 severity across and within racial/ethnic groups. Design Retrospective cohort study. Setting Health records of 12 Midwest hospitals and 60 clinics in the U.S. between March 4, 2020 to August 19, 2020. Patients PCR+ COVID-19 patients. Exposures Main exposures included race/ethnicity, area deprivation index (ADI), and primary language. Main Outcomes and Measures The primary outcome was COVID-19 severity using hospitalization within 45 days of diagnosis. Logistic and competing-risk regression models (censored at 45 days and accounting for the competing risk of death prior to hospitalization) assessed the effects of neighborhood-level deprivation (using the ADI) and primary language. Within race effects of ADI and primary language were measured using logistic regression. Results 5,577 COVID-19 patients were included, 866 (n=15.5%) were hospitalized within 45 days of diagnosis. Hospitalized patients were older (60.9 vs. 40.4 years, p<0.001) and more likely to be male (n=425 [49.1%] vs. 2,049 [43.5%], p=0.002). Of those requiring hospitalization, 43.9% (n=381), 19.9% (n=172), 18.6% (n=161), and 11.8% (n=102) were White, Black, Asian, and Hispanic, respectively. Independent of ADI, minority race/ethnicity was associated with COVID-19 severity; Hispanic patients (OR 3.8, 95% CI 2.72-5.30), Asians (OR 2.39, 95% CI 1.74-3.29), and Blacks (OR 1.50, 95% CI 1.15-1.94). ADI was not associated with hospitalization. Non-English speaking (OR 1.91, 95% CI 1.51-2.43) significantly increased odds of hospital admission across and within minority groups. Conclusions Minority populations have increased odds of severe COVID-19 independent of neighborhood deprivation, a commonly suspected driver of disparate outcomes. Non-English-speaking accounts for differences across and within minority populations. These results support the continued concern that racism contributes to disparities during COVID-19 while also highlighting the underappreciated role primary language plays in COVID-19 severity across and within minority groups.