The star rating system for hospitals of the Centers for Medicare and Medicaid Services (CMS) pools all hospitals together and awards each institution one to five stars for quality, despite variation across hospitals in the numbers and types of measures they report. Thus, hospitals essentially are being evaluated differently, which affects the validity of quality comparisons. We considered the number and types of measures reported and the size of measure denominators to represent different forms of a "test," and we used data from the December 2017 star ratings to show that hospitals took one of three general "test forms." Hospitals taking the most extensive test form reported an average of forty-three measures, while those taking the least extensive test reported an average of twenty-two measures. These test forms were differentially associated with star ratings and hospital characteristics. Our results caution against pooling all hospitals together when assigning star ratings, and they demonstrate a feasible approach to segmenting hospitals into peer groups for evaluation by stakeholders such as CMS.
Keywords: Hospital quality; Hospitals; Medicare; Performance Measurement.