The objective of this study was to compare physician-level survey instruments and estimate the number of patients needed per physician to provide reliable estimates of health care. The setting consisted of 3 health plans and 1 large physician group in the greater Cincinnati metro area. Surveys were mailed to patients of 100 primary care physicians. Patients were mailed either the Consumer Assessment of Health Plans Study (CAHPS) or DoctorGuide survey instrument. A total of 4245 CAHPS surveys and 5519 DoctorGuide surveys were returned. Internal consistency reliability estimates for the multi-item scales (access to care, communication, and preventive care) for both surveys were adequate. The number of patient responses needed to obtain a reliability of 0.70 at the physician level for the access to care, communication, and preventive care scales were 32, 43, and 38, respectively, for the CAHPS survey and 26, 25, and 47, respectively, for the DoctorGuide survey. These results indicate similar and parallel psychometric performance for the DoctorGuide and CAHPS survey instruments.