The objective of this study was to estimate the influence of substance use on the quality of patient-provider communication during HIV clinic encounters. Patients were surveyed about unhealthy alcohol and illicit drug use and rated provider communication quality. Audio-recorded encounters were coded for specific communication behaviors. Patients with vs. without unhealthy alcohol use rated the quality of their provider's communication lower; illicit drug user ratings were comparable to non-users. Visit length was shorter, with fewer activating/engaging and psychosocial counseling statements for those with vs. without unhealthy alcohol use. Providers and patients exhibited favorable communication behaviors in encounters with illicit drug users vs. non-users, demonstrating greater evidence of patient-provider engagement. The quality of patient-provider communication was worse for HIV-infected patients with unhealthy alcohol use but similar or better for illicit drug users compared with non-users. Interventions should be developed that encourage providers to actively engage patients with unhealthy alcohol use.