Assessment of new diagnostic tests should be carried out in a population with suspicion of disease. This 'indicated' population may comprise all patients for whom the target disease is part of the differential diagnosis. To investigate this empirically, data from a study on ultrasonography for scrotal pathology diagnosis was used for assessment of ultrasonography for epididymitis. The population could be selected according to several inclusion criteria, varying from patients with epididymitis mimicking diseases in their differential diagnosis, to patients with only epididymitis mentioned in their differential diagnosis. Different methods of population gathering led to large variations in the test characteristics of ultrasonography (sensitivity 76-83%, specificity 79-97%, prevalence of epididymitis for test-negative patients 56-5%). These results suggest selection bias depending on the true disease status. We advocate prospective assessment studies in populations with disease suspicion, as defined by signs, symptoms and complaints. Constituting an indicated population from an existing file of retrospectively gathered data may pose problems.