Background: Clinicians and researchers often wish to know how patients perceive the likelihoods of health risks. Little work has been done to develop and validate scales and formats to measure perceptions of event probabilities, particularly low probabilities (i.e., <1%).
Objective: To compare a new visual analog scale with three benchmarks in terms of validity and reliability.
Design: Survey with retest after approximately two weeks. Respondents estimated the probabilities of six events with the new scale, which featured a "magnifying glass" to represent probabilities between 0 and 1% on a logarithmic scale. Participants estimated the same probabilities on three benchmarks: two linear visual analog scales (one labeled with words, one with numbers) and a "1 in x" scale.
Subjects: 100 veterans and family members and 107 university faculty and students.
Measures: For each scale, the authors assessed: 1) validity-the correlation between participants' direct rankings (i.e., numbering them from 1 to 6) and scale-derived rankings of the relative probabilities of six events; 2) test-retest reliability-the correlation of responses from test to retest two weeks later; 3) usability (missing/ incorrect responses, participant evaluation).
Results: Both the magnifier and the two linear scales outperformed the "1 in x" scale on all criteria. The magnifier scale performed about as well as the two linear visual analog scales for validity (correlation between direct and scale-derived rankings = 0.72), reliability (test-retest correlation = 0.55), and usability (2% missing or incorrect responses, 65% rated it easy to use). 62% felt the magnifier scale was a "very good or good" indicator of their feelings about chance. The magnifier scale facilitated expression of low-probability judgments. For example, the estimated chance of parenting sextuplets was orders of magnitude lower on the magnifier scale (median perceived chance 10(-5)) than on its linear counterpart (10(-2)). Participants' assessments of high-probability events (e.g., chance of catching a cold in the next year) were not affected by the presence of the magnifier.
Conclusions: The "1 in x" scale performs poorly and is very difficult for people to use. The magnifier scale and the linear number scale are similar in validity, reliability, and usability. However, only the magnifier scale makes it possible to elicit perceptions in the low-probability range (<1%).