Introduction: Electronic quality monitoring(eQuality) from clinical narratives may advance current manual quality measurement techniques.We evaluated automated eQuality measurement tools on clinical narratives of veterans' disability examinations.
Methods: We used a general purpose indexing engine to encode clinical concepts with SNOMED CT. We developed computer usable quality assessment rules from established quality indicators and evaluated the automated approach against a gold standard of double independent human expert review. Rules were iteratively improved using a training set of 1446 indexed exam reports and evaluated on a test set of 1454 indexed exam reports.
Results: The eQuality system achieved 86%sensitivity (recall), 62% specificity, and 96%positive predictive value (precision) for automated quality assessment of veterans' disability exams. Summary data for each exam type and detailed data for joint exam quality assessments are presented.
Discussion: The current results generalize our previous results to ten exam types covering over 200 diagnostic codes. eQuality measurement from narrative clinical documents has the potential to improve healthcare quality and safety.