Background: Eliciting knowledge from geographically dispersed experts given their time and scheduling constraints, while maintaining anonymity among them, presents multiple challenges.
Objectives: Describe an innovative, Internet based method to acquire knowledge from experts regarding patients who need post-acute referrals. Compare, 1) the percentage of patients referred by experts to percentage of patients actually referred by hospital clinicians, 2) experts' referral decisions by disciplines and geographic regions, and 3) most common factors deemed important by discipline.
Methods: De-identified case studies, developed from electronic health records (EHR), contained a comprehensive description of 1,496 acute care inpatients. In teams of three, physicians, nurses, social workers, and physical therapists reviewed case studies and assessed the need for post-acute care referrals; Delphi rounds followed when team members did not agree. Generalized estimating equations (GEEs) compared experts' decisions by discipline, region of the country and to the decisions made by study hospital clinicians, adjusting for the repeated observations from each expert and case. Frequencies determined the most common case characteristics chosen as important by the experts.
Results: The experts recommended referral for 80% of the cases; the actual discharge disposition of the patients showed referrals for 67%. Experts from the Northeast and Midwest referred 5% more cases than experts from the West. Physicians and nurses referred patients at similar rates while both referred more often than social workers. Differences by discipline were seen in the factors identified as important to the decision.
Conclusion: The method for eliciting expert knowledge enabled national dispersed expert clinicians to anonymously review case summaries and make decisions about post-acute care referrals. Having time and a comprehensive case summary may have assisted experts to identify more patients in need of post-acute care than the hospital clinicians. The methodology produced the data needed to develop an expert decision support system for discharge planning.
Keywords: Delphi rounds; Knowledge elicitation; decision support; discharge planning; nursing informatics.