Using Electronic Case Summaries to Elicit Multi-Disciplinary Expert Knowledge about Referrals to Post-Acute Care

Summary

Background

Eliciting knowledge from geographically dispersed experts given their time and scheduling constraints, while maintaining anonymity among them, presents multiple challenges.

Objectives

Describe an innovative, Internet based method to acquire knowledge from experts regarding patients who need post-acute referrals. Compare, 1) the percentage of patients referred by experts to percentage of patients actually referred by hospital clinicians, 2) experts’ referral decisions by disciplines and geographic regions, and 3) most common factors deemed important by discipline.

Methods

De-identified case studies, developed from electronic health records (EHR), contained a comprehensive description of 1,496 acute care inpatients. In teams of three, physicians, nurses, social workers, and physical therapists reviewed case studies and assessed the need for post-acute care referrals; Delphi rounds followed when team members did not agree. Generalized estimating equations (GEEs) compared experts’ decisions by discipline, region of the country and to the decisions made by study hospital clinicians, adjusting for the repeated observations from each expert and case. Frequencies determined the most common case characteristics chosen as important by the experts.

Results

The experts recommended referral for 80% of the cases; the actual discharge disposition of the patients showed referrals for 67%. Experts from the Northeast and Midwest referred 5% more cases than experts from the West. Physicians and nurses referred patients at similar rates while both referred more often than social workers. Differences by discipline were seen in the factors identified as important to the decision.

Conclusion

The method for eliciting expert knowledge enabled national dispersed expert clinicians to anonymously review case summaries and make decisions about post-acute care referrals. Having time and a comprehensive case summary may have assisted experts to identify more patients in need of post-acute care than the hospital clinicians. The methodology produced the data needed to develop an expert decision support system for discharge planning.

Citation: Bowles KH, Ratcliffe S, Potashnik S, Topaz M, Holmes J, Shih N-W, Naylor MD. Using electronic case summaries to elicit multi-disciplinary expert knowledge about referrals to post-acute care.

Keywords

Knowledge elicitation - decision support - discharge planning - nursing informatics - Delphi rounds

 

Conflicts of Interest

The authors declare they have no conflicts of interest in the research.

• References

• 1 Bowles KH, Foust JB, Naylor MD. Hospital discharge referral decision making: a multidisciplinary perspective. Appl Nurs Res 2003; Aug; 16 (03) 134-143.
• 2 Milton NR. Knowledge acquisition in practice: a step-by-step guide. Springer; 2007
• 3 Musen MA, Fagan LM, Combs DM, Shortliffe EH. Use of a domain model to drive an interactive knowledge-editing tool. Int J Human Comput Stud 1999; 51 (02) 479-495.
• 4 Dexter F, Wachtel RE, Epstein RH. Event-based knowledge elicitation of operating room management decision-making using scenarios adapted from information systems data. BMC Med Inform Decis Mak 2011; 11: 02 6947–11–2.
• 5 Gibert K, García-Alonso C, Salvador-Carulla L. Integrating clinicians, knowledge and data: Expert-based cooperative analysis in healthcare decision support. Health Res Policy and Systems 2010; 8.
• 6 Groznik V, Guid M, Sadikov A, Mozina M, Georgiev D, Kragelj V, Ribaric S, Pirtosek Z, Bratko I. Elicitation of neurological knowledge with argument-based machine learning. Artif Intell Med 2013; 57 (02) 133-144.
• 7 Keune H, Gutleb AC, Zimmer KE, Ravnum S, Yang A, Bartonova A, Krayer von Krauss M, Ropstad E, Eriksen GS, Saunders M, Magnati B, Forsberg B. We’re only in it for the knowledge? A problem solving turn in environment and health expert elicitation. Environ Health 2012; 11 (Suppl. 01) S3 069X-11-S1-S3.
• 8 Gavrilova T, Andreeva T. Knowledge elicitation techniques in a knowledge management context. J Knowledge Management 2012; 16 (04) 523-537.
• 9 Ziebell D, Fiore SM, Becerra-Fernandez I. Knowledge Management Revisited. IEEE Intelligent Systems 2008; 23 (03) 84-88.
• 10 Okafor EC, Osuagwu CC. The underlying issues in knowledge elicitation. Interdisc J of Information, Knowledge, and Management 2006; 01: 95-108.
• 11 Cooke NJ. Knowledge Elicitation. In: Durso FT, Nickerson RS, Schvaneveldt RW, Dumais ST, Lindsay DS, Chi MTH. editors. Handbook of Applied Cognition. First ed.. Chichester, New York: Wiley; 1999: 479-501.
• 12 Bech-Larsen T, Nielsen NA. A comparison of five elicitation techniques for elicitation of attributes of low involvement products. J Econ Psych 1999; 20: 315-341.
• 13 Orem DE. Nursing: Concepts of Practice. 5th ed.. St. Louis, MO: Mosby; 1995
• 14 Bowles KH, Holmes JH, Naylor MD, Liberatore M, Nydick R. Expert consensus for discharge referral decisions using online delphi. AMIA Annu Symp Proc 2003; 106-9.
• 15 Bowles K, Potashnik S, Ratcliffe S, Rosenberg M, Shih N, Topaz M, Holmes J, Naylor M. Conducting research using the electronic health record across multi-hospital systems: semantic harmonization implications for administrators. J Nurs Admin 2013; 43 (06) 355-360.
• 16 Ai MY, Li K, Lin DJK. Balanced incomplete Latin square designs. J Statistical Planning and Inference 2013; 143: 1575-1582.
• 17 Rowe G, Wright G. The Delphi technique as a forecasting tool: issues and analysis. Int J Forecasting 1999; 15 (04) 353-375.
• 18 Liang KY, Zeger S. Longitudinal data analysis using generalized linear models. Biometrics 1986; 73: 13-22.
• 19 Zeger S, Liang KY. Longitudinal data analysis for discrete and continuous outcomes. Biometrics 1986; 42: 121-130.
• 20 Walden G, Schwartz A. Walden, Schwartz introduce bipartisan legislation to ensure seniors and disabled can access home health services. [Internet]. Washington, DC: U.S. Representative Greg Walden; 2015. [cited January 18, 2016]. Available from: https://walden.house.gov/media-center/press-releases/waldenschwartz-introduce-bipartisan-legislation-ensure-seniors-and
• 21 Maramba PJ, Richards S, Larrabee JH. Discharge Planning Process, Applying a Model for Evidence-Based Practice. J Nurs Care Qual 2004; 19 (02) 123-129.
• 22 Holland DE, Bowles KH. Standardized discharge planning assessments: impact on patient outcomes. J Nurs Care Qual 2012; 27 (03) 200-208.