Download PDF
Appl Clin Inform 2021; 12(01): 153-163
DOI: 10.1055/s-0041-1722917
Research Article

The Development and Piloting of the Ambulatory Electronic Health Record Evaluation Tool: Lessons Learned

Zoe Co
1   Division of General Internal Medicine, Brigham and Women's Hospital, Boston, Massachusetts, United States
,
A. Jay Holmgren
2   Harvard Business School, Boston, Massachusetts, United States
,
David C. Classen
3   Department of Clinical Epidemiology, University of Utah, Salt Lake City, Utah, United States
,
Lisa P. Newmark
4   Clinical and Quality Analysis, Mass General Brigham, Somerville, Massachusetts, United States
,
Diane L. Seger
4   Clinical and Quality Analysis, Mass General Brigham, Somerville, Massachusetts, United States
,
Jessica M. Cole
5   Department of Internal Medicine, Division of Epidemiology, University of Utah, Salt Lake City, Utah, United States
,
Barbara Pon
6   Collaborative Healthcare Patient Safety Organization, Sacramento, California, United States
,
Karen P. Zimmer
7   Department of Pediatrics, Thomas Jefferson University, Philadelphia, Pennsylvania, United States
,
David W. Bates
1   Division of General Internal Medicine, Brigham and Women's Hospital, Boston, Massachusetts, United States
4   Clinical and Quality Analysis, Mass General Brigham, Somerville, Massachusetts, United States
8   Harvard Medical School, Boston, Massachusetts, United States
› Author Affiliations Funding This study was funded by the Gordon and Betty Moore Foundation.
› Further Information
Also available at
   Permissions and Reprints

Abstract

Background Substantial research has been performed about the impact of computerized physician order entry on medication safety in the inpatient setting; however, relatively little has been done in ambulatory care, where most medications are prescribed.

Objective To outline the development and piloting process of the Ambulatory Electronic Health Record (EHR) Evaluation Tool and to report the quantitative and qualitative results from the pilot.

Methods The Ambulatory EHR Evaluation Tool closely mirrors the inpatient version of the tool, which is administered by The Leapfrog Group. The tool was piloted with seven clinics in the United States, each using a different EHR. The tool consists of a medication safety test and a medication reconciliation module. For the medication test, clinics entered test patients and associated test orders into their EHR and recorded any decision support they received. An overall percentage score of unsafe orders detected, and order category scores were provided to clinics. For the medication reconciliation module, clinics demonstrated how their EHR electronically detected discrepancies between two medication lists.

Results For the medication safety test, the clinics correctly alerted on 54.6% of unsafe medication orders. Clinics scored highest in the drug allergy (100%) and drug–drug interaction (89.3%) categories. Lower scoring categories included drug age (39.3%) and therapeutic duplication (39.3%). None of the clinics alerted for the drug laboratory or drug monitoring orders. In the medication reconciliation module, three (42.8%) clinics had an EHR-based medication reconciliation function; however, only one of those clinics could demonstrate it during the pilot.

Conclusion Clinics struggled in areas of advanced decision support such as drug age, drug laboratory, and drub monitoring. Most clinics did not have an EHR-based medication reconciliation function and this process was dependent on accessing patients' medication lists. Wider use of this tool could improve outpatient medication safety and can inform vendors about areas of improvement.

Keywords

electronic health record - computerized physician order entry - ambulatory care - patient safety - quality of care - medication safety - medication reconciliation

Protection of Human and Animal Subjects

No real patients were used in Ambulatory EHR Evaluation Tool, only test patients were used.


Supplementary Material



Publication History

Received: 11 August 2020

Accepted: 16 December 2020

Article published online:
03 March 2021

© 2021. Thieme. All rights reserved.

Georg Thieme Verlag KG
Rüdigerstraße 14, 70469 Stuttgart, Germany

 

  • References

  • 1 Blumenthal D. Launching HITECH. N Engl J Med 2010; 362 (05) 382-385
    CrossrefPubMedGoogle Scholar
  • 2 Kuperman GJ, Bobb A, Payne TH. et al. Medication-related clinical decision support in computerized provider order entry systems: a review. J Am Med Inform Assoc 2007; 14 (01) 29-40
    CrossrefPubMedGoogle Scholar
  • 3 Bates DW, Teich JM, Lee J. et al. The impact of computerized physician order entry on medication error prevention. J Am Med Inform Assoc 1999; 6 (04) 313-321
    CrossrefPubMedGoogle Scholar
  • 4 Gregory ME, Russo E, Singh H. Electronic health record alert-related workload as a predictor of burnout in primary care providers. Appl Clin Inform 2017; 8 (03) 686-697
    Article in Thieme ConnectPubMedGoogle Scholar
  • 5 Radley DC, Wasserman MR, Olsho LE, Shoemaker SJ, Spranca MD, Bradshaw B. Reduction in medication errors in hospitals due to adoption of computerized provider order entry systems. J Am Med Inform Assoc 2013; 20 (03) 470-476
    CrossrefPubMedGoogle Scholar
  • 6 Bates DW, Leape LL, Cullen DJ. et al. Effect of computerized physician order entry and a team intervention on prevention of serious medication errors. JAMA 1998; 280 (15) 1311-1316
    CrossrefPubMedGoogle Scholar
  • 7 Holmgren AJ, Co Z, Newmark L, Danforth M, Classen D, Bates D. Assessing the safety of electronic health records: a national longitudinal study of medication-related decision support. BMJ Qual Saf 2020; 29 (01) 52-59
    CrossrefPubMedGoogle Scholar
  • 8 Classen DC, Holmgren AJ, Co Z. et al. National trends in the safety performance of electronic health record systems from 2009 to 2018. JAMA Netw Open 2020; 3 (05) e205547
    CrossrefPubMedGoogle Scholar
  • 9 Denham CR, Classen DC, Swenson SJ, Henderson MJ, Zeltner T, Bates DW. Safe use of electronic health records and health information technology systems: trust but verify. J Patient Saf 2013; 9 (04) 177-189
    CrossrefPubMedGoogle Scholar
  • 10 Kaushal R, Kern LM, Barrón Y, Quaresimo J, Abramson EL. Electronic prescribing improves medication safety in community-based office practices. J Gen Intern Med 2010; 25 (06) 530-536
    CrossrefPubMedGoogle Scholar
  • 11 Gandhi TK, Weingart SN, Seger AC. et al. Outpatient prescribing errors and the impact of computerized prescribing. J Gen Intern Med 2005; 20 (09) 837-841
    CrossrefPubMedGoogle Scholar
  • 12 Singh H, Spitzmueller C, Petersen NJ, Sawhney MK, Sittig DF. Information overload and missed test results in EHR-based settings. JAMA Intern Med 2013; 173 (08) 702-704
    CrossrefPubMedGoogle Scholar
  • 13 Ancker JS, Edwards A, Nosal S, Hauser D, Mauer E, Kaushal R. with the HITEC Investigators. Effects of workload, work complexity, and repeated alerts on alert fatigue in a clinical decision support system. BMC Med Inform Decis Mak 2017; 17 (01) 36
    CrossrefPubMedGoogle Scholar
  • 14 Co Z, Holmgren AJ, Classen DC. et al. The tradeoffs between safety and alert fatigue: data from a national evaluation of hospital medication-related clinical decision support. J Am Med Inform Assoc 2020; 27 (08) 1252-1258
    CrossrefPubMedGoogle Scholar
  • 15 Kilbridge PM, Welebob EM, Classen DC. Development of the Leapfrog methodology for evaluating hospital implemented inpatient computerized physician order entry systems. Qual Saf Health Care 2006; 15 (02) 81-84
    CrossrefPubMedGoogle Scholar
  • 16 Metzger J, Welebob E, Bates DW, Lipsitz S, Classen DC. Mixed results in the safety performance of computerized physician order entry. Health Aff (Millwood) 2010; 29 (04) 655-663
    CrossrefPubMedGoogle Scholar
  • 17 Leung AA, Keohane C, Lipsitz S. et al. Relationship between medication event rates and the Leapfrog computerized physician order entry evaluation tool. J Am Med Inform Assoc 2013; 20 (e1): e85-e90
    CrossrefPubMedGoogle Scholar
  • 18 National Quality Forum. Safe Practices for Better Healthcare-2010 Update: A Consensus Report. 2010
    Google Scholar
  • 19 The Leapfrog Group. Leapfrog Hospital Survey: Questions & Reporting Periods Endnotes Measure Specifications FAQS. 2020
    Google Scholar
  • 20 Gandhi TK, Weingart SN, Borus J. et al. Adverse drug events in ambulatory care. N Engl J Med 2003; 348 (16) 1556-1564
    CrossrefPubMedGoogle Scholar
  • 21 Phansalkar S, van der Sijs H, Tucker AD. et al. Drug-drug interactions that should be non-interruptive in order to reduce alert fatigue in electronic health records. J Am Med Inform Assoc 2013; 20 (03) 489-493
    CrossrefPubMedGoogle Scholar
  • 22 Office of the National Coordinator for Health Information Technology. Non-federal Acute Care Hospital Electronic Health Record Adoption. Published 2017. Accessed January 11, 2019 at: https://dashboard.healthit.gov/quickstats/pages/FIG-Hospital-EHR-Adoption.php
    Google Scholar
  • 23 Ratwani R, Fairbanks T, Savage E. et al. Mind the Gap. A systematic review to identify usability and safety challenges and practices during electronic health record implementation. Appl Clin Inform 2016; 7 (04) 1069-1087
    Article in Thieme ConnectPubMedGoogle Scholar
  • 24 Dhillon-Chattha P, McCorkle R, Borycki E. An evidence-based tool for safe configuration of electronic health records: the eSafety checklist. Appl Clin Inform 2018; 9 (04) 817-830
    Article in Thieme ConnectPubMedGoogle Scholar
  • 25 Wahls TL, Cram PM. The frequency of missed test results and associated treatment delays in a highly computerized health system. BMC Fam Pract 2007; 8 (01) 32
    CrossrefPubMedGoogle Scholar
  • 26 Hysong SJ, Sawhney MK, Wilson L. et al. Understanding the management of electronic test result notifications in the outpatient setting. BMC Med Inform Decis Mak 2011; 11: 22 . Doi: 10.1186/1472-6947-11-22
    CrossrefPubMedGoogle Scholar
  • 27 Weingart SN, Toth M, Sands DZ, Aronson MD, Davis RB, Phillips RS. Physicians' decisions to override computerized drug alerts in primary care. Arch Intern Med 2003; 163 (21) 2625-2631
    CrossrefPubMedGoogle Scholar
  • 28 Scott K, Hathaway E, Sharp K, Smailes P. The development and evaluation of an electronic health record efficiency workshop for providers. Appl Clin Inform 2020; 11 (02) 336-341
    Article in Thieme ConnectPubMedGoogle Scholar
  • 29 Wakefield M. Patient safety and quality: an evidence-based handbook for nurses. (The quality chasm series: implications for nursing; ). In: Agency for Healthcare Research and Quality A. ed. Agency for Healthcare Research and Quality, AHRQ. Agency for Healthcare Research and Quality (US); 2008: 1-1403 . Accessed October 28, 2020 at: https://www.ncbi.nlm.nih.gov/books/NBK2648/
    Google Scholar
  • 30 Wagner MM, Hogan WR. The accuracy of medication data in an outpatient electronic medical record. J Am Med Inform Assoc 1996; 3 (03) 234-244
    CrossrefPubMedGoogle Scholar
  • 31 Tamblyn R, Abrahamowicz M, Buckeridge DL. et al. Effect of an electronic medication reconciliation intervention on adverse drug events: a cluster randomized trial. JAMA Netw Open 2019; 2 (09) e1910756
    CrossrefPubMedGoogle Scholar
  • 32 Mekonnen AB, Abebe TB, McLachlan AJ, Brien JAE. Impact of electronic medication reconciliation interventions on medication discrepancies at hospital transitions: a systematic review and meta-analysis. BMC Med Inform Decis Mak 2016; 16 (01) 112
    CrossrefPubMedGoogle Scholar