Differences, Opportunities, and Strategies in Drug Alert Optimization—Experiences of Two Different Integrated Health Care Systems

Abstract

Background Concerns about the number of automated medication alerts issued within the electronic health record (EHR), and the subsequent potential for alarm fatigue, led us to examine strategies and methods to optimize the configuration of our drug alerts.

Objectives This article reports on comprehensive drug alerting rates and develops strategies across two different health care systems to reduce the number of drug alerts.

Methods Standardized reports compared drug alert rates between the two systems, among 13 categories of drug alerts. Both health care systems made modifications to the out-of-box alerts available from their EHR and drug information vendors, focusing on system-wide approaches, when relevant, while performing more drug-specific changes when necessary.

Results Drug alerting rates even after initial optimization were 38 alerts and 51 alerts per 100 drug orders, respectively. Eight principles were identified and developed to reflect the themes in the implementation and optimization of drug alerting.

Conclusion A team-based, systematic approach to optimizing drug-alerting strategies can reduce the number of drug alerts, but alert rates still remain high. In addition to strategic principles, additional tactical guidelines and recommendations need to be developed to enhance out-of-the-box clinical decision support for drug alerts.

Keywords

clinical decision support - process management tools - specific types - clinical information systems - drug–drug interaction - alerting - system improvement - alert fatigue

Background and Significance

With the accelerated adoption of electronic health record (EHR) systems, the amount of data collected and available to physicians for clinical decision making has vastly increased. Computerized physician order entry (CPOE) reduces medication error rates by up to 55%, with an 88% reduction in serious medication errors.[1] New concerns have arisen about the number of automated medication alerts issued within the EHR, and the continued potential for errors. Studies suggest that up to 50% of drug alerts are overridden or ignored in the outpatient setting, and up to 75% of alerts are overridden in the inpatient setting, suggesting alert fatigue, which could potentially result in medical error.[2] [3] Despite the high number of clinical decision support (CDS) alerts, a recent survey of hospitals in the United States showed that up to 39% of potentially harmful drug orders were missed by the EHR, while 13% of potentially fatal medication orders did not generate an alert.[4] Past studies have presented strategies to make the alerts more specific and relevant,[5] [6] but few reports quantify the number of total alerts across all categories of drug alerts and compare the results of these strategies between health care systems. Further, there are no standards about which specific alerts (e.g., drug–drug) to implement or how to display them to providers.[7] The aims of this study were to (1) provide a taxonomy/listing of drug alerts, (2) quantify the number of drug alerts across drug alerting categories, (3) explore ways to provide a detailed look at areas of highest drug alerting, (4) use comparative data to examine the effect of different approaches to optimize drug alerting, and (5) to describe generalizable strategies for drug alerting CDS.

Methods

The MetroHealth System (MHS) is a safety-net academic health care system in Northeast Ohio. MHS employs over 550 physicians and more than 350 physicians in training across 21 health centers, 4 emergency departments, and 13 schools. MHS performed over 1.2 million outpatient visits, 25,000 inpatient stays, and 120,000 emergency room visits in 2017. In 1999, MHS adopted the Epic EHR system (Epic Systems Corporation, Verona, Wisconsin, United States) along with CPOE. MHS has achieved Healthcare Information Management and Systems Society (HIMSS, Chicago, Illinois, United States) Electronic Medical Record Adoption Model Stage 7 certification throughout its hospital and ambulatory settings. MHS uses the drug information vendor MediSpan (Wolters Kluwer Clinical Drug Information, Hudson, Ohio, United States) to provide drug information for CDS. Methodology and strategies for analyzing and optimizing drug-dose alerts at MHS have been previously described.[6]

CaroMont Health (CH) is an independent tertiary care center in Gastonia, North Carolina, United States. CH has over 500 staff physicians across more than 45 care sites. CH performs over 810,000 outpatient visits, 20,000 inpatient stays, and 108,000 emergency room visits annually. CH adopted the Epic EHR system in 2016. CH is a certified Healthcare Information Management and Systems Society Electronic Medical Record Adoption Model Stage 6 health care system and has received a grade of “A” from the Leapfrog Hospital Safety Group (Leapfrog Group, Washington, D.C., United States) since 2012. CH uses First DataBank (Hearst Health Network, New York, New York, United States) to provide drug information for CDS.

Both health care systems created integrated teams that included their Chief Medical Informatics Officer, pharmacy informatics specialists, and physician informatics specialists. The focus of these teams was to implement and optimize their drug alert initiatives. These teams analyzed opportunities for decreasing the number of alerts generated and discussed the clinical utility of the different types of alerts. Each system's pharmacy and therapeutics committee made the final implementation decisions. Alert rates for each health care system were compiled using standardized EHR vendor reports. Subsequent system-level changes included suppression of entire drug alerting categories, as well as modification of default system level settings, and modifications to individual drug-specific alerts.

Both health care systems performed a retrospective study of standardized drug alerts across all care settings, over a 3-month period from December 2016 to February 2017. Inclusion criteria were all alerts, and there were no exclusion criteria. There are 13 possible categories of drug alerts available in the EHR used ([Table 1]), but at baseline, CH only used 9 of these categories, while MHS used 8. We focused on drug–drug interaction, drug allergy, drug dose, lactation, pregnancy, total parenteral nutrition, duplicate therapy, and duplicate medication order. Both systems had previously decided that intravenous compatibility, drug–food, drug–alcohol, drug–disease (CH only), and age/sex provided alerts of low clinical yield and did not turn them on. The drug alerts appear to the ordering provider during order entry at both health care systems.

Table 1 Categories/taxonomy of drug alerts

Category of Drug Alerts
• Drug-drug interactions alerts
• Drug allergy interactions alerts
• Drug dose alerts  ○ Minimum daily dose limit alert  ○ Minimum dose frequency alert  ○ Maximum single dose alert  ○ Maximum daily dose alert  ○ Maximum dose frequency alert  ○ Maximum dose duration alert
• Drug–disease interaction alerts
• Drug–lactation interactions alerts
• Drug–pregnancy interactions alerts
• Drug–intravenous (IV) compatibility alerts
• Drug–total parenteral nutrition (TPN) interaction alerts
• Drug–food interaction alerts
• Drug–alcohol interaction alerts
• Drug–duplicate therapeutic class alerts
• Drug–duplicate medication alerts
• Drug–age/gender interaction alerts

MHS focused on decreasing drug dose alerts, as one of the top two categories, accounting for 12% of all medication alerts. After analysis, the consensus of the multidisciplinary team was to turn off all minimum drug dose alerts and all incomplete information of drug dose alerts, to increase the maximum single-dose drug dose alert threshold to 125%, to increase the daily dose of maximum drug dose alert threshold to 125%, to increase the dose frequency of drug dose alert threshold to more than 2 doses per day above the initial threshold, and to make drug-specific changes to the top 22 offending medications.[6]

CH's strategies reflected an effort to address alerts from multiple categories. Initially turning on all alerts allowed the team to discover where and when the majority of alerts were fired, and to make system-level changes. [Table 2] shows baseline rates of drug alerting for CH across their drug alerting categories. Next, CH suppressed undetermined severity and moderate interaction across all types of drug alerts. This was determined based on the pharmacy and therapeutics committee's consensus that these alert types have low clinical significance. Additionally, CH suppressed all alerts for drug dose daily exceeded, daily as needed exceeded, and frequency exceeded. This step again was done based on committee feedback that these alerts provide few clinical benefits.

Table 2 Baseline CaroMont Health alert rates by category

Alert type	Drug alerts (n)	Drug alerts %	Alert/100 orders
Duplicate therapy	63,305	35	21
Drug–drug	53,650	30	17
Drug allergy (active & inactive ingredients)	20,979	12	7
Duplicate medication order	20,119	11	7
Dose	14,199	8	5
Lactation	5,258	3	2
Pregnancy	1603	1	1
TPN	134	0	0
Total	179,247	100	58

Abbreviation: TPN, total parenteral nutrition.

Alerts were further optimized by turning off duplicate therapy alerts, duplicate medication alerts, drug–drug alerts within order sets, and turning off alerts between transitions of care (such as preoperative to postoperative phase). This intervention was taken based on presence of previously established order sets, which go through robust governance that includes pharmacists and physicians. If similar or duplicate therapies are within an order set, their inclusion is designed to be intentional. We identified phase of care as another high-impact area as multiple similar drugs (antiemetics, pain, etc.) are used in the operating room and again on floor, but they are not concurrently given. Transition to the next phase in care would stop most of these drugs as active orders and it was considered safe to turn them off in this setting.

During the initial analysis of alert frequency, the CH team discovered that nearly every order for calcium gluconate or iron sucrose injection generated an alert due to differences in standard institutional dosing compared with the standard in the drug database. This led to a one-off optimization for these drugs and the disabling of these alerts.

As part of this comparative study, the Chief Medical Informatics officers (coauthors) from the two institutions agreed upon generalizable strategies that each health care system utilized to improve their drug alerts that can be adopted by others.

This study was not submitted for Institutional Review Board approval at either institution because it did not contain any patient-specific data, only drug alert metadata.

Results

MHS reported 1,189,445 drug orders placed between December 2016 and February 2017. During this period, 465,286 drug alerts fired, resulting in an alert rate of approximately 39%. MHS initially focused on decreasing drug dose alerts, which accounted for 12% of all medication alerts. Through system-level strategies, along with drug-specific changes to the top 22 offending medications, the number of drug dose alerts decreased to approximately 3% of all medication alerts ([Table 3]). Standard EHR reports allowed comparison of these strategies with those implemented by CH.

Table 3 The MetroHealth System (MHS) and CaroMont Health (CH) postintervention alert rates by category postoptimization

Alert type	CH alerts (n)	CH alerts/100 orders	MHS alerts (n)	MHS alerts/100 orders
Duplicate therapy	55,685	19	22,392	2
Drug–drug	45,336	15	207,998	17
Drug allergy	20,646	7	72,986	6
Duplicate medication order	18,887	6	99,791	8
Drug–disease	N/A	N/A	17,492	2
Dose	7,768	3	41,004	3
Lactation	109	0.1	1	0
Pregnancy	199	0.1	1,747	0
TPN	131	0.1	1,975	0
Total alerts	148,763	51	465,386	38

Abbreviation: TPN, total parenteral nutrition.

During the 3-month preoptimization period at CH, 179,247 drug alerts fired, resulting in an alert rate of approximately 58% ([Table 2]). Suppressing undetermined severity, moderate interaction, within order sets, and transitions of care for duplicate therapy alerts reduced the alerts from 21 to 19 alerts/100 orders. Additionally, suppressing dose alerts types drug dose daily exceeded, daily as needed exceeded, frequency exceeded, and top two drugs calcium gluconate and iron sucrose injection reduced alerts from 7 to 5 alerts/100 orders. Further turning off drug–drug alerts within order sets and turning off alerts between transitions of care (such as preoperative to postoperative phase) reduced alerts from 17 to 15 alerts/100 orders. Overall, CH reduced drug alerts from 58 to 51% or 179,247 to 148,763. Thus, 9%, or 30,484, unfiltered drug alerts were removed over the 3-month studied period. Specifically, the rate of firing of drug dose alerts was reduced from 5% of all medication alerts to 3%. Interestingly, the baseline drug dose alert rate was initially lower for CH than for MHS (12% at MHS, 5% at CH). [Table 3] shows the postoptimization alert levels at both organizations. The alert categories that remained high despite configuration modifications include drug–drug interactions, drug allergy alerts, and duplicate medication order alerts. These areas generated the most drug alerts in both health care systems regardless of the drug database vendor ([Table 3]). [Table 4] shows the summary of the generalizable approaches to drug alerting shared by the two health care systems.

Table 4 Drug alert approaches shared by the two health care systems

MetroHealth System

• Turn off minimum drug dose and incomplete information drug dose alerts

• Increase maximum single dose alert threshold to 125%

• Increase daily dose maximum drug dose threshold to 125%

• Increase dose frequency threshold to more than 2 doses per day

• Adjust drug-specific alerts to top 22 offending medications

CaroMont Health

• Turn off undetermined and moderate severity across all drug alert types

• Turn off drug dose daily exceeded

• Turn off daily as needed dose exceeded

• Turn off dose frequency exceeded

• Turn off alerts within order sets for duplicate therapy, duplicate medication, drug–drug alerts

• Turn off alerts between transitions of care

• Adjust drug-specific alerts for top 2 offending medications

Discussion

Various methods and strategies can be deployed to make significant changes to reduce the number of unnecessary medication alerts provided to the end user. Many of the methods described here are system-level, global changes that include strategies such as suppressing categories of alerts that may prove to be irrelevant or lacking enough data on the clinical relevance of the alert, modifying alerts for medications included in electronic order sets, and using clinical consensus to modify the doses of drugs allowed by the system. Our conclusion that alerts within certain scenarios, such as transfers between operating rooms and inpatient units, or within order sets are reasonable to suppress, is based on clinician and pharmacy consensus. In addition, using drug-specific strategies can further optimize the drug alerts presented to end users. A recent study in a pediatric inpatient setting used similar strategies by evaluating compliance rates and turning off underdosing alerts.[8] Past studies have shown that improving the specificity and relevance of alerts and their timing can improve patient safety.[3] [5]

We believe our methodology and recommendations can and should be replicated by any health care system that has implemented drug alerts, regardless of the length of time they have had an integrated EHR. The first step is to choose a trusted, regularly updated drug knowledge database. The drug information provided by the vendor is the foundation for the design of CDS and the alerting structure. However, commercial vendor systems have been shown to have much higher rates of both alerts and overrides as compared with homegrown systems.[9] Relying on manual updates or not receiving timely database updates, could lead to outdated or incorrect information being used by the alerting systems, which could cause inadvertent medical errors.[10] Most organizations rely on providers to assess alerts for accuracy, but feedback indicating an “inappropriate alert” is not routinely identified or used by providers.[11] In a retrospective study at a pediatric hospital medical center, fewer alerts (49 alerts/day/per provider) were associated with increasing the number of prescribers taking corrective action on the order that generated an alert.[12]

Next, a health care system needs to have a strategy to consider all categories of drug alerts available. While some generalized principles of CDS can be easily applies to drug alerting, some strategies for drug alerting CDS are unique.[13] [14] Ignoring categories or simply turning on all alerts, as is commonly done due to resource constraints, without further investigation, may lead to alert fatigue and make the health care system prone to medical error. Studies have shown alert workload may be contributing to provider burnout and the need for organizational level intervention.[15] It is also prudent to turn on all, or many, of the drug alerts “silently” at first, to gather data about the frequency of alerts, the type of alerts, and the possible clinical significance of override. A team consisting of pharmacists, physicians, informaticists, and executive leadership should review this data prior to initiating any changes. EHR vendors should be encouraged to produce standard reports of drug alerts by category to facilitate comparison amongst implementations of a specific EHR vendor, and even potentially across institutions with different EHR vendors. Standard drug alerting reports allow for easy tracking of drug alerting within a health care system over time and help to visualize the effect of changes. Most organizations rely on providers to give feedback on alert accuracy. This was shown to not be routinely used and is perhaps less robust for inappropriate medication alerts.[11]

Finally, we recommend that drug CDS be a process of continuous review and improvement. This should include feedback not only from those reviewing the reporting data, but also from end users, who may offer valuable suggestions for improving the clinical utility of the alerts presented to them. Careful review and continuous modification of medication decision support has the potential to improve alert fatigue and patient safety in health care systems. These recommendations for drug alerts are summarized in [Table 5]. We developed these principles based on the strategies and lessons learned at the two health care systems. Both health care systems used some of the strategies but not all to deploy their own optimization plans.

Limitations of this study include the examination of only one EHR vendor rather than multiple EHR vendors. We also are limited by not having baseline alert data from MHS across all alert categories. Our override rates were high, > 90% of times for medication alerts, but we did not evaluate the direct impact of reducing the alert burden on override rate. We believe compliance with clinically significant alerts will increase when insignificant alerts are suppressed, however, future studies should focus on the highest alert categories, such as drug–drug interactions, and analyze possible additional modifications that could be made to them to make them more effective. Further research and comparisons are needed to continue refining and comparing the variety of strategies, EHR vendors, and drug database vendors to ensure that drug alerts presented to end users are relevant, do not contribute significantly to alert fatigue, and improve patient safety. We hope future artificial intelligence and machine learning algorithms can make this analysis more robust.

Table 5 Summary of drug clinical decision support strategic recommendations

Strategies for drug alert implementation and optimization: Lessons from two health care systems
• Team - Need integrated team with a leader, physician/physician informatics, and pharmacy/pharmacy informaticists to design and maintain drug alerting
• Drug information vendor - Need trusted drug information reference source that is updated regularly and automatically
• Comprehensive strategy - Need strategy for all drug CDS categories provided by EHR in use
• Balance risk/benefit of alerts - Focus on “potentially very significant patient harm” alerting
• Silent alerting - Turn on drug CDS silently to evaluate before turning on to prescribers/end users
• Reporting - Utilize a standard reporting to analyze alerting for your system (and possibly to compare across systems)
• Systemic and drug-specific tactics - Focus initially on systemic approaches and second on targeted, limited drug records
• Continual refinement - Drug alerting decision support should have ongoing analysis, end-user feedback, and continuous improvement[16]

Abbreviations: CDS, clinical decision support; EHR, electronic health record.

Conclusion

This study is one of the first cross-organizational studies looking at all types of drug alerts and comparing drug alert rates between two health care systems using standard reports. Further, we looked across the continuum of care, including outpatient, emergency department, inpatient, and operating room settings, and across all ages of patients and all adult and pediatric medical and surgical specialties. It is unique in that the two health care systems used different drug database vendors. This investigation also demonstrates the utility of standard reporting when comparing two systems with the same EHR. Hopefully, future work in this area will develop agreed upon standards for drug alerting so that drug alerts can be built and maintained more efficiently and effectively such that across different health care systems drug alerts become fewer and homogeneous, while improving patient care.

Clinical Relevance Statement

Implementation of robust CDS requires analysis and strategies to optimize drug alert rates across all categories. We identify strategies to optimize drug alerts. With our enhanced configurations of drug alerting, rates dropped to 38 per 100 and 51 per 100 drug orders, respectively. We identify eight principles for implementation of drug alerting that can be adopted by health care systems.

Multiple Choice Questions

1. To implement clinical decision support drug alerts, which of the following must be considered before implementation?

   • Evaluate number of orders placed in organization.

   • Number of physicians in an organization.

   • Review of override rates.

   • Strategy to evaluate and turn off alerts.

   Corrrect Answer: The correct answer is option d. A strategy on evaluation of alerts and postimplementation optimization is to turn off is needed.

2. Which drug alerts cause the most alerts to fire?

   • Drug–drug alerts.

   • Drug dose alerts.

   • Duplicate therapy.

   • a and b.

   • a and c.

   Correct Answer: The correct answer is option c. Drug dose alerts cause fewer alerts than drug–drug and duplicate therapy.

Conflict of Interest

None declared.

Authors' Contributions

All authors made substantial contributions to the manuscript. S.S. and K.D. served as the lead authors, conducting data analysis and leading manuscript preparation and writing. D.K. provided substantial guidance, feedback, and edits during the research and edit process. All authors have approved this work.

Protection of Human and Animal subjects

No human subjects were involved in the project.

• References

• 1 Bates DW, Leape LL, Cullen DJ. , et al. Effect of computerized physician order entry and a team intervention on prevention of serious medication errors. JAMA 1998; 280 (15) 1311-1316
• 2 Nanji KC, Seger DL, Slight SP. , et al. Medication-related clinical decision support alert overrides in inpatients. J Am Med Inform Assoc 2018; 25 (05) 476-481
• 3 Nanji KC, Slight SP, Seger DL. , et al. Overrides of medication-related clinical decision support alerts in outpatients. J Am Med Inform Assoc 2014; 21 (03) 487-491
• 4 Leapfrog Group. Castlight Health. Preventing medication errors in hospitals. 2016 . Available at: http://www.leapfroggroup.org/sites/default/files/Files/Leapfrog-Castlight%20Medication%20Safety%20Report.pdf . Accessed February 28, 2019
• 5 Payne TH, Hines LE, Chan RC. , et al. Recommendations to improve the usability of drug-drug interaction clinical decision support alerts. J Am Med Inform Assoc 2015; 22 (06) 1243-1250
• 6 Saiyed SM, Greco PJ, Fernandes G, Kaelber DC. Optimizing drug-dose alerts using commercial software throughout an integrated health care system. J Am Med Inform Assoc 2017; 24 (06) 1149-1154
• 7 McEvoy DS, Sittig DF, Hickman TT. , et al. Variation in high-priority drug-drug interaction alerts across institutions and electronic health records. J Am Med Inform Assoc 2017; 24 (02) 331-338
• 8 Scharnweber C, Lau BD, Mollenkopf N, Thiemann DR, Veltri MA, Lehmann CU. Evaluation of medication dose alerts in pediatric inpatients. Int J Med Inform 2013; 82 (08) 676-683
• 9 Wong A, Wright A, Seger DL, Amato MG, Fiskio JM, Bates D. Comparison of overridden medication-related clinical decision support in the intensive care unit between a commercial system and a legacy system. Appl Clin Inform 2017; 8 (03) 866-879
• 10 van der Sijs H, Aarts J, Vulto A, Berg M. Overriding of drug safety alerts in computerized physician order entry. J Am Med Inform Assoc 2006; 13 (02) 138-147
• 11 Rehr CA, Wong A, Seger DL, Bates DW. Determining inappropriate medication alerts from “inaccurate warning” overrides in the intensive care unit. Appl Clin Inform 2018; 9 (02) 268-274
• 12 Dexheimer JW, Kirkendall ES, Kouril M. , et al. The effects of medication alerts on prescriber response in a pediatric hospital. Appl Clin Inform 2017; 8 (02) 491-501
• 13 Sittig DF, Singh H. Electronic health records and national patient-safety goals. N Engl J Med 2012; 367 (19) 1854-1860
• 14 Lee J, Han H, Ock M, Lee SI, Lee S, Jo MW. Impact of a clinical decision support system for high-alert medications on the prevention of prescription errors. Int J Med Inform 2014; 83 (12) 929-940
• 15 Gregory ME, Russo E, Singh H. Electronic health record alert-related workload as a predictor of burnout in primary care providers. Appl Clin Inform 2017; 8 (03) 686-697
• 16 Osheroff JA, Teich JM, Middleton B, Steen EB, Wright A, Detmer DE. A roadmap for national action on clinical decision support. [published correction appears in J Am Med Inform Assoc. 2007 May-Jun;14(3):389]. J Am Med Inform Assoc 2007; 14 (02) 141-145