Pediatric Provider Utilization of a Clinical Decision Support Alert and Association with HIV Pre-exposure Prophylaxis Prescription Rates

Abstract

Objectives An electronic clinical decision support (CDS) alert can provide real-time provider support to offer pre-exposure prophylaxis (PrEP) to youth at risk for human immunodeficiency virus (HIV). The purpose of this study was to evaluate provider utilization of a PrEP CDS alert in a large academic-community pediatric network and assess the association of the alert with PrEP prescribing rates.

Methods HIV test orders were altered for patients 13 years and older to include a hard-stop prompt asking if the patient would benefit from PrEP. If providers answered “Yes” or “Not Sure,” the CDS alert launched with options to open a standardized order set, refer to an internal PrEP specialist, and/or receive an education module. We analyzed provider utilization using a frequency analysis. The rate of new PrEP prescriptions for 1 year after CDS alert implementation was compared with the year prior using Fisher's exact test.

Results Of the 56 providers exposed to the CDS alert, 70% (n = 39) responded “Not sure” to the alert prompt asking if their patient would benefit from PrEP, and 54% (n = 30) chose at least one clinical support tool. The PrEP prescribing rate increased from 2.3 prescriptions per 10,000 patients to 6.6 prescriptions per 10,000 patients in the year post-intervention (p = 0.02).

Conclusion Our findings suggest a knowledge gap among pediatric providers in identifying patients who would benefit from PrEP. A hard-stop prompt within an HIV test order that offers CDS and provider education might be an effective tool to increase PrEP prescribing among pediatric providers.

Keywords

clinical decision support - electronic health records - adolescents - CDS alert - order set - health care system

Background and Significance

In 2018, 41% of new human immunodeficiency virus (HIV) diagnoses in the United States were among adolescents and young adults under the age of 30 years, with over half of those occurring in youth under the age of 25 years.[1] Similarly, in California, 40% of new HIV cases were in individuals under 30 years with half occurring in youth under the age of 25 and disproportionately affecting sexual, gender, and racial/ethnic minorities.[2]

HIV pre-exposure prophylaxis (PrEP) has been shown to be highly effective for preventing HIV, reducing the risk of sexual transmission by approximately 99% when taken daily.[3] Since the United States Food and Drug Administration (FDA) approved PrEP for adults in 2012, epidemiological evidence has shown an association between increases in PrEP uptake and significantly lower HIV incidence in the United States.[4] In May 2018, the FDA approved PrEP for adolescents.[5] At this time in California, adolescents represent the group with the greatest unmet PrEP need, calculated as the lowest ratio of PrEP users to new HIV diagnoses.[6] Compared with all age groups in 2018, youth aged 16 to 24 years had the lowest PrEP coverage rate, with only 11.4% of persons with indications for HIV prophylaxis being prescribed PrEP.[7]

Lack of provider training and education about PrEP has been repeatedly identified as one of the biggest barriers to prescribing.[8] [9] [10] [11] In one study surveying primary care providers, only one-third of providers reported any HIV-related training.[8] Studies have also found additional provider- and systems-level barriers to adolescent PrEP prescribing including: insufficient time and staff required for PrEP initiation and follow-up, lack of access to PrEP experts, lack of supportive electronic health record (EHR) workflows, confidentiality concerns, and cost and insurance coverage.[8] [9] [10] [11] However, studies have found that even minimal on-the-job training can help overcome knowledge and self-efficacy barriers.[12] [13] Current literature has identified the need for more interventions to address these barriers.[8] [9] [11]

Previous studies evaluating the effect of an EHR-based clinical decision support (CDS) alert on provider practices have demonstrated mixed results.[14] [15] [16] [17] [18] [19] [20] [21] [22] [23] [24] Studies have previously shown both evidence of electronic alerts impacting clinician practice,[16] [17] [18] [19] [20] [23] [24] as well as unsuccessfully changing provider behavior.[15] [21] Specific to HIV screening, EHR-based CDS systems have been shown to successfully increase rates of HIV screening in various settings including primary care and emergency medicine.[25] [26] [27] Qualitative data evaluating primary care provider preferences for PrEP-specific support demonstrated that PrEP-inexperienced providers were particularly interested in CDS systems and access to PrEP-experienced providers for expert advice or referrals.[28] To our knowledge, there have been no studies evaluating the use of a CDS alert for PrEP among pediatric and adolescent providers.

Objectives

The purpose of this study was to evaluate provider utilization of a PrEP CDS alert in a large academic-community pediatric network and assess the association of the alert with PrEP prescribing rates. We hypothesized that PrEP prescription rates would increase after implementation of the PrEP CDS alert compared with the same time period prior to alert implementation.

Methods

Setting and Participants

This study was conducted at Stanford Children's Health, which includes Lucile Packard Children's Hospital Stanford and over 60 additional clinical service locations for specialty and general pediatric care. The network includes over 120 pediatric primary care providers serving in 25 clinic locations in the San Francisco Bay Area. All providers within the network use an Epic[29] EHR system for patient care and documentation.

Intervention

The intervention went live December 1, 2019 ([Fig. 1]). We modified outpatient HIV testing orders in Epic[29] for patients 13 years and older to include a hard-stop question that read: “Is the patient eligible for PrEP?” We removed the CDS alert prompt from transplant, obstetrics, and reproductive endocrinology clinics, as HIV testing in these settings were part of routine workups instead of targeted to sexual behaviors.

Fig. 1 Modified workflow for HIV test orders in patients 13 years and older to include PrEP clinical decision support (CDS) alert. CDS, clinical decision support; PrEP, exposure prophylaxis; HIV, human immunodeficiency virus.

After 6 months, we chose to change the hard-stop question to: “Would this patient benefit from PrEP (a safe, daily pill to reduce HIV risk by approximately 99%?)” This change was in response to receiving unnecessary early referrals for patients who were being tested for reasons other than sexual behaviors such as patients receiving care for transplant. The purpose of this change was to provide a definition for “PrEP eligibility” to clinicians who may not have been familiar with PrEP prior to seeing the CDS prompt.

By using a hard-stop question, ordering providers could not continue with their HIV test order without selecting an answer among the given options. Providers could select one out of four possible responses: “Yes,” “No,” “Not sure,” or “Already on PrEP.” If providers selected “No” or “Already on PrEP,” no further action was required upon signing the HIV test order. If providers selected “Yes” or “Not sure,” the CDS would launch. Providers would then be given the option to (1) open a PrEP standardized order set to facilitate ordering the appropriate laboratories, medication, patient education, and follow-up, (2) refer the patient to an internal pool of specialized “PrEP providers,” and/or (3) receive a 15-minute educational module sent directly to their EHR message inbox that could be completed at any time. Providers could also accept the CDS with no action or cancel the CDS without taking any further action.

Outcomes

Provider response to and utilization of the PrEP CDS alert were of primary interest. Therefore, the study assessed the proportion of “Yes” versus “Not sure” answers to the CDS prompt that launched the PrEP CDS. We also quantitatively evaluated provider choice to use the PrEP order set, refer to a PrEP provider, and/or receive the educational module. Because providers may launch the CDS alert on multiple occasions, this study evaluated provider CDS alert response and utilization for first launch for each provider included in the analysis as well as total CDS alert launches.

We also evaluated PrEP prescription rates among primary care pediatric and adolescent providers in the first year following the CDS alert intervention going live compared with the year prior. This was defined as the number of new PrEP prescriptions over the number of distinct patients aged 13 to 25 years seen in primary care clinics during the study time period.

Data Collection

For CDS alert responses and follow-up actions, the unit of analysis was individual CDS alert launches. In some instances, the alert appeared to be firing multiple times in short succession, likely due to providers re-entering or editing HIV orders or attempting to choose different or additional CDS alert follow-up actions. Therefore, we defined duplicates as CDS alert launches that occurred in the same location, from the same provider, and for the same patient within a 24-hour period. When CDS alert launches were duplicated but with different follow-up actions taken in different launches, all the different action choices were aggregated and recorded together under a single CDS alert launch record.

All prescribing data were obtained from the Stanford Research Repository (STARR) utilizing EHR data. For PrEP prescriptions, the unit of analysis was individual outpatient PrEP prescriptions written by providers in primary care settings for patients aged 13 to 25 years between December 1, 2018 and November 30, 2020. PrEP prescriptions that were refills were excluded. For the total number of patients, the unit of analysis was distinct patients aged 13 to 25 years seen at a Stanford Children's primary care site between December 1, 2018 and November 30, 2020.

Statistical Analysis

A frequency analysis was done to evaluate provider response to the hard-stop CDS prompt as well as provider launch of the PrEP order set, referral to a PrEP provider, and/or choice to receive the educational module. This was done for both first launches for distinct providers and total CDS alert launches. Additional frequency analyses were done with first launches to describe any change in provider CDS follow-up actions with duplicate launches and to evaluate changes in provider responses after altering the CDS alert prompt wording.

To calculate the proportion of new PrEP prescriptions to adolescent and young adult primary care patients, a denominator of all distinct primary care patients aged 13 to 25 years in the Stanford Children's Health outpatient network was used. The proportions of new PrEP prescriptions for December 1, 2018 through November 30, 2019 and for December 1, 2019 through November 30, 2020 were compared using Fisher's exact test.

Statistical analyses were done using Stata 15.1. This study was approved by the Stanford University Institutional Review Board. Data were analyzed from December 2020 to January 2021.

Results

Provider CDS Alert Response and Utilization

In the first year of the CDS alert going live, the alert launched 244 times. 117 alerts remained after removing duplicates during the same encounter and erroneous launches occurring outside of pediatric clinics in transplant, obstetrics, and reproductive endocrinology clinics. After excluding 14 alerts launched by non-providers such as registered nurses and laboratory technician staff, 103 alerts were included in the analysis launched by 56 distinct pediatric and adolescent providers.

For the 56 providers exposed to the CDS alert intervention, we analyzed data from their first launch of the alert ([Table 1]). When responding to the alert prompt, 70% (n = 39) of providers indicated that they were not sure if their patient would benefit from PrEP. Of the 56 providers, 54% (n = 30) chose at least one of the follow-up actions with all but one (n = 29) choosing to request the PrEP educational module. Nine percent (n = 5) resulted in providers opening the standardized order set, and 5% (n = 3) resulted in internal referrals to a PrEP specialist. Providers took no action in 38% (n = 21) of encounters, and the alert was canceled in 9% (n = 5) of encounters.

Table 1 First time launch clinical decision support (CDS) alert prompt response and follow-up actions

CDS alert prompt response
“Would this patient benefit from PrEP (a safe, daily pill to reduce HIV risk by approximately 99%)?”
	Total responses (n = 56)	Not sure (n = 39)	Yes (n = 17)
CDS alert selection	n (%)	n (% Total)	n (% Total)
Follow-up action taken[a]	30 (54)	18 (32)	12 (21)
PrEP education module	29 (52)	17 (30)	12 (21)
PrEP order set	5 (9)	0 (0)	5 (9)
Referral to PrEP specialist	3 (5)	3 (5)	0 (0)
No action taken	21 (38)	16 (29)	5 (9)
Cancel CDS alert	5 (9)	5 (9)	0 (0)

Abbreviations: CDS, clinical decision support; HIV, human immunodeficiency virus; PrEP, pre-exposure prophylaxis.

^a Providers can pick more than one follow-up action when choosing the education module, order set, or referral to PrEP specialist.

Among the 56 first launches, in 18 instances, there were duplicate launches by providers for the same encounter. Of these, seven providers (all of whom initially canceled the CDS alert) changed their follow-up actions with one choosing to subsequently refer to a PrEP specialist, one requesting the educational module, and the rest accepting the alert with no further action. In the first 6 months prior to changing the wording of the CDS alert prompt, 32 providers launched the CDS tool, of which 56% (n = 18) responded “Not Sure” and 44% (n = 14) responded “Yes.” After changing the CDS alert prompt wording, 24 providers launched the CDS tool, of which 88% (n = 21) responded “Not Sure” and 12% (n = 3) responded “Yes.”

In analyzing total CDS alert launches, multiple launches from the same providers were included ([Table 2]). When providers responded to the alert prompt, in 58% (n = 60) of alerts, providers indicated that they were not sure if their patient would benefit from PrEP. Of the 103 alerts, 36% (n = 37) resulted in providers requesting the PrEP educational module, 9% (n = 9) resulted in providers opening the standardized order set, and 5% (n = 5) resulted in internal referrals to a PrEP specialist. Providers took no action in 49% (n = 50) of alerts, and the alert was canceled in 10% (n = 10) of instances.

Table 2 Total provider clinical decision support (CDS) alert prompt response and follow-up actions

CDS alert prompt response
“Would this patient benefit from PrEP (a safe, daily pill to reduce HIV risk by approximately 99%)?”
	Total responses (n = 103)	Not sure (n = 60)	Yes (n = 43)
CDS alert selection	n (%)	n (% Total)	n (% Total)
Follow-up action taken[a]	42 (41)	22 (21)	20 (19)
PrEP education module	37 (36)	21 (20)	16 (16)
PrEP order set	9 (9)	0 (0)	9 (9)
Referral to PrEP specialist	5 (5)	4 (4)	1 (1)
No action taken	50 (49)	29 (28)	21 (20)
Cancel CDS alert	10 (10)	8 (8)	2 (2)

Abbreviations: CDS, clinical decision support; HIV, human immunodeficiency virus; PrEP, pre-exposure prophylaxis.

^a Providers can pick more than one follow-up action when choosing the education module, order set, or referral to PrEP specialist.

PrEP Prescribing Rates

During the pre-intervention period from December 1, 2018 through November 30, 2019, there were a total of 30,040 patients aged 13 to 25 years seen at Stanford Children's Health primary care sites. Of these patients, seven were prescribed PrEP (rate of 2.3 new prescriptions per 10,000 patients). During the post-intervention period from December 1, 2019 through November 30, 2020, there were a total of 27,335 patients between 13 and 25 years of age seen at primary care sites. Of these patients, 18 were prescribed PrEP (rate of 6.6 new prescriptions per 10,000 patients, p = 0.02). Among the 18 PrEP initiations, 22% (n = 4) were signed through the CDS PrEP order set, and 67% (n = 12) were prescribed by pediatric providers after being exposed to the CDS alert.

Discussion

This study examined pediatric provider utilization of a CDS tool in the form of a PrEP CDS alert that included an education module, order set guidance, and referral options, and the association of the alert with provider prescribing rates. Among first time users of the CDS alert, over half the providers chose at least one follow-up action, all of which included sending a PrEP education module to their EHR message inbox. Additionally, provider prescribing rates more than doubled in the first year post-intervention compared with 1 year pre-intervention with two-thirds of PrEP prescriptions in the year post-intervention written by providers who had previously seen the CDS alert.

For first-time users of the CDS alert, nearly seven out of every ten providers indicated that they were unsure if their patients were eligible for or would benefit from PrEP. This suggests a knowledge gap among pediatric providers in identifying patients who would benefit from PrEP. This is consistent with previous studies that have found a lack of provider training and education to recommend or prescribe PrEP.[8] [9] [10] [11] A previous study surveying members of an adolescent-focused health professional society demonstrated high awareness of PrEP among adolescent providers.[30] Our study is unique in evaluating PrEP knowledge among pediatric primary care providers and suggests a large PrEP educational need among providers who focus on the care of youth. Our PrEP CDS alert aimed to offer an immediate just-in-time educational opportunity and CDS to pediatric providers who may not be aware of PrEP.

Among first time users of the CDS alert, over half of the providers chose to take one of the follow-up actions, almost all of which included requesting the educational module to be delivered to their EHR message inbox. This suggests that a CDS alert may be an effective way to offer PrEP training and educational opportunities to pediatric providers. Additionally, providers chose to utilize real-time PrEP CDS with either the PrEP standardized order set or a referral to a PrEP specialist in 13% of cases. When evaluating total CDS launches, providers continued to use the CDS tools at the same rate as first time users.

Previous studies have shown that clinicians may experience alert fatigue if they feel inundated with alerts that are of low perceived relevance or significance.[31] [32] [33] [34] In primary care settings, up to 91% of alerts relating to drug safety were overridden by providers.[31] [32] [33] [34] In comparison, our PrEP CDS alert was only canceled 11% of the time among both first time and total encounters and no action was taken in 38 and 49% of first time and total encounters, respectively.

We also examined the association between the CDS alert and PrEP prescription rates among providers. Findings suggest that a hard-stop electronic alert that offers CDS tools and provider education may be effective in increasing PrEP prescribing. This study adds to a body of previous literature that demonstrated mixed outcomes regarding the effectiveness of electronic alerts to impact provider behavior.[14] [15] [16] [17] [18] [19] [20] [21] [22] This is also the first study to evaluate use of a CDS alert for HIV PrEP in a pediatric setting.

There are several limitations to our study that warrant consideration. While we were able to evaluate provider CDS alert actions, this study did not include follow-up data to determine if providers completed their chosen actions (i.e., reviewed the educational module). If providers did not complete their follow-up actions, our study would overestimate true CDS alert utilization. Conversely, we may be underestimating the educational impact of the CDS alert on provider PrEP knowledge, as being exposed to the alert prompt alone may increase PrEP awareness and education even if no follow-up action is taken. By using a pre-intervention time period for historical control, it is feasible that other factors other than the introduction of the PrEP CDS alert contributed to our findings. For example, targeted institutional PrEP education during the post-intervention time period may have impacted PrEP prescribing in addition to the effect of the PrEP CDS alert. There was also a disruption in patient care due to the Severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) pandemic during our study period. Adolescent and young adult visit volume in our pediatric network was lower in the year following the CDS alert going live than in previous years. Prior evidence has shown a significant reduction in PrEP initiation during the pandemic.[35] Therefore, it is possible that there was a similar impact on PrEP prescribing in our study population during this time that would require further study to determine. This study was conducted at a single institution, which may limit generalizability of the findings.

Conclusion

This study suggests that a hard-stop CDS alert prompt within an HIV test order can be a potential tool to offer PrEP education opportunities to pediatric providers and support increased youth PrEP prescribing. Additional studies are needed to evaluate acceptability of this intervention among clinicians. Further studies with longer follow-up should also be done to determine longer term effects on provider prescribing behavior. Our PrEP CDS alert can be applied to other academic and community health care systems to offer an immediate just-in-time educational opportunity and CDS to pediatric providers who may not be aware of PrEP.

Clinical Relevance Statement

Lack of provider training about PrEP is a substantial barrier to prescribing, particularly among pediatric providers. We found that an electronic CDS alert that offers real-time clinician support and education may be an effective way to offer PrEP education to providers and increase prescribing among pediatric providers.

Multiple Choice Questions

1. In this study, what was the most common response to the CDS alert prompt, “Would this patient benefit from PrEP (a safe, daily pill to reduce HIV risk by approximately 99%?)”

   • Yes

   • No

   • Not sure

   • Already on PrEP

   Correct Answer: The correct answer is option c. Among first-time users of the PrEP CDS alert, 70% (n = 39) of providers indicated that they were not sure if their patient would benefit from PrEP. This suggests a knowledge gap among pediatric providers in identifying patients who would benefit from PrEP.

2. Among first-time launches of the PrEP CDS alert in this study, what was the most commonly selected alert follow-up action?

   • Take no action.

   • Open a standardized PrEP order set.

   • Refer to an internal PrEP specialist.

   • Receive an education module.

   Correct Answer: The correct answer is d. In this study, 56 providers were exposed to the CDS alert intervention. When evaluating these providers' first experiences with the CDS alert, 54% (n = 30) resulted in providers choosing at least one of the follow-up actions, all but one of which included requesting the PrEP educational module. Nine percent (n = 5) resulted in providers opening the standardized order set, and 5% (n = 3) resulted in internal referrals to a PrEP specialist. Providers took no action in 38% (n = 21) of encounters, and the alert was canceled in 11% (n = 6) of encounters.

Conflict of Interest

None declared.

Acknowledgments

The authors thank Dr. Kenneth Mayer and Dr. Christopher Sudfeld (Harvard T.H. Chan School of Public Health) for their comments and suggestions on the manuscript.

Protection of Human and Animal Subjects

The study was performed in compliance with the World Medical Association Declaration of Helsinki on Ethical Principles for Medical Research Involving Human Subjects, and was reviewed by Stanford University Institutional Review Board.

• References

• 1 Centers for Disease Control and Prevention. HIV Surveillance Report, 2018 (Preliminary); vol. 30. November 2019. Accessed April 14, 2020 at: http://www.cdc.gov/hiv/library/reports/hiv-surveillance.html
• 2 California Department of Public Health, Office of AIDS, California. HIV Surveillance Report—2018. February 2020. Accessed April 14, 2020 at: https://www.cdph.ca.gov/Programs/CID/DOA/CDPH%20Document%20Library/California_HIV_Surveillance_Report2018.pdf
• 3 Pre-Exposure Prophylaxis (PrEP) | HIV Risk and Prevention | HIV/AIDS | CDC. Accessed April 14, 2020 at: https://www.cdc.gov/hiv/risk/prep/index.html
• 4 PrEP use linked to fewer new HIV infections in US states | aidsmap. Accessed April 14, 2020 at https://www.aidsmap.com/news/jul-2018/prep-use-linked-fewer-new-hiv-infections-us-states
• 5 Gilead Sciences Inc. U.S. Food and Drug Administration Approves Expanded Indication for Truvada® (Emtricitabine and Tenofovir Disoproxil Fumarate) for Reducing the Risk of Acquiring HIV-1 in Adolescents. Published 2018. Accessed September 15, 2021 at: https://www.gilead.com/news-and-press/press-room/press-releases/2018/5/us-food-and-drug-administration-approves-expanded-indication-for-truvada-emtricitabine-and-tenofovir-disoproxil-fumarate-for-reducing-the-risk-of
• 6 California—AIDSVu. Accessed April 14, 2020 at: https://aidsvu.org/local-data/united-states/west/california/
• 7 Harris NS, Johnson AS, Huang YA. et al. Vital signs: status of human immunodeficiency virus testing, viral suppression, and HIV preexposure prophylaxis—United States, 2013-2018. Morb Mortal Wkly Rep 2019; 68 (48) 1117-1123
• 8 Henny KD, Duke CC, Geter A. et al. HIV-related training and correlates of knowledge, HIV screening and prescribing of nPEP and PrEP among primary care providers in Southeast United States, 2017. AIDS Behav 2019; 23 (11) 2926-2935
• 9 Liu A, Cohen S, Follansbee S. et al. Early experiences implementing pre-exposure prophylaxis (PrEP) for HIV prevention in San Francisco. PLoS Med 2014; 11 (03) e1001613
• 10 Mullins TLK, Idoine CR, Zimet GD, Kahn JA. Primary care physician attitudes and intentions toward the use of HIV pre-exposure prophylaxis in adolescents in one metropolitan region. J Adolesc Health 2019; 64 (05) 581-588
• 11 Zhang C, McMahon J, Fiscella K. et al. HIV pre-exposure prophylaxis implementation cascade among health care professionals in the United States: implications from a systematic review and meta-analysis. AIDS Patient Care STDS 2019; 33 (12) 507-527
• 12 Sales JM, Cwiak C, Haddad LB. et al. Brief report: impact of PrEP training for family planning providers on HIV prevention counseling and patient interest in PrEP in Atlanta, Georgia. J Acquir Immune Defic Syndr 2019; 81 (04) 414-418
• 13 Pinto RM, Witte SS, Filippone P, Choi CJ, Wall M. Interprofessional collaboration and on-the-job training improve access to HIV testing, HIV primary care, and pre-exposure prophylaxis (PREP). AIDS Educ Prev 2018; 30 (06) 474-489
• 14 Schedlbauer A, Prasad V, Mulvaney C. et al. What evidence supports the use of computerized alerts and prompts to improve clinicians' prescribing behavior?. J Am Med Inform Assoc 2009; 16 (04) 531-538
• 15 Hansen MJ, Carson PJ, Leedahl DD, Leedahl ND. Failure of a best practice alert to reduce antibiotic prescribing rates for acute sinusitis across an integrated health system in the midwest. J Manag Care Spec Pharm 2018; 24 (02) 154-159
• 16 Devries J, Rafie S, Polston G. Implementing an overdose education and naloxone distribution program in a health system. J Am Pharm Assoc (2003) 2017; 57 (2S): S154-S160
• 17 Malte CA, Berger D, Saxon AJ. et al. Electronic medical record alert associated with reduced opioid and benzodiazepine coprescribing in high-risk veteran patients. Med Care 2018; 56 (02) 171-178
• 18 Strom BL, Schinnar R, Aberra F. et al. Unintended effects of a computerized physician order entry nearly hard-stop alert to prevent a drug interaction: a randomized controlled trial. Arch Intern Med 2010; 170 (17) 1578-1583
• 19 Jaspers MWM, Smeulers M, Vermeulen H, Peute LW. Effects of clinical decision-support systems on practitioner performance and patient outcomes: a synthesis of high-quality systematic review findings. J Am Med Inform Assoc 2011; 18 (03) 327-334
• 20 Kwan JL, Lo L, Ferguson J. et al. Computerised clinical decision support systems and absolute improvements in care: meta-analysis of controlled clinical trials. BMJ 2020; 370: m3216
• 21 Ostropolets A, Zhang L, Hripcsak G. A scoping review of clinical decision support tools that generate new knowledge to support decision making in real time. J Am Med Inform Assoc 2020; 27 (12) 1968-1976
• 22 Oluoch T, Santas X, Kwaro D. et al. The effect of electronic medical record-based clinical decision support on HIV care in resource-constrained settings: a systematic review. Int J Med Inform 2012; 81 (10) e83-e92
• 23 Gibbs KD, Shi Y, Sanders N. et al. Evaluation of a Sepsis Alert in the Pediatric Acute Care Setting. Appl Clin Inform 2021; 12 (03) 469-478
• 24 Friebe MP, LeGrand JR, Shepherd BE, Breeden EA, Nelson SD. Reducing inappropriate outpatient medication prescribing in older adults across electronic health record systems. Appl Clin Inform 2020; 11 (05) 865-872
• 25 Lin J, Mauntel-Medici C, Heinert S, Baghikar S. Harnessing the power of the electronic medical record to facilitate an opt-out HIV screening program in an urban academic Emergency Department. J Public Health Manag Pract 2017; 23 (03) 264-268
• 26 Marcelin JR, Tan EM, Marcelin A. et al. Assessment and improvement of HIV screening rates in a Midwest primary care practice using an electronic clinical decision support system: a quality improvement study. BMC Med Inform Decis Mak 2016; 16 (01) 76
• 27 Ridgway JP, Lee A, Devlin S, Kerman J, Mayampurath A. Machine learning and clinical informatics for improving HIV care continuum outcomes. Curr HIV/AIDS Rep 2021; 18 (03) 229-236
• 28 van den Berg P, Powell VE, Wilson IB, Klompas M, Mayer K, Krakower DS. Primary care providers' perspectives on using automated HIV risk prediction models to identify potential candidates for pre-exposure prophylaxis. AIDS Behav 2021; 25 (11) 3651-3657
• 29 Epic [Computer software]. . Verona, WI: Epic Systems Corporation; 2021
• 30 Hart-Cooper GD, Allen I, Irwin Jr CE, Scott H. Adolescent health providers' willingness to prescribe pre-exposure prophylaxis (PrEP) to youth at risk of HIV infection in the United States. J Adolesc Health 2018; 63 (02) 242-244
• 31 Weingart SN, Toth M, Sands DZ, Aronson MD, Davis RB, Phillips RS. Physicians' decisions to override computerized drug alerts in primary care. Arch Intern Med 2003; 163 (21) 2625-2631
• 32 Payne TH, Nichol WP, Hoey P, Savarino J. Characteristics and override rates of order checks in a practitioner order entry system. Proc AMIA Symp 2002; 602-606
• 33 Isaac T, Weissman JS, Davis RB. et al. Overrides of medication alerts in ambulatory care. Arch Intern Med 2009; 169 (03) 305-311
• 34 Gadhiya K, Zamora E, Saiyed SM, Friedlander D, Kaelber DC. Drug alert experience and salience during medical residency at two healthcare institutions. Appl Clin Inform 2021; 12 (02) 355-361
• 35 Krakower D, Solleveld P, Levine K. et al. Impact of COVID-19 on HIV preexposure prophylaxis care at a Boston community health center. Paper presented at: 23rd International AIDS Conference; virtual; 2020: 176-177

Publication History

Received: 22 July 2021

Accepted: 01 November 2021

Article published online:
12 January 2022