Dashboard Design to Identify and Balance Competing Risk of Multiple Hospital-Acquired Conditions

Abstract

Background Hospital-acquired conditions (HACs) are common, costly, and national patient safety priority. Catheter-associated urinary tract infections (CAUTIs), hospital-acquired pressure injury (HAPI), and falls are common HACs. Clinicians assess each HAC risk independent of other conditions. Prevention strategies often focus on the reduction of a single HAC rather than considering how actions to prevent one condition could have unintended consequences for another HAC.

Objectives The objective of this study is to design an empirical framework to identify, assess, and quantify the risks of multiple HACs (MHACs) related to competing single-HAC interventions.

Methods This study was an Institutional Review Board approved, and the proof of concept study evaluated MHAC Competing Risk Dashboard to enhance clinicians' management combining the risks of CAUTI, HAPI, and falls. The empirical model informing this study focused on the removal of an indwelling urinary catheter to reduce CAUTI, which may impact HAPI and falls. A multisite database was developed to understand and quantify competing risks of HACs; a predictive model dashboard was designed and clinical utility of a high-fidelity dashboard was qualitatively tested. Five hospital systems provided data for the predictive model prototype; three served as sites for testing and feedback on the dashboard design and usefulness. The participatory study design involved think-aloud methods as the clinician explored the dashboard. Individual interviews provided an understanding of clinician's perspective regarding ease of use and utility.

Results Twenty-five clinicians were interviewed. Clinicians favored a dashboard gauge design composed of green, yellow, and red segments to depict MHAC risk associated with the removal of an indwelling urinary catheter to reduce CAUTI and possible adverse effects on HAPI and falls.

Conclusion Participants endorsed the utility of a visual dashboard guiding clinical decisions for MHAC risks preferring common stoplight color understanding. Clinicians did not want mandatory alerts for tool integration into the electronic health record. More research is needed to understand MHAC and tools to guide clinician decisions.

Keywords

clinical decision support - testing and feedback - alerts - monitor and manage - inpatient records - dashboard - pressure ulcer - catheter-associated urinary tract infections - patient harm

Background and Significance

Hospital-acquired conditions (HACs) are common, costly, and a national patient safety priority. The passage of the Patient Protection and Affordable Care Act of 2010 (https://www.govinfo.gov/content/pkg/PLAW-111publ148/pdf/PLAW-111publ148.pdf) heightened the focus on patient safety using multifaceted strategies and continued research efforts to reduce patient harm. The Agency for Healthcare Research and Quality (AHRQ) reported a 17% decline in HACs from 2010 to 2014 and a subsequent 8% decrease in overall HACs from 2014 to 2016 preventing an estimated 8,000 deaths and saving $2.9 billion in this 2-year time frame.[1] [2] The Centers for Medicare and Medicaid Services (CMSs) set a goal of reducing HACs by 20% by 2019. The most common and costly HACs considered largely preventable include catheter-associated urinary tract infections (CAUTIs), stage 3 or 4 hospital-acquired pressure injuries (HAPIs), and patient falls.[3] [4]

Prevention strategies often focus on the reduction of a single HAC rather than considering how actions to prevent one condition could have unintended consequences for another HAC. Shared risk factors of advanced age, mobility constraints, and physical and cognitive impairments can increase the possibility of a patient developing multiple HACs (MHACs).[5] Prevention of CAUTI, HAPI, and falls are also nurse-sensitive indicators of high-quality care and frequently the nurse will assess the risk of each using individual tools. Conflicting prevention strategies across MHACs leave clinicians with challenging decisions to promote overall safety. For instance, removal of the indwelling urinary catheter (IUC) reduces the risk of CAUTI, but most falls are reported when the patient is ambulating to the bathroom and incontinence increases the risk of HAPI.[5] [6] [7] Appreciating that focusing on individual HACs, rather than collective risks associated with MHACs, can fail to effectively address the overall safety needs of different patients.[8] Understanding the interrelationship between IUC removal and trade-offs of unintended consequences of this action within the context of each patient assessment is needed to further advance HAC reduction and patient harm.

The complexity of health care data and workflow associated with the electronic health records (EHRs) creates challenges within and among clinicians[9] and makes it difficult to see a clear profile of the patient's interrelated risks.[7] [10] [11] Comprehensive assessment of patient MHAC risks from single HAC information is not ideal because it requires navigation across multiple EHR screens and does not identify competing prevention strategies. Additionally, EHR navigation and displays of HAC risk assessment may be presented differently based on the user's health care role. This unnecessary navigation of the EHR increases a clinician's cognitive load, reduces efficiency, creates fatigue, and increases the risk of errors.[12] Goals of clinical decision support tools are to increase patient safety, quality, and efficiency of care, patient engagement, care coordination, interprofessional communication, and improve clinician efficiency.[12] [13] [14] The most commonly cited frustrations reported by clinicians with EHR use are poor workflow designs, navigation to multiple screens, and alert fatigue.[12] [13] [15]

In principle, dashboards provide data visualization to facilitate clinician decisions.[14] There is a lack of development and evaluation of dashboard tools that clinicians could use to monitor and manage competing risks of MHACs to reduce patient harm.[10] [11] Understanding tradeoffs related to the clinical assessment and interventions focused on the prevention of one condition without considering the entire clinical context for each HAC risk for a patient presents an opportunity to understand how a dashboard might inform clinicians' understanding of risks of MHACs related to competing intervention strategies.

Objectives

The aim of this study was to develop a high-level design of an empirical framework to identify, assess, and quantify the risks of MHACs related to competing interventions to reduce a single HAC. A proof-of-concept design tested the development of an MHAC Competing Risk Dashboard to enhance clinicians' management of MHACs by understanding the competing risks of CAUTI, HAPI, and falls in hospitalized patients. This paper provides the final visualization of a possible dashboard that could facilitate clinician understanding of MHACs to drive intervention decisions.

Methods

The empirical framework of the clinician's decision to remove the IUC and the possible interaction between CAUTI prevention (i.e., IUC removal) and development of HAPI and/or fall is depicted in [Fig. 1].[7] A transition probability model (TPM)[7] [16] was used to provide a comprehensive framework to assess the potential tradeoffs in the risks of MHACs related to a clinical decision, to remove an IUC in this study. The model examined the likelihood of an event (i.e., experiencing one of these three HACs) while an IUC was in place and after its removal, accounting for the potential changes in the probabilities of an event related to the length of time an IUC has been in place, as well as the length of time since its removal (i.e., duration dependence).[7] The 3-day time frame was chosen based on clinical criteria to reduce CAUTI risk by IUC removal as the central decision that may generate a competing risk of HAPI or fall.

Fig. 1 Model of the interaction between CAUTI, HAPI, and falls.[7] The decision to remove the IUC can adversely affect the patient's risk of falls and/or development of pressure injury. Maintaining urinary continence, reducing moisture to prevent pressure injury, and eliminating mobility challenges to prevent falls have interconnected interventions. Understanding the impact of one clinical decision, such as removing an IUC, should consider the influence that decision may have adversely impacting other hospital acquired conditions. CAUTI, Catheter-associated urinary tract infections; HAPI, hospital-acquired pressure injury; IUC, indwelling urinary catheter.

This study was conducted in four parts: (1) developing a multisite database to understand tradeoffs and competing risks of HACs (CAUTI, HAPI, and falls), (2) specifying and estimating the TPM, (3) designing a dashboard, and (4) testing clinical utility of a high-fidelity dashboard that depicts multiple competing risks of hypothetical hospitalized patients.

Part I. A robust multisite database was developed using retrospective EHR data, incident report data, and data gathered from structured phone interviews with quality department leaders. The five participating hospital systems represented 20 hospitals located in the southern and western regions of the United States. Hospitals ranged in size from 20 to over 500 beds. One-on-one dialogues with quality department leaders were conducted to understand EHR documentation, risk assessment tools used for falls and HAPI, and incident reporting system processes. National Healthcare Safety Network reporting information was obtained for each hospital system. Deidentified data from inpatient records for adult patients (age greater than 18 years) with IUC present at some time during hospitalization that occurred from October 1, 2015 to September 30, 2016. The International Classification of Diseases 10th Revision codes were used to categorize comorbidities in patient records with the Charlson Comorbidity Index.[7]

Part II. The TPM model was based on multi-site discrete time-to-event models that examined the time of the occurrence of an event and, given an event has occurred, the specific type of an event. This framework builds on the concept of latent failure times[17] and has been used to analyze time to event and competing risks, as well as multistate duration data, in a variety of health and human service settings.[16] [18] [19] The discrete-time TPM used a patient-day as the unit of time and one definition of a time to an event was the number of consecutive patient-days with an IUC and a second was the number of consecutive patient-days without an IUC following its removal. For both definitions, the competing events were (1) the occurrence of a CAUTI/UTI, (2) the occurrence of a HAPI, and (3) the occurrence of a fall. For the first definition, removal of an IUC for at least 1 patient-day was considered a censoring event, and for the second definition, reinsertion of an IUC for at least 1 patient-day was considered a censoring event. In both definitions, hospital discharges were defined as a censoring event.

The discrete-time TPM examined the likelihood of an event (experiencing one of these three HACs) while an IUC was in place and after its removal including the possibility that the probabilities of an event are related to the length of time an IUC has been in place, as well as the length of time since its removal, which is referred to as duration dependence. This framework also accounted for differences in these competing risks across patients with a variety of characteristics, including whether an IUC was present on admission, comorbidities, demographics (age, sex, race/ethnicity, and primary insurance), source of admission, and assessments of fall risk and HAPI risk, all of which are correlated with the risk of HACs.[20] [21] [22] A binary logistic specification was used for the probability of an event occurring on each patient-day and, given an event occurred on a patient-day, a multinomial logistic specification was used for the probability of each HAC. These logistic specifications provided a flexible empirical framework that accounted for the complex interactions between the timing of the decision to remove or reinsert an IUC and the occurrence of HACs. General estimation equations in SAS v9.4 were used to estimate the TPM specifications.

The point estimates of the logistic specifications provided the information needed to develop predictions of the patient-specific probabilities that were needed to assess the extent to which there are competing risks and the development of the MHAC. Competing Risk Dashboard, specifically the MHAC Competing Risk Dashboard, considers a patient with a given set of characteristics that has had an IUC for several consecutive days and predicts the probability of each HAC for 3 consecutive days first assuming that the IUC remains in place for all 3 days and then assuming that the IUC is removed and is not reinserted for the same 3 consecutive days. These predicted probabilities provided the information presented in the high-fidelity dashboard.

Part III. Designing the dashboard included the engagement of clinician end-users from hospitals not involved in providing data for the TPM model. Clinicians from two hospitals in the western United States engaged in focus groups to review low-fidelity clinical dashboard displays of MHACs. Participatory design methods[10] [23] were used to engage clinicians in the evaluation of preferred dashboard features. Five low-fidelity dashboard designs were shared during focus groups consisting of diverse clinicians (e.g., physicians, nurse managers, and bedside nurses). Participants were asked to think about the relative risks of HACs and identify preferred ways of visualizing competing HACs for clinical decisions using the prototype dashboards. Analysis of this study informed the high-fidelity dashboard prototype design.[23]

Part IV. Using data from the predictive model developed in Part II, the high-fidelity prototype was designed using Excel. To assess usability, six hypothetical scenarios for patients with an IUC in place were developed ([Table 1]). The high-fidelity prototypes used the same gauge design with green, yellow, and red segments, included a text box that summarized patient characteristics, and were designed to be visible on a standard computer display screen. Patient-specific probabilities pertaining to the decision to remove or retain the IUC were displayed. Clinician participants were able to view patient-specific predicted probabilities of experiencing CAUTI, HAPI, or falls over a 3-day time frame under two conditions: (1) retention of IUC for 3 additional patient-days and (2) removal of IUC (with no reinsertion) for the following 3 patient-days.

Table 1 Hypothetical patient vignettes

Case 1	85-year-old Hispanic female admitted from a skilled nursing facility, not in the ICU. Braden score was 14, and she is assessed to be at risk for falls and has three comorbid conditions. Insurance coverage is Medicare.
Case 2	85-year-old Hispanic male admitted from skilled nursing facility, not in the ICU. Braden score was 14, and he is assessed to be at risk for falls and has three comorbid conditions. Insurance coverage is Medicare.
Case 3	55-year-old White non-Hispanic female admitted from the community, not in the ICU. Braden score was 19, and she is assessed to be at risk for falls and has two comorbidities. Her insurance coverage is commercial.
Case 4	55-year-old Black non-Hispanic male admitted from the community, in the ICU. An IUC is present on admission. Braden score was 9, he is assessed to be a falls risk and has six comorbid conditions. His insurance coverage is Medicaid.
Case 5	55-year-old Black non-Hispanic female admitted from the community, not in the ICU. Braden score on was 19, with no falls risk and 0 comorbidities. Her insurance coverage is Medicaid.
Case 6	75-year-old White non-Hispanic male admitted from another facility, not in the ICU. Braden score was 15, and he was assessed to be at risk for falls and has three comorbidities. His insurance coverage is commercial.

Abbreviations: ICU, intensive care unit; IUC, indwelling urinary catheter.

Investigators applied think-aloud methods held in 1:1 interviews with individual clinicians (physicians, advanced practice nurses and nurse managers, and bedside nurses) from three of the hospital systems that provided EHR data in Part I. The intent was to observe participants' exploration and usability of the high-fidelity dashboard using a think-aloud protocol with cognitive walk-throughs and exploratory questions. Think-aloud methods[11] [23] [24] [25] involved inviting the participants to talk aloud about their experiences while performing a task with a designed artifact and to narrate how they interpreted and would use the MHAC dashboard prototype.

The dashboard prototype allowed end users to view patient-specific predicted probabilities of experiencing CAUTI, HAPI, or falls over a 3-day time frame under two conditions: (1) an IUC was inserted for a specific number of days and remained inserted for 3 additional patient-days or (2) an IUC was inserted for a specified number of days and was removed and not reinserted for the following 3 patient-days. The patient-specific probabilities built in the high-fidelity prototype dashboard was developed around the decision to remove or retain the IUC. Initially, the dashboard graphic was evenly divided into green, yellow, and red colors. However, iterative feedback from participants from site one and two indicated a preference for a display that focused on “staying in the green zone” as a priority for clinical decisions. The subsequent division of dashboard colors was such that green and red regions were smaller with the yellow region being larger. We also showed a dashboard in which the green section was small, and yellow and red regions were larger. The white gauge indicator of risk of CAUTI, HAPI, and falls was dynamically displayed. Small numbers of HAC occurrences required a larger yellow field for participants to visualize the dynamic interactions between CAUTI, HAPI, and fall risk based on the decision to retain or remove the IUC. The white gauge indicator was dynamic as were the predicted risk numbers on the gauge image across the 3-day time frame determined by the predicted probabilities for each case scenario. Three variants of the high-fidelity tool were introduced to the participant using a computer interface (see [Figs. 2],[3],[4]).

Fig. 2 High-fidelity prototype option 1: overall risk gauge and without predictive risk numbers on the gauges.

Fig. 3 High-fidelity prototype option 2: overall risk gauge removed.

Fig. 4 High-fidelity prototype option 3: overall risk gauge removed and predicted risk placed on outer rim of each gauge.

After reviewing the purpose of the study and confirming verbal consent, the participants were given an overview of dashboard tool and then asked to explore and interpret the tool. Two researchers conducted this phase of the study: one facilitated the review and use of the Excel-based dashboard tool and the second recorded notes. All interviews were audio recorded and used the same data collection protocol ([Table 2]). Each interview lasted less than 60 minutes. Each participant was shown a total of three hypothetical patient vignettes (see [Table 1]) and was asked to engage with the dashboard prototype applying interpretation to different patient characteristics. The user could navigate between day 1, 2 and 3, to see adjusted predicted probability of risk of CAUTI, HAPI, or falls for the patient-day selected. An interview guide was used to ask a series of questions after using the dashboard tool. Scripted questions were asked to gain an understanding of the perceived usefulness of the MHAC dashboard tool, value in practice, utility within an EHR, and to seek suggested changes or improvements to the prototype. Additional questions focused on the individual's cognitive walk through of the vignette and dashboard design and explored how the participant might use the dashboard in practice.

Table 2 Think-aloud interview guide

We are interested in your opinions about the information presented and visual displays of data for patient care. We'll start with a few questions about yourself and then ask you to view visual displays while you consider three different patient vignettes. This study assumes you are in your current role assessing patients or in a past role when you assess patients.

These visual displays are intended to help with decision-making related to competing risks of HAPU, CAUTI, and falls. This is not a test of you or your knowledge of patient care. We are looking for feedback on how we can change information presented and visual displays to be more helpful and user friendly.

Let's start with a few questions about yourself:

Intro/Demographics

1. What is your position? How long have you worked in this position here and/or at other facilities?

2. What is your background in terms of your training?

3. Do you use the EHR system personally? For how long?

4. How often do you use the EHR system? How comfortable do you feel?

5. Do you use personally us a tablet/smart phone in your work? For how long?

6. How often do you use a tablet/smart phone in your work? How comfortable do you feel?

Vignettes - Review 3

Prompt: Did you notice changes in HAPI risk was lower and falls higher if you took IUC out (versus leaving it in) on Day 1, but by Day 3 HAPI and fall risk were much more likely when you take the ICU out it out. Was the trade off in risk clear to use using the dashboard?

Prompt: Are the predicted risk numbers on the dashboard gauges informative? Intuitive?

Prompt: Is the overall predictive number clinically useful? Intuitive?

Follow-up questions after cognitive walk through/think-aloud session for vignettes

1. Questions for individual dashboard:

a. Were there parts of the dashboards that worked well?

i. What made those features stand out for you?

b. What parts of the dashboard did not work well and why?

c. Do you think such a dashboard is intuitive?

i. Why or why not?

d. Would you see value in having such dashboards available to you to monitor your patient's risk of complications/hospital acquired conditions and in making care-related decisions?

i. Why or why not?

e. Do you have any further comments to make about this dashboard?

f. Do you see any competing risks for this patient?

i. If so, how would you use this information in thinking about clinical decisions?

ii. What protocols/interventions do those risks prompt you to think about implementing?

2. Now that we have looked at these prototypes individually, we would like your opinions about them:

a. Do you see any value in having such a dashboards to inform your decision-making with regard to patient care?

b. Do you see any value in having such a dashboard during discussions with nurses?

i. Physicians?

ii. With patients?

iii. With family members?

c. Which of these dashboards would be least useful to you and why?

d. What elements make the dashboard easier to understand? (cues: colors, text, trends).

e. Is there value in being able to see changes in patient risks over time visually using a graph?

i. Is there value in being alerted to such changes?

f. Would you prefer to have the information come to you directly based on alert thresholds (push) OR would you prefer to access the information yourself at your discretion (pull)?

g. What would be your preferred way to access such information? (Prompt: Would you prefer integrated into the EHR, as an email, accessible with login to a secure Website, as a paper printout, mobile application on tablet/smartphone).

h. How might these dashboard fit into your workflow?

i. Are there any additional ideas or thoughts that you want to share?

Abbreviations: CAUTI, Catheter-associated urinary tract infections; EHR, electronic health record; HAC, hospital-acquired conditions; HAPI, hospital-acquired pressure injury; HAPU, hospital-acquired pressure ulcers.

Following an iterative user-centered qualitative approach, small modifications were made to the high-fidelity prototype based on reviewer feedback prior to meeting with participants from subsequent hospitals. For example, one participant from the second hospital site mentioned a color vision deficiency stating that yellow was difficult to discern. Thus, at the third and final site, during the summary feedback session, participants were shown static documents that were versions of the gauge dashboard tool using green, orange, and red color segments. Participants were also shown a binary, green, and red gauges to address concerns with color vision deficits along with gauges in which the color segments were varied in size, with green being smaller and yellow and red equally divided.

Data from each hospital site think-aloud session were iteratively analyzed using inductive processes and transcript review to gain an understanding of participants' ability to navigate the dashboard. Recordings from each site were transcribed verbatim, and data from all transcripts were analyzed by three researchers independently using the framework method of qualitative analysis.[24] [26]

Results

Part I: Predictive models of competing risks were developed and used to test the usefulness of the high-fidelity predictive MHAC dashboards. These models were developed using data from a total of 67,631 admissions of patients with an IUC. This translated to 461,145 patient-days being used in the analysis. Across the hospitals, multiple fall assessment tools were reported, IUC days were tracked for CAUTI risk, and all sites assessed HAPI risk with the Braden score.

Part II: TPM[7] [16] [17] [18] [19] [27] was used as the framework to assess potential tradeoffs of risks of MHACs related to the clinical decision to remove the IUC. Among the six hypothetical patient vignettes ([Table 1]) developed using the TPM framework, each had varying characteristics and competing risks for MHACs. Age, admission from the community, number of comorbid conditions, and payer source influenced the predictive probability of MHAC for when an IUC was retained or removed. However, data obtained from the combined five hospital systems showed a very low HAC reoccurrence rate (CAUTI, HAPI, and falls), limiting the predictive probability model beyond the first occurrence.

Part III: Five low fidelity prototypes were shared with targeted clinicians (N = 11). One design that used gauges with the familiar green, yellow, and red (i.e., traffic lights) segments was found to be the preferred visual. Participants suggested additional EHR design elements that would be helpful to the end user including (1) hovering over information in the EHR to understand patient-specific information, (2) predicting risk for multiple days, (3) viewing data of risk prospectively and retrospectively to help inform clinical decisions and “drilling down” for more patient data to understand multiple risks. Reeder et al.[11] provided a full report of this segment of the study and results. Full analysis of findings from the low fidelity focus groups informed the final high-fidelity prototype.

Part IV: The high-fidelity prototype of the MHAC Competing Risks Dashboard was shared at three hospitals that participated in part one of the study. A total of 25 clinicians engaged in the talk-aloud interviews. Individuals from a variety of clinical roles (i.e., physician N = 3, advanced practice nurse and nurse manager N = 8, and bedside nurse N = 14) were the participants engaged in the 1:1 experience with the MHAC Competing Risk Dashboard. Participants quickly adapted to navigating between day 1, 2, and 3 to assess the risk of MHACs. Opinions varied on the knowledge of the overall risk of MHAC (bottom gauge; Fig. 2), contrasted to the gauges displaying the individual risk of CAUTI, HAPI, or fall ([Figs. 3] and [4]). During iterative analysis of interviews with participants from site one and two, it was understood that participants viewed risk assessment within the green segment of the gauge favorably irrespective of the predicted probability of MHAC (outside numbers on gauge) in the yellow and red segments. Thus, with the third and final site, the width from green to yellow to red was altered. Participants made decisions to remove the IUC while weighing the associated risk of HAPI and falls depending on the gauge being within “green” zone or not. Meaning, if the risk of a CAUTI was less (gauge indicator within the green zone) with the IUC removed, the increased risk of HAPI and falls with the removal of IUC was noted, but it did not influence the IUC removal decision if the overall risk of MHAC remained low. Participants noticed the difference in gauge color segment size and verbalized the importance of keeping outcomes in the “green” zone and using data from the overall risk to address HAPI and fall risks. Participants at the third and final site were shown color options other than green, yellow, and red; however, they universally preferred the more common traffic light color segments. Only one participant self-disclosed colorblindness, and we were not able to establish a universal color scheme. The final preferred graphic presentation was the familiar traffic light (green, yellow, and red) color scheme with the predictive values for HAC risk placed on the outside of the gauges and the inclusion of an overall risk gauge ([Fig. 5]).

Fig. 5 High-fidelity prototype: final MHAC high-fidelity prototype: Included external predictive risk and overall risk gauge. MHAC, multiple hospital-acquired conditions.

Audio recordings from the 1:1 interview from 25 participants were transcribed verbatim and analyzed independently by three members of the research team. Using the framework method of qualitive analysis[20] [22] and iterative user-centered informed analysis,[10] [19] 10 themes emerged from the qualitative analysis of participant feedback ([Table 3]). Primary takeaways from the thematic analysis revealed that participants learned to navigate the tool quickly moving from day 1 to 3 to evaluate MHACs potentially influenced by the removal of the IUC. Several participants expressed that they had not considered the adverse impact of removing an IUC on HAPI or fall risk. They expressed that the dashboard showing the associated risks of the MHACs could stimulate clinical conversations about intervention decisions. Clinicians were familiar with the green, yellow, and red traffic light visual and aligned decisions toward staying within green (lower risk) outcomes. Most clinicians expressed that the tool would be helpful with interprofessional communication and could be used as a training tool or inform patients and families about clinical decisions to remove an IUC or not. The dashboard was not demonstrated within an EHR but was an Excel file on a personal computer. This limiting factor made it difficult for participants to articulate where the tool might be most beneficial within an EHR, but most participants expressed favorable use of an MHAC tool. Universally, all participants did not want hard stops or alerts associated with the use of a dashboard clinical decision tool.

Table 3 Themes identified from high-fidelity prototype evaluation sessions

Theme	Trends
Learnability—high-fidelity prototype is easy to learn	All participants expressed that the high-fidelity prototype clinical dashboard was easy to learn. By the third hypothetical patient case, all participants navigated the tool with ease and verbalized greater focus of factors to inform clinical decisions. Participant feedback indicated a high learnability for the prototype design.
Trustworthy information—sources of data for visual displays must be understood to be trustworthy	Most participants asked about the source of the data for the visual displays with some communicated their assumption that it was driven by data from the EHR, such as Braden scale and falls risk assessments. There was broad agreement that the data that drove the visual displays must be understood before the tool itself would be useful, with some participants requesting the ability to see underlying data or documentation of the model. Once sources of data for visual displays were accepted, participants used the tool as a trusted source of information.
Usefulness of tool—useful for clinical increased awareness, decision-making, and communication	All participants stated they found the tool useful. Many commented on the ability for the tool to raise awareness of the interactions of risks that might not be considered in making clinical decisions. There was broad agreement that the tool would be useful to inform clinical decision-making and facilitate communication between members of the care team. There was less alignment in participant feedback about the usefulness of the tool for patient engagement though a few participants felt the visualization of the risks might improve patient engagement in clinical decisions. Nurse managers believed the tool would be useful to support unit-level communications and for use in hand-off report between nurses. Many participants commented on the usefulness of the tool for new clinical team members for training purposes.
Usefulness of overall risk display—value of overall likelihood visual display is dependent on patient case	Participants were divided concerning the value of the visual display that included overall likelihood of risk. Some participants found the overall risk gauge to be helpful and others found it to be distracting. Participants voiced the overall risk display was most helpful when the case patient was more complex. If the overall likelihood gauge showed green, broad perceptions were that there was nothing of concern for that patient.
Usefulness of background data—value of patient background data is depended on patient case	The value of displaying patient background and demographic data was questioned by participants and it was often ignored when using the tool. In particular, participants questioned the value of insurer information, ethnicity or admission source and stated that patient background information was probably more relevant for case managers or social workers.
No alerts from tool	Alert fatigue was identified as a barrier to workflow. Accordingly, most participants disfavored “hard stop alerts” that required interaction from the user. Instead, the recommendation was a change in visual display that highlighted changes in patient status.
Location in EHR	Most participants expressed a preference to see the HAC visual displays in their EHR “summary view” as most participants had a custom EHR view and wanted the visual displays available at logon. One nurse participant wanted to see individual risk displays when using individual assessment features in the EHR.
Visual displays of risk proportions inform decision making	All participants noted they used the gauge colors to assign meaning to patient risk and urgency for action. Of note, participants at all three sites noted the larger yellow and red segments in the version of the tool they used but defaulted to green to inform meaning.
Represent risk with traditional green/yellow/red	Participants chose the traffic light green/yellow/red displays over green/orange/red displays.
No binary representation of risk	The two provider participants who evaluated the green/red binary risk visual displays strongly argued against representing patients as “good” or “bad” because patient condition is thought of as a continuum.

Abbreviations: EHR, electronic health record; HAC, hospital-acquired conditions.

Discussion

Numerous barriers exist in providing concise information to guide clinician decisions and interventions to reduce the risk of MHACs.[28] One strategy to overcome these barriers is the application of user-centered design principles to communicate information more effectively about multiple risks; this approach can facilitate a more comprehensive understanding of the interrelatedness of MHACs and competing nature of treating one condition at the risk of adversely impacting a different condition. Health care and MHAC prevention are complex; thus, looking at new ways in which clinicians can address dynamic situations can allow innovations that provide new thinking paradigms that may improve overall safety and performance.[29] Padula et al[8] address the concept of complexity bias in prevention and MHACs suggesting that rather than breaking down interventions based on one condition, clinicians miss the larger opportunity to address overlapping risks and potential patient harm. This research team proposes that when clinicians effectively manage moisture, mobility, nutrition, and CAUTI risk as a whole the overlapping benefits and risks may be more effectively managed. Understanding and quantifying the risks of MHACs related to competing interventions focused on reducing a single HAC in this proof-of-concept study aligns with emerging work associated with complexity science as a means to address strategies for health care improvement.[8] [29]

Dashboard tools are increasingly used to support clinical decisions.[14] [25] Tools that display data across a time spectrum during hospitalization can be helpful in providing a larger understanding of the patient's interrelated clinical needs.[30] The proof of concept design to better understand MHAC explored in this study suggested clinicians value dashboards that are visually easy to interpret. However, the impact of one decision (i.e., removal of an IUC) on another HAC was less obvious. Participants in this study verbalized understanding of MHAC; however, developing a more robust utility of clinical decision tools is needed. Using direct observation and think-aloud techniques provided a useful understanding of potential future development of MHAC dashboard tools in which the displays are intuitive and informative.

There were several limitations identified in this study. First, within the retrospective dataset used to create the predictive model, the HACs were relatively rare events. Only 1.86% of admissions where the patient had an IUC for at least 1 day experienced any of the three HACs analyzed in the data obtained from five hospital systems covering 67,631 patient admissions. As such, a dataset with a larger number of hospital admissions is needed to substantiate the extent to which there are competing risks and tradeoffs in the likelihood of MHACs related to a clinical decision focused on managing a single HAC. Second, the analysis is limited to the first occurrence of any one of the three HACs and a better understanding of the sequencing of HAC events relative to other HAC events is needed to illustrate the role of MHAC competing risks in clinical decision-making. Third, this study considered only three HACs and it is likely that there are additional HACs that are adversely impacted by addressing one condition over another. Fourth, the study did not use a standard system usability scale to evaluate participant engagement. Fifth, initial colors for the traffic light were informed by participants in the third part of the study.[11] Elements of color blindness for clinicians that may use the study were identified but not further developed in this proof of concept study. Further research is needed developing dashboard tools that provide meaningful color schemes addressing color blindness yet align with familiar symbols such as traffic light colors. Finally, there is a lack of validated thresholds for the probability of an HAC that are needed to determine the clinically relevant values at which the dashboard gauges change from green to yellow and then from yellow to red in the MHAC Competing Risk Dashboard.

Conclusion

Providing a visual display of completing risk using a dashboard shows promise in guiding clinical assessment of the interrelated risks of HACs. A familiar dashboard design inclusive of a common color code for visual and numeric risk assessment, tradeoffs related to interventions, and guidance for clinical decisions to reduce patient harm appeared easy for clinicians to interpret but did not necessarily support the understanding of a decision to reduce CAUTI risk and the subsequent impact on HAPI or fall risk. The MHAC Competing Risk Dashboard can raise awareness of the interrelatedness of risk and interventions and provide a communication channel across clinicians and with patients and families. More research is needed to understand the competing risks of MHACs, how interventions to address one condition may adversely impact other outcomes, and to determine the effective ways to incorporate decision support tools to facilitate clinician care.

Clinical Relevance Statement

Further research and development are needed to validate the predictive model and create user-friendly dashboards that can be used within the EHR to inform clinical decisions associated with competing risks for MHACs. Well-designed dashboard tools can facilitate clinicians' assessments and interventions to prevent CAUTI, HAPI, and falls and avoid conflicting prevention strategies to reduce MHACs and promote overall patient safety.

Multiple Choice Questions

1. The HACs—CAUTI, HAPI, and falls—were selected for inclusion because these HACs:

   • Rarely occur

   • Are most common and costly

   • Are easiest to quantify

   • Are prevented with single interventions

   Correct Answer: The correct answer is option b. The most common and costly HACs include CAUTI, HAPI, and falls. All three HACs have overlapping risk factors.

2. The study sought to enhance clinicians' management of MHACs by underscoring the competing risks through

   • Care coordination and attention to avoidable hospital readmissions

   • Engagement of patients in decision-making

   • Providing a high-fidelity dashboard depicting competing risks

   • Using the Braden score for all patients

   Correct Answer: The correct answer is option c. Risk assessment tools often occur independent of each HAC. Using a high-fidelity dashboard was tested to allow the visualization of risk assessments to better inform clinician's interventions/decisions.

3. The results showed which of the following:

   • Improved understanding of competing risks for CAUTI, HAPI, and falls

   • Clinician participants favored a familiar visual dashboard to reduce patient harm

   • User-centered design principles resulted in a more effective presentation of clinical decision choices

   • All of these

   Correct Answer: The correct answer is option d. Results from this study found that a visual dashboard improved clinician understanding of the overlapping risks of CAUTI, HAPI, and falls and could better guide intervention decisions related to patient's MHAC risks.

Conflict of Interest

None declared.

Acknowledgment

We are very grateful to Peter Ferrarone, MA, MSW for his assistance in organizing interview logistics for this study.

Note

The findings and conclusions in this document are those of the author(s) who are responsible for the content and do not necessarily represent the views of AHRQ. No statement in this document should be construed as an official position of AHRQ or of the U.S. Department of Health and Human Services. Identifiable information on which this publication is based is protected by federal law, Section 934(c) of the Public Health Service Act, 42 USC 299c-3(c). No identifiable information about any individuals or entities supplying the information or described in it may be knowingly used except in accordance with their prior consent. Any confidential identifiable information in this publication that is knowingly disclosed is disclosed solely for the purpose for which it was provided. C.D.M. was a PhD student at the University of Colorado School of Public Health throughout the study period. She is now employed by IBM Watson Health, Ann Arbor, MI.

Protection of Human and Animal Subjects

All study procedures were approved by an academic institutional review board, the Colorado Multiple Institutional Review Board, (#16-2020) Aurora, CO. Study participants provided verbal informed consent and received a $50 gift card incentive for participation.

• References

• 1 Agency for Healthcare Research and Quality. Saving lives and saving money: hospital-acquired conditions update. Published November 2015. Updated January 2018. Accessed September 30, 2021. https://www.ahrq.gov/professionals/quality-patient-safety/pfp/2014-final.html
• 2 Agency for Healthcare Research and Quality. Declines in hospital-acquired conditions save 8,000 lives and $2.9 billion in costs. Published June 5, 2018. Accessed September 30, 2021. https://www.ahrq.gov/news/newsroom/press-releases/declines-in-hacs.html
• 3 Hamadi H, Tafili A, Apatu E, Park S, Spaulding A. Medicare' hospital-acquired condition reduction program and community diversity in the United States: the need to account for racial and ethnic segregation. Hosp Top 2019; 97 (04) 148-155
• 4 Sankaran R, Gulseren B, Nuliyalu U. et al. A comparison of estimated cost savings from potential reductions in hospital-acquired conditions to levied penalties under the CMS hospital-acquired condition reduction program. Jt Comm J Qual Patient Saf 2020; 46 (08) 438-447 [published correction appears in Jt Comm J Qual Patient Saf 2021 Jan 19]
• 5 Manojlovich M, Lee S, Lauseng D. A systematic review of the unintended consequences of clinical interventions to reduce adverse outcomes. J Patient Saf 2016; 12 (04) 173-179
• 6 Oman KS, Makic MB, Fink R. et al. Nurse-directed interventions to reduce catheter-associated urinary tract infections. Am J Infect Control 2012; 40 (06) 548-553
• 7 Gritz RM, Wald H, Makic MB. et al. Identifying, Assessing, and Balancing Competing Risks of Multiple Hospital-Acquired Conditions (HACs). Report to US Department of Health and Human Services, Agency for Healthcare Research and Quality; September 2018 Contract HHSP2332015000251 and HHSP23337003T
• 8 Padula WV, Armstrong DG, Goldman DP. Complexity bias in the prevention of iatrogenic injury: why specific harms may inhibit performance. Mayo Clin Proc 2022; 97 (02) 221-224
• 9 Senathirajah Y, Kaufman DR, Cato KD, Borycki EM, Fawcett JA, Kushniruk AW. Characterizing and visualizing display and task fragmentation in the electronic health record: mixed methods design. JMIR Human Factors 2020; 7 (04) e18484
• 10 Wald H, Richard A, Dickson VV, Capezuti E. Chief nursing officers' perspectives on Medicare's hospital-acquired conditions non-payment policy: implications for policy design and implementation. Implement Sci 2012; 7: 78
• 11 Reeder B, Makic MBF, Morrow C. et al. Design and evaluation of low-fidelity visual display prototypes for multiple hospital-acquired conditions. Comput Inform Nurs 2020; 38 (11) 562-571
• 12 Roman LC, Ancker JS, Johnson SB, Senathirajah Y. Navigation in the electronic health record: a review of the safety and usability literature. J Biomed Inform 2017; 67: 69-79
• 13 Mills S. Electronic health records and use of clinical decision support. Crit Care Nurs Clin North Am 2019; 31 (02) 125-131
• 14 Fazaeli S, Khodaveisi T, Vakilzadeh AK. et al. Development, implementation, and user evaluation of COVID-19 dashbaord in a third-level hospital in Iran. Appl Clin Inform 2021; 12 (05) 1091-1100
• 15 Nimjee T, Miller E, Solomon S. Exploring generational differences in physicians' perspectives on the proliferation of technology within the medical field: a narrative study. Healthc Q 2020; 23 (SP): 53-59
• 16 MaCurdy T, Gritz RM. Measuring the influence of unemployment insurance on unemployment experiences. J Bus Econ Stat 1997; 15 (02) 130-152
• 17 Prentice RL, Kalbfleisch JD, Peterson Jr AV, Flournoy N, Farewell VT, Breslow NE. The analysis of failure times in the presence of competing risks. Biometrics 1978; 34 (04) 541-554
• 18 Gritz RM, Theobald ND. The effects of school district spending priorities on length of stay in teaching. J Hum Resour 1996; 31 (03) 477-512
• 19 Theobald ND, Gritz RM. The effects of school district spending priorities on the exit paths of beginning teachers leaving a district. Econ Educ Rev 1996; 15 (01) 11-22
• 20 Shashikumar SA, Waken RJ, Luke AA, Nerenz DR, Joynt Maddox KE. Association of stratification by proportion of patients dually enrolled in Medicare and Medicaid with financial penalties in the hospital-acquired condition reduction program. JAMA Intern Med 2021; 181 (03) 330-338
• 21 Zogg CK, Thumma JR, Ryan AM, Dimick JB. Medicare's hospital acquired condition reduction program disproportionately affects minority-serving hospitals: variation by race, socioeconomic status, and disproportionate share hospital payment receipt. Ann Surg 2020; 271 (06) 985-993
• 22 Buntin MB, Ayanian JZ. Social risk factors and equity in Medicare payment. N Engl J Med 2017; 376 (06) 507-510
• 23 Jeffery AD, Novak LL, Kennedy B, Dietrich MS, Mion LC. Participatory design of probability-based decision support tools for in-hospital nurses. J Am Med Inform Assoc 2017; 24 (06) 1102-1110
• 24 Jaspers MW, Steen T, van den Bos C, Geenen M. The think aloud method: a guide to user interface design. Int J Med Inform 2004; 73 (11-12): 781-795
• 25 Richter Lagha R, Burningham Z, Sauer BC. et al. Usability testing a potentially inappropriate medication dashboard: a core component of the dashboard development process. Appl Clin Inform 2020; 11 (04) 528-534
• 26 Kilsdonk E, Peute LW, Jaspers MW. Factors influencing implementation success of guideline-based clinical decision support systems: a systematic review and gaps analysis. Int J Med Inform 2017; 98: 56-64
• 27 Gale NK, Heath G, Cameron E, Rashid S, Redwood S. Using the framework method for the analysis of qualitative data in multi-disciplinary health research. BMC Med Res Methodol 2013; 13: 117
• 28 Bersani K, Fuller TE, Garabedian P. et al. Use, perceived usability, and barriers to implementation of a patient safety dashboard integrated within a vendor EHR. Appl Clin Inform 2020; 11 (01) 34-45
• 29 Braithwaite J. Changing how we think about healthcare improvement. BMJ 2018; 361: k2014
• 30 Nelson O, Sturgis B, Gilbert K. et al. A visual analytics dashboard to summarize serial anesthesia records in pediatric radiation treatment. Appl Clin Inform 2019; 10 (04) 563-569

Publication History

Received: 05 December 2021

Accepted: 27 April 2022

Article published online:
08 June 2022