Subscribe to RSS
Download PDF
DOI: 10.1055/s-0044-1782604
A Usability Survey of a Quality Improvement Data Visualization Tool among Medical Intensive Care Unit Nurses
Authors
Funding A.J.B. is an iTHRIV Scholar. The iTHRIV Scholars Program is supported in part by the National Center for Advancing Translational Sciences of the National Institutes of Health under Award numbers UL1TR003015 and KL2TR003016.
Abstract
Background Cognitive overload is prevalent among intensive care unit (ICU) clinicians. Data visualization may decrease cognitive load by assisting with data interpretation and task prioritization. We developed the Bundle Board to display real-time data from the electronic medical record (EMR), highlighting opportunities for action in standardized ICU patient care. This study evaluates the practical usability of this data visualization tool among nurses in the ICU.
Methods The tool is offered as an application separate from the EMR and was available in the medical ICU for 8 months before we surveyed unit nursing staff. To evaluate usability of the tool, we adapted the Health-Information Technology Usability Scale and included an option to provide open-ended feedback. Survey data were analyzed using quantitative and qualitative methods.
Results ICU nurses were invited to participate through email and verbal announcements. Of the potential participants, 38% (N = 47) responded. The survey demonstrated that the tool was perceived as usable. For each subscale, mean scores were as follows: Perceived Ease of Use 4.40, Impact 4.14, User Control 4.07, and Perceived Usefulness 3.61. There were no significant differences between core and contracted nurses or after stratifying by duration of Bundle Board use. Fifteen respondents completed the optional free-text portion of the survey. Qualitative analysis revealed six subthemes focusing on perceived impacts on quality and safety, cognitive burden and workload, and emotional impact of the Bundle Board.
Conclusion The Bundle Board demonstrated good usability among ICU nurses, who provided substantive feedback for its improvement. These observations may be generalizable to other comparable interventions. Iterative feedback from end users is vital to developing and implementing a digital health intervention. Our study provides a framework for performing a usability analysis within a specific clinician population and environment.
Keywords
data visualization - intensive and critical care - interfaces and usability - dashboard - quality* Co-first authors.
Publication History
Received: 24 May 2023
Accepted: 30 January 2024
Article published online:
05 April 2024
© 2024. The Author(s). This is an open access article published by Thieme under the terms of the Creative Commons Attribution License, permitting unrestricted use, distribution, and reproduction so long as the original work is properly cited. (https://creativecommons.org/licenses/by/4.0/)
Georg Thieme Verlag KG
Rüdigerstraße 14, 70469 Stuttgart, Germany
-
References
- 1 Pickering BW, Herasevich V, Ahmed A, Gajic O. Novel representation of clinical information in the ICU: developing user interfaces which reduce information overload. Appl Clin Inform 2010; 1 (02) 116-131
- 2 Patel V, Buchman T. Cognitive overload in the ICU. Patient Safety Network. August 21, 2016. Accessed December 20, 2023 at: https://psnet.ahrq.gov/web-mm/cognitive-overload-icu
- 3 Moacdieh N, Sarter N. Clutter in electronic medical records: examining its performance and attentional costs using eye tracking. Hum Factors 2015; 57 (04) 591-606
- 4 Kutney-Lee A, Brooks Carthon M, Sloane DM, Bowles KH, McHugh MD, Aiken LH. Electronic health record usability: associations with nurse and patient outcomes in hospitals. Med Care 2021; 59 (07) 625-631
- 5 Moy AJ, Hobensack M, Marshall K. et al. Understanding the perceived role of electronic health records and workflow fragmentation on clinician documentation burden in emergency departments. J Am Med Inform Assoc 2023; 30 (05) 797-808
- 6 Zhao JY, Kessler EG, Guo WA. Interprofessional communication goes up when the electronic health record goes down. J Surg Educ 2019; 76 (02) 512-518
- 7 Borgert MJ, Goossens A, Dongelmans DA. What are effective strategies for the implementation of care bundles on ICUs: a systematic review. Implement Sci 2015; 10: 119
- 8 Ely EW. The ABCDEF bundle: science and philosophy of how ICU liberation serves patients and families. Crit Care Med 2017; 45 (02) 321-330
- 9 Hales B, Terblanche M, Fowler R, Sibbald W. Development of medical checklists for improved quality of patient care. Int J Qual Health Care 2008; 20 (01) 22-30
- 10 Fuller TE, Garabedian PM, Lemonias DP. et al. Assessing the cognitive and work load of an inpatient safety dashboard in the context of opioid management. Appl Ergon 2020; 85: 103047
- 11 Khairat SS, Dukkipati A, Lauria HA, Bice T, Travers D, Carson SS. The Impact of visualization dashboards on quality of care and clinician satisfaction: integrative literature review. JMIR Human Factors 2018; 5 (02) e22
- 12 Lin YL, Trbovich P, Kolodzey L, Nickel C, Guerguerian AM. Association of data integration technologies with intensive care clinician performance: a systematic review and meta-analysis. JAMA Netw Open 2019; 2 (05) e194392
- 13 Chemparathy A, Seneviratne MG, Ward A. et al. Development and implementation of a real-time bundle-adherence dashboard for central line-associated bloodstream infections. Pediatr Qual Saf 2021; 6 (04) e431
- 14 Davis CL, Bjoring M, Hursh J. et al. The ICU bundle board: a novel real-time data visualization tool to improve maintenance care for invasive catheters. Appl Clin Inform 2023; 14 (05) 892-902
- 15 Pageler NM, Longhurst CA, Wood M. et al. Use of electronic medical record-enhanced checklist and electronic dashboard to decrease CLABSIs. Pediatrics 2014; 133 (03) e738-e746
- 16 Shaw SJ, Jacobs B, Stockwell DC, Futterman C, Spaeder MC. Effect of a real-time pediatric ICU safety bundle dashboard on quality improvement measures. Jt Comm J Qual Patient Saf 2015; 41 (09) 414-420
- 17 Lai CH, Li KW, Hu FW. et al. Integration of an intensive care unit visualization dashboard (i-Dashboard) as a platform to facilitate multidisciplinary rounds: cluster-randomized controlled trial. J Med Internet Res 2022; 24 (05) e35981
- 18 Khasnabish S, Burns Z, Couch M, Mullin M, Newmark R, Dykes PC. Best practices for data visualization: creating and evaluating a report for an evidence-based fall prevention program. J Am Med Inform Assoc 2020; 27 (02) 308-314
- 19 Yen PY, Bakken S. Review of health information technology usability study methodologies. J Am Med Inform Assoc 2012; 19 (03) 413-422
- 20 Liu K, Nakamura K, Katsukawa H. et al. Implementation of the ABCDEF bundle for critically ill ICU patients during the COVID-19 pandemic: a multi-national 1-day point prevalence study. Front Med (Lausanne) 2021; 8: 735860
- 21 International Organization of Standardization. Ergonomics of Human-System Interaction—Part 11: Usability: Definitions and Concepts; 2018. Accessed January 1, 2023 at: https://www.iso.org/obp/ui/#iso:std:iso:9241:-11:ed-2:v1:en
- 22 Vincent JL, Moreno R, Takala J. et al. The SOFA (Sepsis-related Organ Failure Assessment) score to describe organ dysfunction/failure. On behalf of the Working Group on Sepsis-Related Problems of the European Society of Intensive Care Medicine. Intensive Care Med 1996; 22 (07) 707-710
- 23 Yen PY, Wantland D, Bakken S. Development of a customizable Health IT Usability Evaluation Scale. AMIA Annu Symp Proc 2010; 2010: 917-921
- 24 Yen PY, Sousa KH, Bakken S. Examining construct and predictive validity of the Health-IT Usability Evaluation Scale: confirmatory factor analysis and structural equation modeling results. J Am Med Inform Assoc 2014; 21 (e2): e241-e248
- 25 Bersani K, Fuller TE, Garabedian P. et al. Use, perceived usability, and barriers to implementation of a patient safety dashboard integrated within a vendor EHR. Appl Clin Inform 2020; 11 (01) 34-45
- 26 Mlaver E, Schnipper JL, Boxer RB. et al. User-centered collaborative design and development of an inpatient safety dashboard. Jt Comm J Qual Patient Saf 2017; 43 (12) 676-685
- 27 R Studio Team. R Studio. 2022 . Accessed March 1, 2023 at: http://www.rstudio.com/
- 28 Dedoose. 2023 . March 1, 2023 at: www.dedoose.com
- 29 Hennink M, Hutter I, Bailey A. Qualitative Research Methods. 1st ed.. Sage; 2010
- 30 Schall Jr MC, Cullen L, Pennathur P, Chen H, Burrell K, Matthews G. Usability evaluation and implementation of a health information technology dashboard of evidence-based quality indicators. Comput Inform Nurs 2017; 35 (06) 281-288
- 31 Effken JA, Loeb RG, Kang Y, Lin ZC. Clinical information displays to improve ICU outcomes. Int J Med Inform 2008; 77 (11) 765-777
- 32 Sittig DF, Singh H. A new sociotechnical model for studying health information technology in complex adaptive healthcare systems. Qual Saf Health Care 2010; 19 (Suppl 3, Suppl 3): i68-i74
- 33 Wisner K, Lyndon A, Chesla CA. The electronic health record's impact on nurses' cognitive work: an integrative review. Int J Nurs Stud 2019; 94: 74-84
- 34 Coblis—Color Blindness Simulator – Colblindor. Accessed September 1, 2023 at: https://www.color-blindness.com/coblis-color-blindness-simulator/
- 35 Meyer VM, Benjamens S, Moumni ME, Lange JFM, Pol RA. Global overview of response rates in patient and health care professional surveys in surgery: a systematic review. Ann Surg 2022; 275 (01) e75-e81
- 36 Loh KP, Liu J, Ganzhorn S, Sanabria G, Schnall R. Establishing a usability cut-point for the health information technology usability evaluation scale (Health-ITUES). Int J Med Inform 2022; 160: 104713
- 37 Elias J, Moy A, Lucas E. et al. Assessing clinical staff usability & satisfaction with documentation & information retrieval prior to an electronic health record implementation. Podium Presentation presented at: AMIA Annual Symposium; 2020 ; Virtual. Accessed May 6, 2023 at: https://knowledge.amia.org/72332-amia-1.4602255/t004-1.4605866/t004-1.4605867/3414558-1.4606120/3414558-1.4606121?timeStamp=1683400189035#tab_0=1
- 38 Campbell CM, Prapanjaroensin A, Anusiewicz CV, Baernholdt M, Jones T, Patrician PA. Variables associated with missed nursing care in Alabama: a cross-sectional analysis. J Nurs Manag 2020; 28 (08) 2174-2184
- 39 Kuniecki M, Pilarczyk J, Wichary S. The color red attracts attention in an emotional context. An ERP study. Front Hum Neurosci 2015; 9: 212
- 40 Nielsen J. Usability inspection methods. In: Conference Companion on Human Factors in Computing Systems—CHI '94. ACM Press; 1994: 413-414
- 41 Turner P, Kushniruk A, Nohr C. Are we there yet? human factors knowledge and health information technology - the challenges of implementation and impact. Yearb Med Inform 2017; 26 (01) 84-91
- 42 Carayon P, Hoonakker P. Human factors and usability for health information technology: old and new challenges. Yearb Med Inform 2019; 28 (01) 71-77
- 43 Kushniruk AW, Patel VL. Cognitive and usability engineering methods for the evaluation of clinical information systems. J Biomed Inform 2004; 37 (01) 56-76