Download PDF
CC BY-NC-ND 4.0 · Appl Clin Inform 2022; 13(04): 961-970
DOI: 10.1055/s-0042-1757553
State of the Art/Best Practice Paper

Developing the VA Geriatric Scholars Programs' Clinical Dashboards Using the PDSA Framework for Quality Improvement

Zachary Burningham
1   Division of Epidemiology, Department of Internal Medicine, University of Utah, Salt Lake City, Utah, United States
2   Informatics, Decision-Enhancement and Analytic Sciences (IDEAS) Centers of Innovation (COIN), Salt Lake City Veterans Affairs Medical Center, Salt Lake City, Utah, United States
,
Regina Richter Lagha
3   Geriatric Research Education and Clinical Center (GRECC), Veterans Affairs Greater Los Angeles Medical Center, Los Angeles, California, United States
,
Brittany Duford-Hutchinson
2   Informatics, Decision-Enhancement and Analytic Sciences (IDEAS) Centers of Innovation (COIN), Salt Lake City Veterans Affairs Medical Center, Salt Lake City, Utah, United States
,
Carol Callaway-Lane
4   Geriatric Research Education and Clinical Center (GRECC), Veterans Affairs Tennessee Valley Health Care System, Murfreesboro, Tennessee, United States
,
Brian C. Sauer
1   Division of Epidemiology, Department of Internal Medicine, University of Utah, Salt Lake City, Utah, United States
2   Informatics, Decision-Enhancement and Analytic Sciences (IDEAS) Centers of Innovation (COIN), Salt Lake City Veterans Affairs Medical Center, Salt Lake City, Utah, United States
,
Ahmad S. Halwani
2   Informatics, Decision-Enhancement and Analytic Sciences (IDEAS) Centers of Innovation (COIN), Salt Lake City Veterans Affairs Medical Center, Salt Lake City, Utah, United States
5   Department of Hematology, University of Utah, Salt Lake City, Utah, United States
,
Jamie Bell
1   Division of Epidemiology, Department of Internal Medicine, University of Utah, Salt Lake City, Utah, United States
2   Informatics, Decision-Enhancement and Analytic Sciences (IDEAS) Centers of Innovation (COIN), Salt Lake City Veterans Affairs Medical Center, Salt Lake City, Utah, United States
,
Tina Huynh
1   Division of Epidemiology, Department of Internal Medicine, University of Utah, Salt Lake City, Utah, United States
2   Informatics, Decision-Enhancement and Analytic Sciences (IDEAS) Centers of Innovation (COIN), Salt Lake City Veterans Affairs Medical Center, Salt Lake City, Utah, United States
,
Joseph R. Douglas
3   Geriatric Research Education and Clinical Center (GRECC), Veterans Affairs Greater Los Angeles Medical Center, Los Angeles, California, United States
,
B. Josea Kramer
3   Geriatric Research Education and Clinical Center (GRECC), Veterans Affairs Greater Los Angeles Medical Center, Los Angeles, California, United States
6   Division of Geriatric Medicine, David Geffen School of Medicine, University of California at Los Angeles, Los Angeles, United States
› Author Affiliations Funding This work was supported by a Promising Practice award from the United States Department of Veterans Affairs Office of Rural Health and by the resources and use of the United States Department of Veterans Affairs facilities at the Greater Los Angeles Healthcare System and the Salt Lake City Health Care System.
› Further Information
Also available at
   Permissions and Reprints

Abstract

Background Involving clinician end users in the development process of clinical dashboards is important to ensure that user needs are adequately met prior to releasing the dashboard for use. The challenge with following this approach is that clinician end users can undergo periodic turnover, meaning, the clinicians that played a role in the initial development process may not be the same individuals that use the dashboard in future.

Objectives Here, we summarize our Plan, Do, Study, Act (PDSA)-guided clinical dashboard development process for the VA Geriatric Scholars Program (GSP) and the value of continuous, iterative development. We summarize dashboard adaptations that resulted from two PDSA cycles of improvement for the potentially inappropriate medication dashboard (PIMD), one of many Geriatric Scholars clinical dashboards. We also present the evaluative performance of the PIMD.

Methods Evaluation of the PIMD was performed using the system usability scale (SUS) and through review of user interaction logs. Routine end users that were Geriatric Scholars and had evidence of 5 or more dashboard views were invited to complete an electronic form that contained the 10-item SUS.

Results The proportion of Geriatric Scholars that utilized the PIMD increased for each iterative dashboard version that was produced as a byproduct from feedback (31.0% in 2017 to 60.2% in 2019). The overall usability of the PIMD among routine users was found to be above average (SUS score: 75.2 [95% CI 70.5–79.8]) in comparison to the recommended standard of acceptability (SUS score: 68)

Conclusion The solicitation of feedback during dashboard orientations led to iterative adaptations of the PIMD that broadened its intended use. The presented PDSA-guided process to clinical dashboard development for the VA GSP can serve as a valuable framework for development teams seeking to produce well-adopted and usable health information technology (IT) innovations.

Keywords

visualization - American Geriatrics Society - clinical dashboard - early-stage usability - dashboard - electronic health record - quality improvement


Publication History

Received: 20 May 2022

Accepted: 05 August 2022

Article published online:
12 October 2022

© 2022. The Author(s). This is an open access article published by Thieme under the terms of the Creative Commons Attribution-NonDerivative-NonCommercial License, permitting copying and reproduction so long as the original work is given appropriate credit. Contents may not be used for commercial purposes, or adapted, remixed, transformed or built upon. (https://creativecommons.org/licenses/by-nc-nd/4.0/)

Georg Thieme Verlag KG
Rüdigerstraße 14, 70469 Stuttgart, Germany

 

  • References

  • 1 Evans RS. Electronic health records: then, now, and in the future. Yearb Med Inform 2016; (Suppl. 01) S48-S61
    PubMedGoogle Scholar
  • 2 Ghazisaeidi M, Safdari R, Torabi M, Mirzaee M, Farzi J, Goodini A. Development of performance dashboards in healthcare sector: key practical issues. Acta Inform Med 2015; 23 (05) 317-321
    CrossrefPubMedGoogle Scholar
  • 3 Lau MK, Bounthavong M, Kay CL, Harvey MA, Christopher MLD. Clinical dashboard development and use for academic detailing in the U.S. Department of Veterans Affairs. J Am Pharm Assoc (Wash DC) 2019; 59 (2S): S96
    CrossrefPubMedGoogle Scholar
  • 4 Randell R, Alvarado N, McVey L. et al. Requirements for a quality dashboard: lessons from national clinical audits. AMIA Annu Symp Proc 2020; 2019: 735-744
    PubMedGoogle Scholar
  • 5 Burningham Z, Jackson GL, Kelleher J. et al. The enhancing quality of prescribing practices for older veterans discharged from the emergency department (EQUIPPED) potentially inappropriate medication dashboard: a suitable alternative to the in-person academic detailing and standardized feedback reports of traditional EQUIPPED?. Clin Ther 2020; 42 (04) 573-582
    CrossrefPubMedGoogle Scholar
  • 6 Dowding D, Merrill J, Russell D. Using feedback intervention theory to guide clinical dashboard design. AMIA Annu Symp Proc 2018; 2018: 395-403
    PubMedGoogle Scholar
  • 7 Twohig PA, Rivington JR, Gunzler D, Daprano J, Margolius D. Clinician dashboard views and improvement in preventative health outcome measures: a retrospective analysis. BMC Health Serv Res 2019; 19 (01) 475
    CrossrefPubMedGoogle Scholar
  • 8 Koopman RJ, Kochendorfer KM, Moore JL. et al. A diabetes dashboard and physician efficiency and accuracy in accessing data needed for high-quality diabetes care. Ann Fam Med 2011; 9 (05) 398-405
    CrossrefPubMedGoogle Scholar
  • 9 Fischer MJ, Kourany WM, Sovern K. et al. Development, implementation and user experience of the Veterans Health Administration (VHA) dialysis dashboard. BMC Nephrol 2020; 21 (01) 136
    CrossrefPubMedGoogle Scholar
  • 10 Greco KJ, Rao N, Urman RD, Brovman EY. A dashboard for tracking mortality after cardiac surgery using a national administrative database. Cardiol Res 2021; 12 (02) 86-90
    CrossrefPubMedGoogle Scholar
  • 11 Makic MBF, Stevens KR, Gritz RM. et al. Dashboard design to identify and balance competing risk of multiple hospital-acquired conditions. Appl Clin Inform 2022; 13 (03) 621-631
    Article in Thieme ConnectPubMedGoogle Scholar
  • 12 Bersani K, Fuller TE, Garabedian P. et al. Use, perceived usability, and barriers to implementation of a patient safety dashboard integrated within a vendor EHR. Appl Clin Inform 2020; 11 (01) 34-45
    Article in Thieme ConnectPubMedGoogle Scholar
  • 13 Ivers NM, Sales A, Colquhoun H. et al. No more ‘business as usual’ with audit and feedback interventions: towards an agenda for a reinvigorated intervention. Implement Sci 2014; 9: 14
    CrossrefPubMedGoogle Scholar
  • 14 Foster M, Albanese C, Chen Q. et al. Heart failure dashboard design and validation to improve care of veterans. Appl Clin Inform 2020; 11 (01) 153-159
    Article in Thieme ConnectPubMedGoogle Scholar
  • 15 Hartzler AL, Chaudhuri S, Fey BC, Flum DR, Lavallee D. Integrating patient-reported outcomes into spine surgical care through visual dashboards: lessons learned from human-centered design. EGEMS (Wash DC) 2015; 3 (02) 1133
    PubMedGoogle Scholar
  • 16 Murphy DR, Savoy A, Satterly T, Sittig DF, Singh H. Dashboards for visual display of patient safety data: a systematic review. BMJ Health Care Inform 2021; 28 (01) e100437
    CrossrefPubMedGoogle Scholar
  • 17 Sittig DF, Singh H. A new sociotechnical model for studying health information technology in complex adaptive healthcare systems. Qual Saf Health Care 2010; 19 (Suppl. 03) i68-i74
    CrossrefPubMedGoogle Scholar
  • 18 Risling TL, Risling DE. Advancing nursing participation in user-centred design. J Res Nurs 2020; 25 (03) 226-238
    CrossrefPubMedGoogle Scholar
  • 19 Kramer BJ, Creekmur B, Howe JL. et al. Veterans affairs geriatric scholars program: enhancing existing primary care clinician skills in caring for older veterans. J Am Geriatr Soc 2016; 64 (11) 2343-2348
    CrossrefPubMedGoogle Scholar
  • 20 Knudsen SV, Laursen HVB, Johnsen SP, Bartels PD, Ehlers LH, Mainz J. Can quality improvement improve the quality of care? A systematic review of reported effects and methodological rigor in plan-do-study-act projects. BMC Health Serv Res 2019; 19 (01) 683
    CrossrefPubMedGoogle Scholar
  • 21 Taylor MJ, McNicholas C, Nicolay C, Darzi A, Bell D, Reed JE. Systematic review of the application of the plan-do-study-act method to improve quality in healthcare. BMJ Qual Saf 2014; 23 (04) 290-298
    CrossrefPubMedGoogle Scholar
  • 22 Kane DW, Hohman MM, Cerami EG, McCormick MW, Kuhlmman KF, Byrd JA. Agile methods in biomedical software development: a multi-site experience report. BMC Bioinformatics 2006; 7: 273
    CrossrefPubMedGoogle Scholar
  • 23 Tumosa N, Horvath KJ, Huh T. et al. Health care workforce development in rural America: when geriatrics expertise is 100 miles away. Gerontol Geriatr Educ 2012; 33 (02) 133-151
    CrossrefPubMedGoogle Scholar
  • 24 Esquer Rochin MA, Gutierrez-Garcia JO, Rosales JH, Rodriguez LF. Design and evaluation of a dashboard to support the comprehension of the progression of patients with dementia in day centers. Int J Med Inform 2021; 156: 104617
    CrossrefPubMedGoogle Scholar
  • 25 U.S. Department of Health and Human Services. The Research-Based Web Deisgn & Usability Guidelines, Enlarged/Expanded edition. Washington: US Government Printing Office; 2006
    Google Scholar
  • 26 Richter Lagha R, Burningham Z, Sauer BC. et al. Usability testing a potentially inappropriate medication dashboard: a core component of the dashboard development process. Appl Clin Inform 2020; 11 (04) 528-534
    Article in Thieme ConnectPubMedGoogle Scholar
  • 27 By the American Geriatrics Society Beers Criteria Update Expert P. American Geriatrics Society 2019 Updated AGS Beers Criteria(R) for Potentially Inappropriate Medication Use in Older Adults. J Am Geriatr Soc 2019; 67 (04) 674-694
    CrossrefPubMedGoogle Scholar
  • 28 van Deen WK, Cho ES, Pustolski K. et al. Involving end-users in the design of an audit and feedback intervention in the emergency department setting – a mixed methods study. BMC Health Serv Res 2019; 19 (01) 270
    CrossrefPubMedGoogle Scholar
  • 29 Tuti T, Nzinga J, Njoroge M. et al. A systematic review of electronic audit and feedback: intervention effectiveness and use of behaviour change theory. Implement Sci 2017; 12 (01) 61
    CrossrefPubMedGoogle Scholar
  • 30 Brehaut JC, Colquhoun HL, Eva KW. et al. Practice feedback interventions: 15 suggestions for optimizing effectiveness. Ann Intern Med 2016; 164 (06) 435-441
    CrossrefPubMedGoogle Scholar
  • 31 Lu CY, Vinci LM, Quinn MT, Wilkes AE, Chin MH, Peek ME. Using feedback to change primary care physician behavior. J Ambul Care Manage 2015; 38 (02) 118-124
    CrossrefPubMedGoogle Scholar
  • 32 Yarahuan JKW, Lo HY, Bass L, Wright J, Hess LM. Design, usability, and acceptability of a needs-based, automated dashboard to provide individualized patient-care data to pediatric residents. Appl Clin Inform 2022; 13 (02) 380-390
    Article in Thieme ConnectPubMedGoogle Scholar
  • 33 Yen PY, Bakken S. Review of health information technology usability study methodologies. J Am Med Inform Assoc 2012; 19 (03) 413-422
    CrossrefPubMedGoogle Scholar