Download PDF
CC BY-NC-ND 4.0 · Appl Clin Inform 2024; 15(01): 145-154
DOI: 10.1055/a-2235-9557
Case Report

Seamless Integration of Computer-Adaptive Patient Reported Outcomes into an Electronic Health Record

Kyle Nolla 
1   Department of Medical Social Sciences, Feinberg School of Medicine, Northwestern University, Chicago, Illinois, United States
,
Luke V. Rasmussen 
2   Department of Preventative Medicine, Feinberg School of Medicine, Northwestern University, Chicago, Illinois, United States
,
Nan E. Rothrock 
1   Department of Medical Social Sciences, Feinberg School of Medicine, Northwestern University, Chicago, Illinois, United States
,
Zeeshan Butt
3   Phreesia, Inc, Clinical Content, Wilmington, DE, USA
4   Department of Psychiatry and Behavioral Sciences, Feinberg School of Medicine, Northwestern University, Chicago, Illinois, United States
,
Michael Bass 
1   Department of Medical Social Sciences, Feinberg School of Medicine, Northwestern University, Chicago, Illinois, United States
,
Kristina Davis 
5   Department of Nursing Quality, Stanford Health Care, Stanford, California, United States
,
David Cella 
1   Department of Medical Social Sciences, Feinberg School of Medicine, Northwestern University, Chicago, Illinois, United States
,
Richard Gershon 
1   Department of Medical Social Sciences, Feinberg School of Medicine, Northwestern University, Chicago, Illinois, United States
,
Cynthia Barnard 
6   Department of General Internal Medicine, Feinberg School of Medicine, Northwestern University and Northwestern Memorial HealthCare, Chicago, Illinois, United States
,
Ryan Chmiel
7   Department of Information Services, Northwestern Memorial HealthCare, Chicago, Illinois, United States
,
Federico Almaraz
7   Department of Information Services, Northwestern Memorial HealthCare, Chicago, Illinois, United States
,
Michael Schachter
7   Department of Information Services, Northwestern Memorial HealthCare, Chicago, Illinois, United States
,
Therese Nelson 
8   Clinical and Translational Sciences Institute, Northwestern University, Chicago, Illinois, United States
,
Michelle Langer
1   Department of Medical Social Sciences, Feinberg School of Medicine, Northwestern University, Chicago, Illinois, United States
,
Justin Starren
1   Department of Medical Social Sciences, Feinberg School of Medicine, Northwestern University, Chicago, Illinois, United States
2   Department of Preventative Medicine, Feinberg School of Medicine, Northwestern University, Chicago, Illinois, United States
8   Clinical and Translational Sciences Institute, Northwestern University, Chicago, Illinois, United States
› Author Affiliations Funding The initial development of NMPRO was funded by the Vice Dean for Scientific Affairs and Graduate Education at the Feinberg School of Medicine. NMPRO and J.S. were funded by UL1TR000150 and UL1TR001422 from the National Center for Advancing Translational Science (NCATS). Evaluation and lessons learned development was funded by U01TR001806 from NCATS. K.N. was supported by NIH/NCI training grant CA193193. In kind implementation support was provided by Northwestern Memorial HealthCare.
› Further Information
   Permissions and Reprints

Abstract

Background Patient-reported outcome (PRO) measures have become an essential component of quality measurement, quality improvement, and capturing the voice of the patient in clinical care. In 2004, the National Institutes of Health endorsed the importance of PROs by initiating the Patient-Reported Outcomes Measurement Information System (PROMIS), which leverages computer-adaptive tests (CATs) to reduce patient burden while maintaining measurement precision. Historically, PROMIS CATs have been used in a large number of research studies outside the electronic health record (EHR), but growing demand for clinical use of PROs requires creative information technology solutions for integration into the EHR.

Objectives This paper describes the introduction of PROMIS CATs into the Epic Systems EHR at a large academic medical center using a tight integration; we describe the process of creating a secure, automatic connection between the application programming interface (API) which scores and selects CAT items and Epic.

Methods The overarching strategy was to make CATs appear indistinguishable from conventional measures to clinical users, patients, and the EHR software itself. We implemented CATs in Epic without compromising patient data security by creating custom middleware software within the organization's existing middleware framework. This software communicated between the Assessment Center API for item selection and scoring and Epic for item presentation and results. The middleware software seamlessly administered CATs alongside fixed-length, conventional PROs while maintaining the display characteristics and functions of other Epic measures, including automatic display of PROMIS scores in the patient's chart. Pilot implementation revealed differing workflows for clinicians using the software.

Results The middleware software was adopted in 27 clinics across the hospital system. In the first 2 years of hospital-wide implementation, 793 providers collected 70,446 PROs from patients using this system.

Conclusion This project demonstrated the importance of regular communication across interdisciplinary teams in the design and development of clinical software. It also demonstrated that implementation relies on buy-in from clinical partners as they integrate new tools into their existing clinical workflow.

Keywords

clinical informatics - electronic health records - IT design and development methodologies - patient-reported outcome measures - health informatics interoperability

Protection of Human and Animal Subjects

NMPRO was a quality improvement effort on behalf of NM. Consequently, it was not considered Human Subjects Research.




Publication History

Received: 25 August 2023

Accepted: 06 December 2023

Accepted Manuscript online:
28 December 2023

Article published online:
21 February 2024

© 2024. The Author(s). This is an open access article published by Thieme under the terms of the Creative Commons Attribution-NonDerivative-NonCommercial License, permitting copying and reproduction so long as the original work is given appropriate credit. Contents may not be used for commercial purposes, or adapted, remixed, transformed or built upon. (https://creativecommons.org/licenses/by-nc-nd/4.0/)

Georg Thieme Verlag KG
Rüdigerstraße 14, 70469 Stuttgart, Germany

 

  • References

  • 1 Basch E. Patient-reported outcomes - harnessing patients' voices to improve clinical care. N Engl J Med 2017; 376 (02) 105-108
    CrossrefPubMedGoogle Scholar
  • 2 Kotronoulas G, Kearney N, Maguire R. et al. What is the value of the routine use of patient-reported outcome measures toward improvement of patient outcomes, processes of care, and health service outcomes in cancer care? A systematic review of controlled trials. J Clin Oncol 2014; 32 (14) 1480-1501
    CrossrefPubMedGoogle Scholar
  • 3 Cella D, Hahn EA, Jensen SE. et al. Patient-Reported Outcomes in Performance Measurement. North Carolina, USA: RTI Press; 2015
    CrossrefGoogle Scholar
  • 4 Gershon RC, Rothrock N, Hanrahan R, Bass M, Cella D. The use of PROMIS and assessment center to deliver patient-reported outcome measures in clinical research. J Appl Meas 2010; 11 (03) 304-314
    PubMedGoogle Scholar
  • 5 HealthMeasures.net. Intro to PROMIS®. Accessed August 17, 2023 at: https://www.healthmeasures.net/explore-measurement-systems/promis/intro-to-promis
    PubMedGoogle Scholar
  • 6 Fries JF, Bruce B, Cella D. The promise of PROMIS: using item response theory to improve assessment of patient-reported outcomes. Clin Exp Rheumatol 2005; 23 (5 Suppl 39) S53-S57
    PubMedGoogle Scholar
  • 7 Kane LT, Namdari S, Plummer OR, Beredjiklian P, Vaccaro A, Abboud JA. Use of computerized adaptive testing to develop more concise patient-reported outcome measures. JBJS Open Access 2020; 5 (01) e0052
    CrossrefPubMedGoogle Scholar
  • 8 Long C, Beres LK, Wu AW, Giladi AM. Patient-level barriers and facilitators to completion of patient-reported outcomes measures. Qual Life Res 2022; 31 (06) 1711-1718
    CrossrefPubMedGoogle Scholar
  • 9 Segawa E, Schalet B, Cella D. A comparison of computer adaptive tests (CATs) and short forms in terms of accuracy and number of items administrated using PROMIS profile. Qual Life Res 2020; 29 (01) 213-221
    CrossrefPubMedGoogle Scholar
  • 10 Cook KF, Jensen SE, Schalet BD. et al. PROMIS measures of pain, fatigue, negative affect, physical function, and social function demonstrated clinical validity across a range of chronic conditions. J Clin Epidemiol 2016; 73: 89-102
    CrossrefPubMedGoogle Scholar
  • 11 Gershon R, Rothrock NE, Hanrahan RT, Jansky LJ, Harniss M, Riley W. The development of a clinical outcomes survey research application: Assessment Center. Qual Life Res 2010; 19 (05) 677-685
    CrossrefPubMedGoogle Scholar
  • 12 Gensheimer SG, Wu AW, Snyder CF. PRO-EHR Users' Guide Steering Group, PRO-EHR Users' Guide Working Group. Oh, the places we'll go: patient-reported outcomes and electronic health records. Patient 2018; 11 (06) 591-598
    CrossrefPubMedGoogle Scholar
  • 13 Bass M, Oncken C, McIntyre AW, Dasilva C, Spuhl J, Rothrock NE. Implementing an Application Programming Interface for PROMIS Measures at three medical centers. Appl Clin Inform 2021; 12 (05) 979-983
    Article in Thieme ConnectPubMedGoogle Scholar
  • 14 De Bruijne M, Wijnant A. Improving response rates and questionnaire design for mobile web surveys. Public Opin Q 2014; 78 (04) 951-962
    CrossrefPubMedGoogle Scholar
  • 15 Biber J, Ose D, Reese J. et al. Patient reported outcomes - experiences with implementation in a University Health Care setting. J Patient Rep Outcomes 2018; 2 (01) 34
    CrossrefPubMedGoogle Scholar
  • 16 Nelson TA, Anderson B, Bian J. et al. Planning for patient-reported outcome implementation: Development of decision tools and practical experience across four clinics. J Clin Transl Sci 2020; 4 (06) 498-507
    CrossrefPubMedGoogle Scholar
  • 17 Zhang R, Burgess ER, Reddy MC. et al. Provider perspectives on the integration of patient-reported outcomes in an electronic health record. JAMIA Open 2019; 2 (01) 73-80
    CrossrefPubMedGoogle Scholar
  • 18 May JR, Klass E, Davis K. et al. Leveraging patient reported outcomes measurement via the electronic health record to connect patients with cancer to smoking cessation treatment. Int J Environ Res Public Health 2020; 17 (14) 5034
    CrossrefPubMedGoogle Scholar
  • 19 Cella D, Garcia SF, Cahue S. et al. Implementation and evaluation of an expanded electronic health record-integrated bilingual electronic symptom management program across a multi-site Comprehensive Cancer Center: The NU IMPACT protocol. Contemp Clin Trials 2023; 128: 107171
    CrossrefPubMedGoogle Scholar
  • 20 Garcia SF, Wortman K, Cella D. et al. Implementing electronic health record-integrated screening of patient-reported symptoms and supportive care needs in a comprehensive cancer center. Cancer 2019; 125 (22) 4059-4068
    CrossrefPubMedGoogle Scholar
  • 21 Hassett MJ, Cronin C, Tsou TC. et al. eSyM: an electronic health record-integrated patient-reported outcomes-based cancer symptom management program used by six diverse health systems. JCO Clin Cancer Inform 2022; 6: e2100137
    CrossrefPubMedGoogle Scholar
  • 22 Epic on FHIR. Connection Hub. Accessed August 17, 2023 at: https://fhir.epic.com/ConnectionHub
    PubMedGoogle Scholar
  • 23 Sayeed R, Gottlieb D, Mandl KD. SMART Markers: collecting patient-generated health data as a standardized property of health information technology. NPJ Digit Med 2020; 3: 9
    CrossrefPubMedGoogle Scholar
  • 24 Wesley DB, Blumenthal J, Shah S. et al. A novel application of SMART on FHIR architecture for interoperable and scalable integration of patient-reported outcome data with electronic health records. J Am Med Inform Assoc 2021; 28 (10) 2220-2225
    CrossrefPubMedGoogle Scholar
  • 25 Papuga MO, Dasilva C, McIntyre A, Mitten D, Kates S, Baumhauer JF. Large-scale clinical implementation of PROMIS computer adaptive testing with direct incorporation into the electronic medical record. Health Syst (Basingstoke) 2017; 7 (01) 1-12
    PubMedGoogle Scholar
  • 26 Hyland CJ, Guo R, Dhawan R. et al. Implementing patient-reported outcomes in routine clinical care for diverse and underrepresented patients in the United States. J Patient Rep Outcomes 2022; 6 (01) 20
    CrossrefPubMedGoogle Scholar
  • 27 Austin AM, Carmichael D, Berry S. et al. Chronic condition measurement requires engagement, not measurement alone. J Ambul Care Manage 2019; 42 (04) 295-304
    CrossrefPubMedGoogle Scholar
  • 28 Stover AM, Tompkins Stricker C, Hammelef K. et al. Using stakeholder engagement to overcome barriers to implementing patient-reported outcomes (PROs) in cancer care delivery: approaches from 3 prospective studies. Med Care 2019; 57 (Suppl 5 Suppl 1) S92-S99
    CrossrefPubMedGoogle Scholar
  • 29 Turchioe MR, Mangal S, Goyal P. et al. A RE-AIM evaluation of a visualization-based electronic patient-reported outcome system. Appl Clin Inform 2023; 14 (02) 227-237
    Article in Thieme ConnectPubMedGoogle Scholar
  • 30 Chugh R, Liu AW, Idomsky Y. et al. A digital health intervention to improve the clinical care of inflammatory bowel disease patients. Appl Clin Inform 2023; 14 (05) 855-865
    Article in Thieme ConnectPubMedGoogle Scholar
  • 31 Rotenstein LS, Agarwal A, O'Neil K. et al. Implementing patient-reported outcome surveys as part of routine care: lessons from an academic radiation oncology department. J Am Med Inform Assoc 2017; 24 (05) 964-968
    CrossrefPubMedGoogle Scholar
  • 32 Perry LM, Morken V, Peipert JD. et al. Patient-reported outcome dashboards within the electronic health record to support shared decision-making: protocol for co-design and clinical evaluation with patients with advanced cancer and chronic kidney disease. JMIR Res Protoc 2022; 11 (09) e38461
    CrossrefPubMedGoogle Scholar
  • 33 Cimino JJ. Putting the “why” in “EHR”: capturing and coding clinical cognition. J Am Med Inform Assoc 2019; 26 (11) 1379-1384
    CrossrefPubMedGoogle Scholar
  • 34 Harle CA, Listhaus A, Covarrubias CM. et al. Overcoming barriers to implementing patient-reported outcomes in an electronic health record: a case report. J Am Med Inform Assoc 2016; 23 (01) 74-79
    CrossrefPubMedGoogle Scholar
  • 35 Burton SV, Valenta AL, Starren J. et al. Examining perspectives on the adoption and use of computer-based patient-reported outcomes among clinicians and health professionals: a Q methodology study. J Am Med Inform Assoc 2022; 29 (03) 443-452
    CrossrefPubMedGoogle Scholar