Potential of an Electronic Health Record-Integrated Patient Portal for Improving Care Plan Concordance during Acute Care

Abstract

Background Care plan concordance among patients and clinicians during hospitalization is suboptimal.

Objective This article determines whether an electronic health record (EHR)-integrated patient portal was associated with increased understanding of the care plan, including the key recovery goal, among patients and clinicians in acute care setting.

Methods The intervention included (1) a patient portal configured to solicit a single patient-designated recovery goal and display the care plan from the EHR for participating patients; and (2) an electronic care plan for all unit-based nurses that displays patient-inputted information, accessible to all clinicians via the EHR. Patients admitted to an oncology unit, including their nurses and physicians, were enrolled before and after implementation. Main outcomes included mean concordance scores for the overall care plan and individual care plan elements.

Results Of 457 and 283 eligible patients approached during pre- and postintervention periods, 55 and 46 participated in interviews, respectively, including their clinicians. Of 46 postintervention patients, 27 (58.7%) enrolled in the patient portal. The intention-to-treat analysis demonstrated a nonsignificant increase in the mean concordance score for the overall care plan (62.0–67.1, adjusted p = 0.13), and significant increases in mean concordance scores for the recovery goal (30.3–57.7, adjusted p < 0.01) and main reason for hospitalization (58.6–79.2, adjusted p < 0.01). The on-treatment analysis of patient portal enrollees demonstrated significant increases in mean concordance scores for the overall care plan (61.9–70.0, adjusted p < 0.01), the recovery goal (30.4–66.8, adjusted p < 0.01), and main reason for hospitalization (58.3–81.7, adjusted p < 0.01), comparable to the intention-to-treat analysis.

Conclusion Implementation of an EHR-integrated patient portal was associated with increased concordance for key care plan components. Future efforts should be directed at improving concordance for other care plan components and conducting larger, randomized studies to evaluate the impact on key outcomes during transitions of care.

Clinical Trials Identifier NCT02258594.

Keywords

patient engagement - patient portals - care plan concordance - patient–clinician communication

Background and Significance

Engaging hospitalized patients in understanding their care plan and establishing recovery goals is fundamental to patient-centered care. Unfortunately, patients and clinicians are often not “on the same page” about the care plan during hospitalization, and goals for recovery are typically not established as part of routine hospital care.[1] [2] [3] Although efforts at understanding the plan and establishing goals through geographic regionalization of care teams (i.e., a team of clinicians delivering care on a specific unit) and structured interdisciplinary rounds have resulted in some improvements,[4] [5] patient–clinician concordance remains suboptimal.[2] [3] [5] For example, we previously reported poor concordance among patients and key clinicians with regard to identifying a single recovery goal for hospitalization—patients, nurses, and physicians identified the same goal in just 20% of cases—and we observed no difference among regionalized (a regionalized care teams refers to a team of clinicians delivering care on a geographically contained unit) versus nonregionalized care teams (a nonregionalized care team is a team of clinicians delivering care on multiple units that may be geographically distant from one another).[3]

Promoting shared understanding of the care plan and goals is increasingly important: health care systems are being penalized for hospital readmissions and poor patient satisfaction scores,[6] which can occur when the care delivered by the care team is not congruent with patients' expectations and preferences.[7] [8] [9] [10] Achieving “goal-concordant” care (i.e., when clinicians deliver care aligned with the values of patients) is particularly important for seriously ill patients, such as those with advanced cancer or severe chronic illness (e.g., emphysema) who are at elevated mortality risk and are often hospitalized.[11] [12] When goals are clearly established and the plan is seamlessly communicated among seriously ill patients and their clinicians, patients are more likely to receive high-quality care consistent with stated preferences and experience better outcomes.[12] [13] Though validated tools to categorize patients' recovery goals during hospitalization currently exist,[14] interventions that effectively communicate patient-designated recovery goals directly to clinicians within the electronic health record (EHR) have not yet materialized, even for patients with advanced cancer.[9] [12] [15] [16] [17] [18]

In recent years, health care systems have been trying to engage patients by offering online access to their health records via patient portals, and reported experiences regarding implementation of patient portals for acute care are now starting to emerge.[15] [19] [20] [21] [22] [23] [24] [25] [26] [27] Still, few studies have demonstrated meaningful impact on key outcomes,[20] [28] [29] [30] and to our knowledge, none have demonstrated the potential for using patient portals to improve concordance about the care plan, including recovery goals, among patients and clinicians in the acute care setting.[23] [31] [32] Improving patient–clinician communication through the meaningful use of patient portals represents a promising strategy to enhance mutual understanding about the plan and facilitate goal-concordant care for seriously ill patients during hospitalization and the transition back to the ambulatory setting.[33] [34] [35] [36] [37]

Objective

We aimed to determine the degree to which hospitalized oncology patients and their clinicians were concordant about the overall care plan, including the key goal for recovery, before and after implementation of an EHR-integrated patient portal. The patient portal ([Fig. 1]), designed and developed specifically for acute care,[38] was configured to facilitate shared understanding of key elements of the care plan and patient-designated recovery goals among patients and clinicians during hospitalization.[38] [39] [40]

Fig. 1 Acute Care Patient Portal integrated with Electronic Health Record (EHR). During the postintervention period, the acute care patient portal (background, left) was accessible to patients and designated caregivers via tablet computers and configured to improve communication with clinicians by synchronizing with the EHR (foreground, right). In the patient portal, patients could select a single recovery goal which was communicated to their care team via the EHR. Patients could also view other key elements of their care plan, including the main reason for hospitalization, active problems, and a schedule for the day; these elements were maintained by nurses via the EHR.

Methods

Study Design, Setting, and Participants

We received approval from the Partners Human Research Committee to conduct a prospective, pre-post interventional study on an oncology unit at Brigham and Women's Hospital, a large academic medical center in Boston, Massachusetts, United States. We collected data on all enrolled patients admitted to the study unit during a baseline period from November 2013 to May 2014 prior to implementation of the intervention (described below), and during the postintervention period from January 2015 to May 2015.

All adult patients (> 18 years) admitted to the oncology unit for at least 24 hours were eligible to participate. Patients who demonstrated capacity (determined by a nurse or physician member of the care team) or had a legally designated health care proxy (who spoke English and was available to participate on their behalf) were eligible. Patients who did not have capacity or an available caregiver, declined to participate, or were admitted to the care unit for less than 24 hours were excluded. Two nonregionalized medical teams cared for patients admitted to the study unit, each consisting of a “first responder” (e.g., intern or a physician assistant [PA]), a resident, and an attending physician. Nurses were unit-based and cared only for patients admitted to the study unit.

During the intervention period, patients could elect to enroll in the patient portal independently from this study of care plan concordance: patients who enrolled in the patient portal were trained by research assistants to use all features (see below) approximately 24 hours upon arrival to the study unit.[38] All nurses working on study units were required to use a new electronic care plan and received training by study staff at the beginning of the intervention period. Physicians and PAs caring for patients on study units were introduced to the intervention components prior to the start of their clinical rotations.

Intervention

The components of intervention included the patient portal and clinician-facing care planning tools. The patient portal designed and developed for acute care ([Fig. 1], left) has been previously described.[38] Briefly, patients (or authorized caregivers) could select a single goal for their recovery (based on work by Haberle et al, [Table 1]) during the current hospitalization[14]; navigate their care plan (main diagnoses, care team goals, schedule of tests and procedures); message their care team; review medications and test results; and view educational content.[38] The patient portal was tightly integrated with the clinician-facing care planning tools ([Fig. 1], right) that were accessible from the EHR. The bedside nurse (or a physician) could update the new electronic care plan (main diagnoses, care team goals, schedule) and view the patient-designated recovery goal directly from the EHR ([Fig. 1], right) when notified of new information via flags and/or automated emails.

Table 1 Haberle recovery goals

The seven previously validated Haberle recovery goals include the following:

• Be cured

• Be comfortable

• Improve or maintain health

• Live longer

• Accomplish personal goal

• Provider support for family

• Other

Note: In the study, if a patient within lymphoma was admitted with pneumonia, s/he was asked to select a single recovery goal for the main reason for hospitalization (i.e., pneumonia). In this context, selecting “be cured” would mean a cure for pneumonia, not a cure for cancer. If a patient was admitted for elected chemotherapy for refractory leukemia, then “be cured” would mean a cure for cancer.

Patient portal enrollees were provided hospital-issued tablet devices (iPad Air, Apple, Inc.) which were managed centrally as previously described.[41] All nurses working on the study unit could view information inputted by the patient (e.g., Haberle recovery goal) and update the electronic care plan, including main reason for hospitalization and active problems, care team-designated goals for patient, and schedule for tests, procedures, and consults, regardless of whether these would be shared with the patient via the patient portal.[38] [39] [40]

Data Collection

We modified a validated, structured care plan interview instrument to ask study participants to identify the patient's single Haberle recovery goal for the hospitalization ([Table 1]).[2] [14] In addition to standard care plan concordance questions, we asked patients (or the designated health care proxy) to select their single, most important Haberle goal for recovery during hospitalization. As in the original validation study, patients or proxies were asked the following question: “Please tell me your most important goal of care for this hospitalization.” If they did not understand this question, we asked a follow-up question: “What are you expecting will be accomplished during this hospitalization?” Research assistants approached and interviewed eligible patients who had been admitted to the study unit for at least 48 hours in random order using the patient version of this data collection instrument ([Supplementary Appendix A], available in the online version) until reaching a weekly target of six participants. During the postintervention period, we approached eligible patients with the goal of sampling patient portal enrollees and nonenrollees in a 1:1 ratio. Research assistants then approached the patient's bedside nurse and, when possible, a physician (first responder or attending) from the primary medical team to participate in an interview on the same day or within 24 hours of the patient's interview using the clinician version of this data collection instrument ([Supplementary Appendix B], available in the online version). All participants were blinded to the responses of others. We obtained patient demographic data from administrative databases.

Scoring

Two board-certified internists (A.D. and L.S.) independently assessed and scored all eight care plan elements for each dyad (i.e., patient–nurse, patient–physician, nurse–physician) for no (0), partial (0.5), or complete agreement[1] between the dyad participants. Physician reviewers were blinded to each other's assessments. For each care plan element, the concordance score was based on the average of all three dyads (when available), or just the patient–nurse dyad alone. All scoring discrepancies were resolved using a two-person consensus approach.

Outcomes

The primary outcome, concordance for the overall care plan, was defined as the mean concordance score of all eight care plan elements per patient admission. Secondary outcomes included concordance for individual care plan elements (defined as the mean concordance score for each care plan element, e.g., Haberle recovery goal).

Power and Sample Size

We estimated baseline mean concordance scores among patients and key members of the care team at 52% based on studies by O'Leary et al.[2] Based on prior work, we anticipated that our intervention could achieve mean concordance scores of 72% by virtue of improving mutual awareness.[42] We estimated a sample size of approximately 50 patients in each arm would be adequate to measure an increase in the mean concordance score from 52 to 72%, with power of 80%, an alpha of 0.05, and a sigma of 0.5.

Statistical Analysis

Mean concordance scores were calculated as the average of the sum of all dyad scores across the eight care plan elements (overall care plan) or as the sum of all dyad scores (individual care plan elements), reported on a scale of 0 to 100. For cases in which a physician was not interviewed, scores were based solely on the patient–nurse dyad. In the main intention-to-treat analysis, mean concordance scores for the overall care plan and individual care plan elements were compared between the pre- and postintervention periods for all study participants using a generalized estimating equations z-test. Thus, the intention-to-treat analysis compared care plan concordance for patient participants in the preintervention period to care plan concordance for all patient participants (both portal and nonportal enrollees) in the postintervention period. Weighted propensity score methods were used to adjust for key demographics differences between the pre- and postintervention periods in our study cohort. In the a priori planned on-treatment analysis, mean concordance scores for the overall care plan and individual care plan elements were compared between preintervention controls and postintervention patient portal enrollees and similarly analyzed.

Results

Of 457 and 283 eligible patients ([Fig. 2]) admitted during the pre- and postintervention periods, we approached 212 and 101 patients, respectively, in random order. Of these, 55 and 46 patients were available, had capacity (or had an available caregiver), and consented to participate in the interviews during the pre- and postintervention periods, respectively. Care plan concordance interviews were conducted with the patient (or a designated caregiver) and the patient's bedside nurse for all patient admissions. A physician was interviewed in 27 (49.1%) and 12 (26.1%) patient admissions in the pre- and postintervention periods, respectively. Of the 46 patients who participated in care plan concordance interviews during the postintervention period, 27 (58.7%) had independently enrolled in the patient portal upon admission.

Fig. 2 CONSORT diagram.

In general, patient demographics ([Table 2]) were balanced in both study periods, but length of stay was longer in the postintervention period. In the intention-to-treat analysis ([Table 3]), there was a nonsignificant increase in the mean concordance score for the overall care plan from 62.0 to 67.1 (adjusted p = 0.13) among patient and clinician participants. However, there was a significant increase in mean concordance scores for the Haberle recovery goal (30.3–57.7, adjusted p < 0.01) and main reason for hospitalization (58.6–79.2, adjusted p < 0.01).

Table 2 Demographics of patient admissions

Characteristics	Pre (n = 55)	Post (n = 46)	p-Value
Patient admissions (no. unique patients)	55 (54)	46 (46)
Female (%)	22 (40.0)	21 (45.7)	0.57
Mean age (SD)	58.6 (12.8)	58.4 (13.5)	0.83
Race/Ethnicity – White (%)	47 (88.7)	38 (82.6)	0.37
Insurance (%)
Medicaid/Medicare	19 (35.2)	13 (28.3)	0.88
Private	34 (63.0)	31 (67.4)
Self-pay/Other	1 (1.9)	2 (4.4)
Mean Charlson score (SD)	4.0 (2.8)	3.2 (2.8)	0.14
Mean of median income by Zip-code (SD)	$68,754 ($21,460)	$70,359 ($19,525)	0.42
Primary diagnosis at admission (%)
Oncologic	29 (52.7)	29 (63.0)	0.30
Infectious	12 (21.8)	4 (8.7)
Neurologic	3 (5.5)	1 (2.2)
Gastrointestinal	3 (5.5)	5 (10.9)
Cardiovascular/Respiratory	4 (7.3)	4 (8.7)
Genitourinary/Renal	0 (0)	2 (4.4)
Metabolic/Other	4 (7.3)	1 (2.2)
Length of stay
Care unit–mean days (SD)	7.18 (6.5)	12.33 (9.3)	< 0.01
Hospital–mean days (SD)	10.65 (8.7)	14.39 (9.6)	0.04
Participant Interviewed
Patient	49	45
Caregiver (healthcare proxy)	6	1
Nurse (no. unique)	55 (24)	46 (27)
Physician (no. unique)	27 (22)	12 (11)
Attending	6 (6)	0 (0)
First responder	21 (16)	12 (11)

Abbreviation: SD, standard deviation.

Table 3 Intention-to-treat analysis: Care plan concordance

Outcome	Unadjusted analysis			Adjusted analysis[b]
	Pre, n = 55	Post, n = 46	p-Value	Pre, n = 55	Post, n = 46	p-Value[b]
Mean concordance score for overall care plan (0–100)	62.2	68.1	0.06	62.0	67.1	0.13
Mean concordance scores for individual care plan elements (0–100)
Haberle recovery goal[a]	31.1	60.0	< 0.01	30.3	57.7	< 0.01
Main reason for hospitalization	59.0	80.0	< 0.01	58.6	79.2	< 0.01
Tests scheduled	53.8	47.1	0.41	53.5	46.7	0.41
Procedures scheduled	70.8	72.5	0.85	70.2	72.8	0.77
Medications changed	49.1	56.1	0.36	48.9	54.4	0.48
Consults planned	53.3	45.4	0.36	51.4	44.6	0.45
Time of discharge	94.7	98.5	0.18	95.4	97.9	0.42
Discussion with patient or clinician	87.3	79.1	0.18	88.3	76.8	0.06

Note: The mean concordance score for overall care plan is an average of individual concordance scores across each of the eight care plan elements: Haberle recovery goal, main reason for hospitalization, tests scheduled, procedures scheduled, medications changed, consults planned, time of discharge, and discussion with patient or clinician. p-values in bold are statistically significant.

^a The choices for the Haberle recovery goal are: “be cured,” “live longer,” “improve health or maintain health,” “be comfortable,” “accomplish a particular life goal,” or “other.”

^b Adjusted for care unit length of stay.

In the on-treatment analysis ([Table 4]) of the 27 postintervention patient portal enrollees ([Supplementary Appendix C], available in the online version), there was a significant increase in the mean concordance score for the overall care plan from the pre- to postintervention period (61.9–70.1, adjusted p < 0.01) among patient and clinician participants. There were also significant increases in mean concordance scores for the Haberle recovery goal (30.4–66.8, adjusted p < 0.01) and main reason for hospitalization (58.3–81.7, adjusted p < 0.01), but not for other care plan elements.

Table 4 On-treatment analysis: Care plan concordance

Outcome	Unadjusted analysis			Adjusted analysis[b]
	Pre, n = 55	Post, portal users, n = 27	p-Value	Pre, n = 55	Post, portal users, n = 27	p-Value[b]
Mean concordance score for overall care plan (0–100)	62.2	70.0	0.02	61.9	70.1	< 0.01
Mean concordance scores for individual care plan elements (0–100)
Haberle recovery goal[a]	31.1	66.7	< 0.01	30.4	66.8	< 0.01
Main reason for hospitalization	59.0	81.1	< 0.01	58.3	81.7	< 0.01
Tests scheduled	53.8	44.3	0.31	53.5	45.8	0.45
Procedures scheduled	70.8	68.2	0.81	70.2	68.9	0.91
Medications changed	49.1	54.9	0.52	49.0	53.7	0.61
Consults planned	53.3	58.8	0.58	51.3	59.5	0.43
Time of discharge	94.7	97.6	0.37	95.5	95.5	1.00
Discussion with patient or clinician	87.3	79.8	0.30	88.5	79.2	0.14

Note: The mean concordance score for overall care plan is an average of individual concordance scores across each of the eight care plan elements: Haberle recovery goal, main reason for hospitalization, tests scheduled, procedures scheduled, medications changed, consults planned, time of discharge, and discussion with patient or clinician. p-values in bold are statistically significant.

^a The choices for the Haberle recovery goal are: “be cured,” “live longer,” “improve health or maintain health,” “be comfortable,” “accomplish a particular life goal,” or “other.”

^b Adjusted for care unit length of stay.

Discussion

We evaluated the potential for an EHR-integrated patient portal—configured to share key clinical information among patients and clinicians—to improve care plan concordance, including patient-designated recovery goals, for patients admitted to an oncology unit before and after implementation. In the intention-to-treat analysis, we observed a nonsignificant increase in overall care plan concordance, and significant increases in concordance related to key care plan elements, namely, the patient-designated Haberle recovery goal and the main reason for hospitalization from the pre- to postintervention period. The on-treatment analysis demonstrated significant improvement in overall care plan concordance from the pre- to postintervention period that was primarily related to these key elements, and comparable in magnitude with the intention-to-treat analysis.

Our findings can be explained in part by our participatory approach to designing the EHR-integrated patient portal,[40] and by how we engaged unit-based clinical staff during implementation.[39] First, patients, nurses, and physicians had a single source of truth to view these static components of the care plan (e.g., Haberle recovery goal, main reason for hospitalization) at any point during hospitalization. Second, the patient portal was specifically configured to encourage patients to enter recovery goals, and this was reinforced via teach-back.[38] In our previous pilot, we reported high use of the patient portal by patients and caregivers during hospitalization for entering recovery goals and viewing the main reason for hospitalization.[38] Third, this information was directly communicated to nurses and physicians via the EHR within their workflow, thereby ensuring clinicians could easily view the Haberle recovery goal selected by patients.[38] [40] Additionally, physicians and nurses were automatically notified (via EHR flags or automated emails) when recovery goals were entered or updated by patient portal participants.[38] [43] Finally, as part of the implementation program, unit-based nurses were required to use the new electronic care plan on all patients admitted to the study unit, regardless of whether those patients had enrolled in using the patient portal or enrolled in this study: nurses were specifically trained to view recovery goals inputted by patients and update the main reason for hospitalization based on clinical documentation and discussions with physicians.[40]

Similar to other studies, we did not observe increases in concordance scores for dynamic components of the care plan (e.g., tests scheduled, planned procedures, medication changes, consults, etc.).[28] This type of information could change frequently depending on whether orders for tests, medications, or consults are correctly placed, or schedules for tests and procedures are accurate. Thus, if patients did not review the portal or the information had changed prior to participating in care plan concordance interviews, then they would not have had time to acquire pertinent knowledge about planned tests, procedures, and medication changes.[28] Furthermore, if clinicians did not regularly update this information via the clinician-facing care planning tools, then patients would not have been aware of the most current information. Conversely, if patients were diligent about looking up specific information (e.g., noting a medication change overnight), and clinicians were unaware of this information (i.e., medication change not communicated during a hand-off), then this may have also led to poor concordance at the time of interview.

Our study represents an early attempt at quantifying the potential for EHR-integrated patient portals to improve care plan concordance among patients and clinicians in the acute care setting. We underscore the workflow integration of clinician-facing care planning tools and unit-based training as crucial aspects of our implementation. When integrated in this way, we believe that patient portals, configured to promote transparency of the care plan and recovery goals, can complement efforts at engaging patients in earlier serious illness conversations and facilitating goal-concordant care over acute episodes of care.[8] [10] We note that the mean score for care plan concordance in our study was still 67.1 out of 100 during the postintervention period; thus, to achieve high levels of concordance among patients and clinicians, implementation of patient portals must also be aligned with efforts at improving in-person communication during hospitalization (e.g., through bedside rounding). When implemented in this way, we believe that a health information technology-enabled approach could lead to more realistic expectations of treatment and potentially higher patient satisfaction, particularly for those patients with serious illness, such as advanced cancer.[10] [35] [36] [37] [44] [45] Of course, the potential benefits must be balanced with unintended consequences, particularly for the more dynamic components of the care plan (e.g., increased anxiety when an incorrect medication is ordered but not administered). Still, the potential for patient/caregiver-initiated error detection mediated by such tools would likely outweigh these risks.[46] [47]

Our study has several limitations. First, this was a pre-post study with a small sample size that was conducted at a single institution and clinical service. Of note, while we did observe a nonsignificant trend toward improvement in our main outcome, concordance scores for the overall care plan in the preintervention period was higher (62%) than we had anticipated (52%) based on our a priori power calculation—this likely diminished our ability to detect a statistically significant increase. Second, patient portal enrollment, independent from enrollment in this study, may have been prone to selection bias: we note that patient portal enrollees tended to be privately insured and have longer lengths of stay ([Supplementary Appendix C], available in the online version). Third, an analysis for temporal trends was not performed due to small numbers; however, cointerventions aimed at improving patient–clinician communication about the care plan would have been expected to increase concordance of all elements, not just the static components. Furthermore, we were unaware of any other interventions specifically aimed at improving concordance of recovery goals or other care plan elements—attempts at regionalization of our inpatient oncology service have not been as successful as for our other clinical services.[5]

Conclusion

In summary, we assessed how patient portals tethered to the EHR could serve as a platform for improving patient–clinician communication and demonstrated the potential for improving concordance of key care plan elements, such as the recovery goal and main reason for hospitalization. The clinician-facing intervention components and unit-based training were crucial to engaging patients in identifying recovery goals and understanding main diagnoses as part of our implementation effort. Future efforts should be directed at addressing complexities of improving concordance for dynamic care plan elements, and conducting larger, randomized studies to assess impact on key care transitions outcomes, such as delivery of care congruent to patients' stated goals and hospital readmissions.

Clinical Relevance Statement

Patient–clinician communication during acute care is suboptimal. Patient portals tethered to the EHR have the potential for improving patient–clinician communication over acute episodes of care by enhancing mutual understanding about the care plan and facilitating goal-concordant care, which are increasingly important for seriously ill hospitalized patients.

Multiple Choice Questions

1. In the acute care setting, patients and clinicians will most likely share a mutual understanding of which of the following components of the care plan that are communicated via an EHR-integrated patient portal?

   • Medication list.

   • Test results.

   • Main reason for hospitalization.

   • Recovery goal.

   • C and D.

   • None of the above.

   Correct Answer: The correct answer is option e. In the acute care setting, patient and clinicians will most likely share a mutual understanding of the static components of the care plan, such as main reason for hospitalization and patient-designated goal for recovery, when communicated via an EHR-integrated patient portal. Other care plan elements, such as medications, test results, schedule of procedures, etc., may change more frequently during hospitalization.

2. Improving care plan concordance among patients and clinicians to optimal levels in the acute care setting will likely require:

   • A patient portal integrated with the electronic health record.

   • Training for unit-based nurses and physicians.

   • Aligning implementation of patient portals and clinician-facing care planning tools with efforts at promoting in-person communication (e.g., bedside rounding).

   • Addressing technical and implementation barriers for improving dynamic components of the care plan.

   • Options a, b, and c,

   • All of the above (a, b, c, and d).

   Correct Answer: The correct answer is option f. In this study, even after implementation of the EHR-integrated patient portal, overall care plan concordance was suboptimal (67.1 out of 100). Realizing further improvement will not only require technological tools and training for patients and clinicians, but also alignment with efforts at improving in-person communication and addressing technical and implementation barriers for improving the dynamic components of the care plan.

Conflict of Interest

None declared.

Acknowledgments

The authors would like to thank their colleagues and the patient and family advocates who contributed to the PROSPECT project: Dianna Stade, Conny Morrison, John Hanna, Sarah Collins, Kumiko Schnock, Priscilla Gazarian, Ronen Rozenblum, Marsha Clements, Kathy Leone, Frank Chang, Martie Carnie, Maureen Fagan, Anthony Massaro, and Lisa Lehmann.

Protection of Human and Animal Subjects

This study was reviewed and approved by the Partners Healthcare Institutional Review Board.

• References

• 1 O'Leary KJ, Thompson JA, Landler MP. , et al. Patterns of nurse-physician communication and agreement on the plan of care. Qual Saf Health Care 2010; 19 (03) 195-199
• 2 O'Leary KJ, Kulkarni N, Landler MP. , et al. Hospitalized patients' understanding of their plan of care. Mayo Clin Proc 2010; 85 (01) 47-52
• 3 Figueroa JF, Schnipper JL, McNally K, Stade D, Lipsitz SR, Dalal AK. How often are hospitalized patients and providers on the same page with regard to the patient's primary recovery goal for hospitalization?. J Hosp Med 2016; 11 (09) 615-619
• 4 O'Leary KJ, Wayne DB, Landler MP. , et al. Impact of localizing physicians to hospital units on nurse-physician communication and agreement on the plan of care. J Gen Intern Med 2009; 24 (11) 1223-1227
• 5 Mueller SK, Schnipper JL, Giannelli K, Roy CL, Boxer R. Impact of regionalized care on concordance of plan and preventable adverse events on general medicine services. J Hosp Med 2016; 11 (09) 620-627
• 6 Centers for Medicare & Medicaid Services (CMS), HHS. Medicare Program: changes to hospital outpatient prospective payment and ambulatory surgical center payment systems and quality reporting programs. Final rule with comment period. Fed Regist 2018; 83 (225) 58818-59179
• 7 Howard-Anderson J, Lonowski S, Vangala S, Tseng CH, Busuttil A, Afsar-Manesh N. Readmissions in the era of patient engagement. JAMA Intern Med 2014; 174 (11) 1870-1872
• 8 Bernacki R, Hutchings M, Vick J. , et al. Development of the serious illness care program: a randomised controlled trial of a palliative care communication intervention. BMJ Open 2015; 5 (10) e009032
• 9 Paladino J, Bernacki R. Precision communication—a path forward to improve goals-of-care communication. JAMA Intern Med 2018; 178 (07) 940-942
• 10 Geerse OP, Lamas DJ, Sanders JJ. , et al. A qualitative study of serious illness conversations in patients with advanced cancer. J Palliat Med 2019
• 11 Sanders JJ, Curtis JR, Tulsky JA. Achieving goal-concordant care: a conceptual model and approach to measuring serious illness communication and its impact. J Palliat Med 2018; 21 (S2): S17-S27
• 12 Curtis JR, Downey L, Back AL. , et al. Effect of a patient and clinician communication-priming intervention on patient-reported goals-of-care discussions between patients with serious illness and clinicians: a randomized clinical trial. JAMA Intern Med 2018; 178 (07) 930-940
• 13 Bernacki RE, Block SD. ; American College of Physicians High Value Care Task Force. Communication about serious illness care goals: a review and synthesis of best practices. JAMA Intern Med 2014; 174 (12) 1994-2003
• 14 Haberle TH, Shinkunas LA, Erekson ZD, Kaldjian LC. Goals of care among hospitalized patients: a validation study. Am J Hosp Palliat Care 2011; 28 (05) 335-341
• 15 Dalal AK, Bates DW, Collins S. Opportunities and challenges for improving the patient experience in the acute and postacute care setting using patient portals: the patient's perspective. J Hosp Med 2017; 12 (12) 1012-1016
• 16 Harle I, Karim S, Raskin W, Hopman WM, Booth CM. Toward improved goals-of-care documentation in advanced cancer: report on the development of a quality improvement initiative. Curr Oncol 2017; 24 (06) 383-389
• 17 Lamas D, Panariello N, Henrich N. , et al. Advance care planning documentation in electronic health records: current challenges and recommendations for change. J Palliat Med 2018; 21 (04) 522-528
• 18 Modes ME, Engelberg RA, Downey L, Nielsen EL, Curtis JR, Kross EK. Did a goals-of-care discussion happen? Differences in the occurrence of goals-of-care discussions as reported by patients, clinicians, and in the electronic health record. J Pain Symptom Manage 2019; 57 (02) 251-259
• 19 O'Leary KJ, Sharma RK, Killarney A. , et al. Patients' and healthcare providers' perceptions of a mobile portal application for hospitalized patients. BMC Med Inform Decis Mak 2016; 16 (01) 123
• 20 Griffin A, Skinner A, Thornhill J, Weinberger M. Patient portals: who uses them? What features do they use? And do they reduce hospital readmissions?. Appl Clin Inform 2016; 7 (02) 489-501
• 21 Woollen J, Prey J, Wilcox L. , et al. Patient experiences using an inpatient personal health record. Appl Clin Inform 2016; 7 (02) 446-460
• 22 Collins SA, Rozenblum R, Leung WY. , et al. Acute care patient portals: a qualitative study of stakeholder perspectives on current practices. J Am Med Inform Assoc 2017; 24 (e1): e9-e17
• 23 Kelly MM, Coller RJ, Hoonakker PL. Inpatient portals for hospitalized patients and caregivers: a systematic review. J Hosp Med 2018; 13 (06) 405-412
• 24 Couture B, Lilley E, Chang F. , et al. Applying user-centered design methods to the development of an mHealth application for use in the hospital setting by patients and care partners. Appl Clin Inform 2018; 9 (02) 302-312
• 25 Walker DM, Menser T, Yen PY, McAlearney AS. Optimizing the user experience: identifying opportunities to improve use of an inpatient portal. Appl Clin Inform 2018; 9 (01) 105-113
• 26 Kelly MM, Hoonakker PLT, Coller RJ. Inpatients sign on: an opportunity to engage hospitalized patients and caregivers using inpatient portals. Med Care 2019; 57 (02) 98-100
• 27 McAlearney AS, Fareed N, Gaughan A, MacEwan SR, Volney J, Sieck CJ. Empowering patients during hospitalization: perspectives on inpatient portal use. Appl Clin Inform 2019; 10 (01) 103-112
• 28 O'Leary KJ, Lohman ME, Culver E, Killarney A, Randy Smith Jr G, Liebovitz DM. The effect of tablet computers with a mobile patient portal application on hospitalized patients' knowledge and activation. J Am Med Inform Assoc 2016; 23 (01) 159-165
• 29 Greysen SR, Harrison JD, Rareshide C. , et al. A randomized controlled trial to improve engagement of hospitalized patients with their patient portals. J Am Med Inform Assoc 2018; 25 (12) 1626-1633
• 30 Masterson Creber RM, Grossman LV, Ryan B. , et al. Engaging hospitalized patients with personalized health information: a randomized trial of an inpatient portal. J Am Med Inform Assoc 2019; 26 (02) 115-123
• 31 Collins S, Dykes P, Bates DW. , et al. An informatics research agenda to support patient and family empowerment and engagement in care and recovery during and after hospitalization. J Am Med Inform Assoc 2018; 25 (02) 206-209
• 32 Shoffeitt M, Lanham H. Inpatient portals: the questions that remain. J Hosp Med 2018; 13 (06) 435-436
• 33 Fawole OA, Dy SM, Wilson RF. , et al. A systematic review of communication quality improvement interventions for patients with advanced and serious illness. J Gen Intern Med 2013; 28 (04) 570-577
• 34 Lakin JR, Block SD, Billings JA. , et al. Improving communication about serious illness in primary care: a review. JAMA Intern Med 2016; 176 (09) 1380-1387
• 35 Cooper Z, Koritsanszky LA, Cauley CE. , et al. Recommendations for best communication practices to facilitate goal-concordant care for seriously ill older patients with emergency surgical conditions. Ann Surg 2016; 263 (01) 1-6
• 36 Rosenberg LB, Greenwald J, Caponi B. , et al. Confidence with and barriers to serious illness communication: a national survey of hospitalists. J Palliat Med 2017; 20 (09) 1013-1019
• 37 Lakin JR, Koritsanszky LA, Cunningham R. , et al. A Systematic intervention to improve serious illness communication in primary care. Health Aff (Millwood) 2017; 36 (07) 1258-1264
• 38 Dalal AK, Dykes PC, Collins S. , et al. A web-based, patient-centered toolkit to engage patients and caregivers in the acute care setting: a preliminary evaluation. J Am Med Inform Assoc 2016; 23 (01) 80-87
• 39 Collins SA, Gazarian P, Stade D. , et al. Clinical workflow observations to identify opportunities for nurse, physicians and patients to share a patient-centered plan of care. AMIA Annu Symp Proc 2014; 2014: 414-423
• 40 Dykes PC, Stade D, Chang F. , et al. Participatory design and development of a patient-centered toolkit to engage hospitalized patients and care partners in their plan of care. AMIA Annu Symp Proc 2014; 2014: 486-495
• 41 Dykes PC, Stade D, Dalal A. , et al. Strategies for managing mobile devices for use by hospitalized inpatients. AMIA Annual Symposium Proceedings. AMIA Symposium 2015 ; 2015:522–531
• 42 Dalal AK, Roy CL, Poon EG. , et al. Impact of an automated email notification system for results of tests pending at discharge: a cluster-randomized controlled trial. J Am Med Inform Assoc 2014; 21 (03) 473-480
• 43 Dalal AK, Schnipper J, Massaro A. , et al. A web-based and mobile patient-centered “microblog” messaging platform to improve care team communication in acute care. J Am Med Inform Assoc 2017; 24 (e1): e178-e184
• 44 Wright AA, Zhang B, Ray A. , et al. Associations between end-of-life discussions, patient mental health, medical care near death, and caregiver bereavement adjustment. JAMA 2008; 300 (14) 1665-1673
• 45 Mack JW, Weeks JC, Wright AA, Block SD, Prigerson HG. End-of-life discussions, goal attainment, and distress at the end of life: predictors and outcomes of receipt of care consistent with preferences. J Clin Oncol 2010; 28 (07) 1203-1208
• 46 Khan A, Furtak SL, Melvin P, Rogers JE, Schuster MA, Landrigan CP. Parent-reported errors and adverse events in hospitalized children. JAMA Pediatr 2016; 170 (04) e154608
• 47 Collins SA, Couture B, Smith AD. , et al. Mixed-methods evaluation of real-time safety reporting by hospitalized patients and their care partners: the MySafeCare application. J Patient Saf 2018 Doi: 10.1097/PTS.0000000000000493