An Evaluation of System End-User Support during Implementation of an Electronic Health Record Using the Model for Improvement Framework

Abstract

Background Electronic health record (EHR) system implementation is complex. Strong support for clinicians and other end-users during the initial phase of implementation requires dedicated resources and commitment to ensure a smooth transition, reduce frustration, and assist in adoption.

Objectives Evaluate end-user support processes and personnel employed during the initial phase of EHR implementation at an academic medical center and identify facilitators of success, challenges, and lessons learned.

Methods Using the model for improvement framework, this case report describes the activities that follow the concepts of planning, doing, studying, and acting (PDSA) when providing on-site support to system end users during an EHR implementation in a complex health care system.

Results Strengths included engaged and supportive leadership, use of internal support personnel, use of zone leaders, daily huddles for effective dissemination of information, and an evaluation survey tool to provide data supporting rapid changes in support personnel allocation. Challenges primarily surrounded the management of over 1,000 external support personnel which included limited EHR system knowledge, scheduling, and transportation logistics.

Conclusion Implementation of EHR systems continues. Supporting end users is one aspect of these complex projects that require dedicated resources to manage effectively. Organizations taking a more proactive approach, based on learning from experiences, as well as from other health care organizations, can improve their ability to take on this challenge armed with best practices and lessons learned.

Keywords

implementation and deployment - clinical information systems - electronic health records and systems - leadership (enterprise level) - facilitators and barriers - adoption - evaluation - process improvement - satisfaction

Background and Significance

Electronic health record (EHR) system acquisition is one of the most important decisions a health care organization makes.[1] Not only is the financial investment substantial, but the change to the organization can be transformational. The 2018 report to Congress from the Office of the National Coordinator for Health Information and Technology (IT) indicated that as of 2015, 96% of hospitals and 78% of outpatient practices had implemented a certified EHR.[2] While these higher numbers might seem to indicate a slowing of EHR acquisition, the United States continues to see relatively high numbers of implementations. Two prominent EHR vendors, Epic and Cerner, report contracts for implementations that continue to number between 60 and 80 annually.[3] Although initial EHR system implementations may be decreasing, organizations with basic or homegrown systems are moving to certified systems, while other organizations are transitioning from one certified system to another. Even same system upgrades can be significant enough of a change that the efforts mimic initial deployment. These complex implementations and upgrades require an organization to address how clinicians and other system end-users are supported during the transition.

Literature on technology adoption, including the best practices to support end-users during initial implementation, continues to emerge. Without strong support to help clinicians and other system users to effectively adapt a new EHR, distrust, frustration, and slow adoption can result.[4] In an article by Boonstra and Broekhus, physicians perceived a need for proper support and were reluctant to use the EHR without it.[5] Simon et al, found that two-thirds of physicians indicated a lack of support as a barrier to them adopting EHRs.[6] In a recent study, conducted by Colicchio et al, 14 factors were identified that potentially impacted outcomes related to quality, productivity, and safety during and after a large commercial EHR implementation. One of the factors identified was the concept of a learning curve and points to the importance of having “technology champions” available to assist as a new paradigm is learned. They noted that the number of people allocated for implementation support could be a covariate and/or a moderator and impact clinicians' efficiency post–go-live, potentially contributing to longer lengths of stay, increased wait times, and a lower volume of visits.[7] A common theme in the literature emphasizes the need for providing on-site support during implementation and is seen as a critical factor to effective adoption.[4] [5] [7] [8] [9] [10] [11] [12]

Using the model for improvement, endorsed by the Institute for Healthcare Improvement as a framework, this case report will describe the activities that follow the planning, doing, studying, and acting of providing on-site support to clinicians and other system end-users at Vanderbilt University Medical Center (VUMC) during a recent EHR implementation. Activities and processes deemed as effective during the first 3 weeks of implementation will be shared along with challenges and lessons learned.

Background

VUMC is a large, complex health care system whose main campus located in Nashville, Tennessee comprised an adult hospital, a children's hospital, a behavioral health hospital, and a rehabilitation hospital with 1,131 licensed beds in total. In addition, VUMC has ambulatory clinics located on campus and in over 240 remote locations. Annually, they average 64,500 admissions, perform 57,000 surgical procedures, see 2 million patients in their ambulatory clinics, and 116,652 patients in their emergency departments.

On November 2, 2017, a new vendor-based EHR was implemented at all VUMC sites to replace a home-grown legacy system. The implementation impacted over 18,000 employees. Systems included in the implementation were inpatient and outpatient computerized provider order entry, clinical documentation, inpatient pharmacy, emergency department, labor and delivery, anesthesia/perioperative, ophthalmology, transplant, orthopaedics, oncology, billing and coding, patient access, population health, and a patient portal.

To ensure adequate on-site support during the initial weeks of implementation, or “go-live,” VUMC allocated dedicated personnel to lead the effort. The goal was to provide knowledgeable and helpful support personnel that would be available on-site, working side by side or “at the elbow” with system users as they learned the new EHR. VUMC wanted support personnel with the skills and expertise to respond quickly to help and answer questions regarding system use. Significant planning went into this effort and started 6 months prior to the go-live date.

Model for Improvement Framework

The first question, the model for improvement asks, is “what are you trying to accomplish?”[13] In this case, VUMC's efforts to support end-users with an adequate number of trained and skilled personnel during the initial phase of EHR implementation were the goal. End-user perception of the adequacy of support during implementation was a key measure in this effort. The following sections will describe the actions VUMC took in each phase of the model for improvement's concepts of plan, do, study, and act to evaluate effectiveness and the need for midcourse modifications to the plan to support end-users. Each phase is defined below for this initiative and is described in more detail in the sections that follow:

• Plan: planning activities conducted prior to implementation to prepare for the provision of on-site support to system end-users

• Do: executing the plan, including the activities performed during the initial 3 weeks of implementation of the new EHR

• Study: evaluating and monitoring the effectiveness of the planned on-site support provided to end-users including the identification of any unexpected issues

• Act: modifying and improving the plan as necessary

Plan

Established End-User Support Ratios

Working with organizational leaders and based on advice from industry consultants, ratios were established to determine the number of personnel needed to support users of the new system during initial implementation. See [Table 1] for the ratios used at VUMC.

Table 1 Estimated support personnel ratios

System users	Ratio
Providers (physicians, nurse practitioners, and physician assistants)	1:4
Clinical staff (nurses, medical assistants, case managers, social workers, physical therapy, occupational therapy, and respiratory therapy)	1:6
Nonclinical staff (researchers and medical records)	1:6
Back office and billing staff	1:8

Calculated Total Number of End-User Support Personnel Needed

Interviews were conducted with operational leaders at VUMC to determine the number of support personnel needed for each unit, clinic, and procedural area. Final numbers were based on information that included the number of employees at each location, times that the units, or clinics were open, the disciplines employed in each area and the shifts typically worked. Using the determined ratios, the total number of support personnel needed for around the clock support coverage during the initial implementation was identified at 1,636.

Identified Internal Support Personnel

Operational leaders were also asked to identify internal, direct care personnel, who could be removed from daily staffing to serve in a support role during implementation. Taking front line staff out of direct patient care activities proved to be a challenge, but a total of 552 staff members were allocated for initial support.

Identified External Support Personnel

An external agency was hired to identify, recruit, and schedule the additional support personnel needed. VUMC partnered with an external agency to recruit 1,084 support personnel needed to supplement the internal personnel. Recruitment was based on reported prior experience and system knowledge specific to the new vendor-based EHR. Arrangements for housing and transportation for these external personnel were made. [Table 2] represents the initial calculations for the overall number of internal and external support personnel needed.

Table 2 Calculated personnel needs for go-live support based on established ratios

Entity	Day	Day	Night	Night
	External support	Internal support	External support	Internal support
Offsite adult and children's ambulatory clinics	310	139	4	1
Behavioral health inpatient and ambulatory	17	15	6	3
Children's hospital inpatient	75	43	34	20
Children's hospital ambulatory	81	29	0	0
Adult hospital inpatient	202	120	106	52
Adult ambulatory	243	128	6	2
Grand total	928	474	156	78

Created End-User Support Schedules

The end-user support schedule was developed in Microsoft Excel. The department/unit names and physical locations, staffing mix, and hours of operation were preloaded into the schedule. End-user support personnel were scheduled and entered into the spreadsheet based on the information gathered during the initial operational leader interviews.

Identified and Scheduled Zone Leaders

Given the size of the medical center campus and the need to manage end-user support issues as early and as proactively as possible, VUMC developed a zone leader strategy. Zone leaders were personnel who were assigned to coordinate implementation support efforts for a defined set of areas or “zones” across the organization. Zone leaders served as the communication liaison between the front-line support personnel and the end-user support command center established to monitor support activities. A total of 91 zones were established and close to 200 zone leaders were identified, recruited, and scheduled in 12-hour shifts to provide round-the-clock coverage for the first 3 weeks of implementation.

Established a Centralized Command Center

A centralized command center was established to prioritize and manage issues related to end-user support personnel during the implementation. Activities for this center included managing calls related to end-user support concerns, adjusting end-user support schedules based on feedback, and providing updated information to zone leaders. Total staffing was to consist of two VUMC employees and five leaders from the external agency. The external agency would take the lead in managing the center. The 10 feet × 10 feet space located in the corner of an open area, consisted of one landline phone, one small table, two chairs, and a projection monitor to track issues as they were being called into the command center.

Developed Communication Plan with Zone Leaders

A communication plan was established between the command center and the zone leaders. Bidirectional communication was the goal as command center leaders could disseminate system issues through the zone leaders and the zone leaders could escalate issues arising in their respective front-line areas to the command center. Zone leader meetings called “huddles” were scheduled for the beginning and end of each shift where key information was shared and phones were distributed. Two designated huddle locations were identified and remote access was established for off-site zone leaders to call in.

Training Scheduled and Conducted for End-User Support Personnel

End-user support personnel were scheduled to attend a 6-hour training class in designated training classrooms on campus and at satellite locations. In addition to providing EHR system training on workflows specific to the areas being supported, information was shared on role responsibilities, schedules, and communication and escalation processes with zone leaders. All support personnel received their schedules during training which included the physical address and date and time for their assigned support location.

Developed an End-User Support Evaluation Tool

To evaluate the overall status of implementation activities, and to evaluate for any unexpected issues, an electronic survey tool was developed to collect quantitative and qualitative data. The survey consisted of eight questions and measured the following: adjusting to new workflows, biggest challenges during the shift (hardware or new system navigation), effectiveness of external support personnel, readiness to release external support, and overall satisfaction of system end-users. Responses evaluated for this case study included those related to external support personnel and overall satisfaction of system end-users. Evaluated, as well were responses, to the question, “is there any additional feedback you would like to share?” See [Table 3] for the ratings evaluated in this report.

Table 3 Support evaluation survey

Ratings	Likert's scale scoring (1–5)
• Rate: overall effectiveness of external support personnel	1 = Not effective at all 2 = Slightly effective 3 = Moderately effective 4 = Very effective 5 = Extremely effective
• Rate: readiness to release external support personnel	1 = Definitely not 2 = Probably not 3 = Getting better, but not yet 4 = We still require support, but can consider reductions 5 = Definitely yes
• Rate: overall satisfaction level of system end-users	1 = Very dissatisfied 2 = Not satisfied 3 = Neutral 4 = Satisfied 5 = Very satisfied
• Is there any additional feedback you would like to share?	Free text response

Distributed End-User Support and Zone Leader Identifying Vests

During training sessions, end-user support personnel were issued yellow vests. Zone leaders were issued red vests. This was to provide a visual indicator to system end-users so they would know who to reach out to for assistance.

Do

Internal and External End-User Support Personnel Deployed

One hour prior to the new EHR system go-live, internal support personnel arrived at their typical work locations to provide help with system use and navigation supporting the workflows in their department. External support personnel arrived on campus via charter buses 2 hours prior to the go-live and were directed by the external agency's logistics team. This team directed over 600 end-user support personnel to their assigned areas.

Zone Leader Rounding and Twice Daily Huddles Conducted

Zone leaders began the day at their huddle location to pick up support personnel schedules for their shift and other key information needed to support the sites within their zone. Zone leaders managing remote locations called into the huddle and received information electronically. Throughout their shifts, zone leaders made rounds and met with their respective front-line internal and external support personnel to discuss key issues. They communicated any problems to the command center. Zone leaders also shared updates from the command center during rounding with their front-line end-user support staff.

End-User Support Evaluation Tool Completed

As part of their responsibilities, zone leaders completed the end-user support evaluation survey before the end of each shift and submitted it to the command center. There were, on average, 304 completed surveys submitted each day for the initial 3 weeks of post–go-live. The total number of completed surveys during these first 3 weeks of implementation was 4,243.

Study

Evaluation of Support Personnel Scheduling Process

The scheduling process for end-user support personnel was complex and managing shifts for over 1,600 end-user support personnel in an Excel spreadsheet proved to be challenging. While the spreadsheet was used to manage overall go-live support scheduling, assignments for external support personnel were communicated by the external agency in a separate system. The schedules between these two systems were not synchronized, leading to instances where support personnel did not show up for their shift or they showed up to the wrong location.

Evaluation of External Support Personnel

Based on training assessments prior to implementation and early feedback from zone leaders, operational leaders, and front-line clinicians, it was determined that the knowledge level of many of the external support personnel was not adequate enough to provide assistance to system end-users. Many departments reported that they did not have support personnel with the right skillset in their areas to address their needs.

Evaluation of Internal Support Personnel

Zone leaders and front-line clinicians reported consistently throughout the organization that the internal support personnel were highly valuable resources. They were the preferred resource to provide support due to their departmental knowledge of the people, processes, and technology.

Evaluation of Zone Leaders and Huddles

The role of zone leader was identified as a strategy integral to the success of the implementation. These volunteers were a core resource contributing to the strength of the program to support end-users during the initial phase of implementation. They performed their role of communicator and disseminator of information effectively. The huddles they conducted twice each shift proved effective for communicating implementation updates with end-user support personnel and for escalating end-user support challenges to the command center. This structure provided a strong mechanism for bidirectional dissemination of information throughout a broad and complex network.

Survey Evaluations

The survey results at the end of the first day of go-live for overall effectiveness of external support personnel revealed a rating of 1.87 on a 5-point scale. Interestingly, the readiness to release these same personnel was rated at 0.96, indicating that end-users were not yet ready to let them go. Overall satisfaction level of system end-users, showed a first day rating of 1.90. [Fig. 1] shows the trending of these parameters during the first 16 days of go-live. In addition to the quantitative data, over 600 free-text responses were submitted containing additional feedback related to external support personnel (inadequate skillset, not enough personnel available). A representative comment received, stated, “external staff lack a working knowledge of the new EHR, which creates quite a bit of frustration among the staff and providers.”

Fig. 1 Support evaluation survey tool results.

Act

Schedule Improvements

Given the challenges with managing two schedules, a decision was made by the VUMC command center leaders to utilize the Excel spreadsheet as the one and only source for support personnel scheduling. The system used by the external vendor could still be used for communication purposes, but it was the Excel spreadsheet that became the source of truth for all assignments. The spreadsheet was projected on a large screen monitor in the command center and updated in real-time as adjustments were made.

External Support Personnel Adjustments

Using input from surveys completed by zone leaders and feedback from VUMC operational leaders, external staffing was reduced in areas where support personnel were not providing value. There were many external support personnel who were unable to assist with basic support needs or answer end-user questions. This reduction in force started prior to go-live during training sessions where trainers identified a lack of core EHR knowledge amongst some of the external support personnel. A total of 300 external support personnel were released during training sessions and an additional 250 were released during the first 5 days of implementation. As a result of this unforeseen reduction, adjustments were made to the schedule based on areas that had the highest need for support (i.e., high-volume ambulatory clinics, procedural areas, and high-acuity units). With modifications made, within 2 weeks, satisfaction scores of the external support personnel increased to an average rating of 3.64 with a concomitant increase in end-user overall satisfaction from 1.90 to 3.55 ([Fig. 1]).

Command Center Space Modifications

On day 2 of go-live, the end-user support command center relocated from a small corner in an open area to a larger, dedicated room. Three additional phone lines were established to manage the volume of incoming phone calls, and four additional VUMC command center personnel were added to staff the command center around the clock. On the second day of go-live, overall leadership of the command center transitioned from the external agency to VUMC.

Dedicated Rapid Response Team Established

A dedicated team of support personnel was established in the third week post implementation to respond to immediate needs that arose throughout the organization. This rapid response team consisted of personnel from the EHR vendor and the most skilled external agency personnel, based on feedback from zone leaders and system users. This team responded to different locations throughout the medical center based on zone leader requests, as well as requests called into the command center. They also provided one-on-one training sessions for clinicians requiring additional support.

Discussion

The model for improvement's framework helped to describe and evaluate the activities to support system end-users during the initial phase of implementation of a new EHR. Conceptualizing the work using the categories of plan, do, study, and act helped to frame the evaluative process and identify why improvements were needed in several areas. It also provides valuable information for VUMC, and other organizations, to use in future implementations, as they take on the task of supporting end-users during system implementation.

VUMC leadership engagement and support throughout the planning and implementation period ensured clarity of roles, strong communication practices and ultimately a successful outcome. This support contributed to the ability to make agile and sometimes difficult decisions in a timely manner. Recognizing the challenges that arose, VUMC would have taken a more proactive approach in several areas. Understandably, issues arose from the management of over 1,000 external support personnel. The three main challenges included knowledge gaps, scheduling issues, and transportation logistics. To address these issues in the future, VUMC identified, first, that planning should include an open and transparent relationship with the external support vendor beginning with initial negotiations. Organizations needing to supplement their support with external personnel need to understand that unique system customizations may reduce the knowledge level of external personnel brought in to assist. However, organizations should take an active role in completing spot audits on the selected pool of candidates to ensure a minimum set of core navigational skills for the system being supported. Second, discussions during the planning phase on how a single centralized schedule will be managed and maintained, including the process for communicating assignments with support personnel, is another important lesson learned. And third, it would be an understatement to say that to house and transport the large number of external personnel for round-the-clock coverage was logistically complicated. Organizations should take more of a hands-on approach to validate planning in terms of transportation logistics, orientation to physical locations, and dissemination of maps clearly marked with important locations. Reviewing logistical details during the early planning stages with the external agency including housing locations and transportation to and from support sites would have reduced much of the confusion that VUMC experienced.

Unintended consequences during an implementation are often blamed on the technology itself. However, if viewed through a sociotechnical lens, contributors can include issues with workflows, organizational culture, social interactions, and the human resources used to support clinicians.[14] VUMC observed several nontechnical problems emerge that required action as support personnel attempted to assist areas with unique and customized workflows. The leadership team found themselves involved in an ongoing formative evaluation for several weeks as unintended issues arose. In hindsight, these support efforts may have benefitted from the use of an interactive sociotechnical analysis instead of what might have appeared to be more of a fire fighting approach. Additional research that focuses on the adequacy of go-live support personnel in relation to outcomes, such as quality, productivity, and safety would add value and strengthen the evidence for best practices. Identified recommendations based on lessons learned at VUMC have been captured in [Table 4] and can be instituted proactively for future implementations.

Table 4 Recommendations based on lessons learned

Activity	Recommendations
1. Identify and schedule zone leaders	• Establish a zone leader structure to function as communicator and disseminator of information • Identify internal personnel within the organization that can work in the role of zone leader during the initial phase of implementation • Define zone leader roles and responsibilities
2. Establish dedicated end-user support command center	• Establish and locate command center within proximity of the centralized operational command center • Staff with organizational personnel supplemented by external agency (if used) • Provide phone lines, electronic displays, computers and whiteboards to support bi-directional communication channels and support high volume incoming calls
3. Establish process for daily rounding and twice daily huddles	• Provide zone leaders with end user support schedules specific to the zones being supported • Conduct twice daily huddles in a centralized location for all zone leaders • Organize huddle agenda around key communication points from operational command center and issues gathered during daily rounding • Zone leaders share key updates with end user support personnel during daily rounding
4. Develop end-user support evaluation tool and leverage data to adjust support needs	• Develop evaluation tool during the planning stages of implementation • Establish questions that address issues related to system use and effectiveness of end user support personnel • Review results at the end of each shift and modify support plan based on feedback
5. Establish rapid response team for end-user support issues	• Establish rapid response team, consisting of highly skilled support personnel • Develop communication plan and deployment strategy as part of the planning process • Manage team from the end user support command center and deploy to different areas based on highest needs
If external support personnel are used
6. Manage external support personnel: transportation logistics	Review details of the transportation plan to include: • Hotels/locations where personnel will be housed • The number of buses needed to transport personnel from hotel to support locations • Drop off locations for support personnel for their shift • Navigation logistics to guide personnel from drop off to support location including onsite resources to help with directing • Conducting practice drill prior to implementation to evaluate effectiveness of logistics plan
7. Manage external support personnel: knowledge gaps	• Review selected pool of candidates to validate their experience and skillset • Develop competency checklist and have personnel complete an onsite pre-training competency assessment to determine foundational system knowledge • Establish training curriculum • Validate that personnel attend training course relevant to the area(s) supported • Stay engaged in training activities during the early stages of training
8. Manage external support personnel: scheduling resources	• Review and agree upon the scheduling tool during the early planning phases • Test the scheduling tool prior to implementation • Review process for communicating schedules to personnel and integration with scheduling tool

The formation of a dedicated command center for end-user support was a decision that proved to be a successful strategy for the VUMC leadership team, zone leaders, and support personnel. This center was eventually physically located adjacent to the enterprise-level operational command center, ensuring direct communication with executive leaders. Also proving valuable was the ability to leverage data from the survey tool. Daily results from the survey allowed the leadership team to modify and update support priorities for the next shift and adjust schedules based on the effectiveness of the personnel. Additionally, the established rapid response team, consisting of highly skilled personnel, became a strength identified as one that should have been instituted earlier than 3 weeks of post–go-live. This roving band of experts proved to be one the most valued resources for the clinicians. And finally, the roles of zone leaders and internal support personnel with strong communication processes have been added to the list of strengths of the implementation. These resources in their roles provided the foundation that ultimately led to stabilization at the front lines as system end-users navigated their new cognitive world in care delivery.

Conclusion

Implementation of EHR systems continue. Supporting end-users is one aspect of these complex projects that requires dedicated personnel to manage effectively. Organizations taking a more proactive approach, based on learning from experiences and from other health care organizations, can improve their ability to take on this challenge armed with best practices and lessons learned.

Clinical Relevance Statement

The review of strengths, challenges, and lessons learned from this implementation emphasizes the importance of dedicating resources to address how end-users are supported during the go-live to reduce the negative impact of system issues and delays in system adoption. Strong leadership engagement and a solid understanding of external personnel recruitment, implementation support needs, scheduling, and operational logistics will ensure that the efforts align with the needs of the organization.

Multiple Choice Questions

1. Identify an important strategy in the development of end-user support for an EHR implementation:

   • Planning months in advance and including proactive strategies.

   • Hiring an external agency to direct all go-live activities.

   • Creating an off-site command center.

   • Providing training of staff on the day of go-live to ensure knowledge retention.

   Correct Answer: The correct answer is option a. Months of planning are needed to develop a strong implementation plan to support end-users during a new EHR system to go-live. The multiple aspects of planning require attention months in advance to develop a sound plan based on the best practices. The identification and acquisition of resources including funding, space, and personnel take time and dedicated organizational leadership. Employing proactive strategies to improve preparedness for unexpected issues needs to be included in planning to support quick decision making if needing to make improvements throughout the initial implementation of the system.

2. What is a key factor for ensuring a smooth transition to the new EHR and system adoption?

   • Training executive leaders on system use.

   • Use of a strong theoretical framework to develop the end-user support plan.

   • Organizational leadership commitment and engagement.

   • Providing off site housing for external support personnel within walking distance of support areas.

   Correct Answer: The correct answer is option c. The engagement of executive and operational leaders, all participating as team members throughout the implementation, is essential. With supportive leadership assisting in the decision-making process, it provides the necessary authority to carry through with needed actions for improvement.

Conflict of Interest

None declared.

Acknowledgments

The authors would like to thank and acknowledge VUMC Executive Leadership for their support and the countless number of VUMC faculty and staff that volunteered their time to help support the go-live in many different capacities.

Protection of Human and Animal Subjects

This work falls into the category of nonresearch/quality improvement and IRB approval was not required.

• References

• 1 Shahmoradi L, Darrudi A, Arji G, Farzaneh Nejad A. Electronic health record implementation: A SWOT analysis. Acta Med Iran 2017; 55 (10) 642-649
• 2 Office of the National Coordinator for Health Information Technology. 2018 Report to Congress: Annual update on the adoption of a nationwide system for the electronic use and exchange of health information. Available at: https://www.healthit.gov/sites/default/files/page/2018-12/2018-HITECH-report-to-congress.pdf . Accessed December 5, 2019
• 3 Research KLAS. U.S. hospital EMR market share 2017. Available at: https://klasresearch.com/report/us-hospital-emr-market-share-2017/1182 . Accessed December 5, 2019
• 4 Shachak A, Montgomery C, Dow R. , et al. End-user support for primary care electronic medical records: a qualitative case study of users' needs, expectations and realities. Health Syst (Basingstoke) 2013; 2 (03) 198-212
• 5 Boonstra A, Broekhuis M. Barriers to the acceptance of electronic medical records by physicians from systematic review to taxonomy and interventions. BMC Health Serv Res 2010; 10 (01) 231
• 6 Simon SR, Kaushal R, Cleary PD. , et al. Physicians and electronic health records: a statewide survey. Arch Intern Med 2007; 167 (05) 507-512
• 7 Colicchio TK, Borbolla D, Colicchio VD. , et al. Looking behind the curtain: identifying factors contributing to changes on care outcomes during a large commercial EHR implementation. EGEMS (Wash DC) 2019; 7 (01) 21
• 8 Jaén CR. Successful health information technology implementation requires practice and health care system transformation. Ann Fam Med 2011; 9 (05) 388-389
• 9 Bossen C, Jensen LG, Udsen FW. Evaluation of a comprehensive EHR based on the DeLone and McLean model for IS success: approach, results, and success factors. Int J Med Inform 2013; 82 (10) 940-953
• 10 Novak LL, Anders S, Gadd CS, Lorenzi NM. Mediation of adoption and use: a key strategy for mitigating unintended consequences of health IT implementation. J Am Med Inform Assoc 2012; 19 (06) 1043-1049
• 11 O'Connell RT, Cho C, Shah N, Brown K, Shiffman RN. Take note(s): differential EHR satisfaction with two implementations under one roof. J Am Med Inform Assoc 2004; 11 (01) 43-49
• 12 Myers K. At-the-elbow physician support a must-have for EHR implementations. Available at: http://searchhealthit.techtarget.com/healthitexchange/CommunityBlog/at-the-elbow-physician-support-a-must-have-for-ehr-implementations/ . Accessed December 5, 2019
• 13 Institute for Healthcare Improvement. How to improve. Available at: http://www.ihi.org/resources/Pages/HowtoImprove/default.aspx . Accessed December 5, 2019
• 14 Harrison MI, Koppel R, Bar-Lev S. Unintended consequences of information technologies in health care--an interactive sociotechnical analysis. J Am Med Inform Assoc 2007; 14 (05) 542-549