Resident Physician Experience and Duration of Electronic Health Record Use

Abstract

Background Electronic health records (EHRs) demand a significant amount of physician time for documentation, orders, and communication during care delivery. Resident physicians already work long hours as they gain experience and develop both clinical and socio-technical skills.

Objectives Measure how much time resident physicians spend in the EHR during clinic hours and after-hours, and how EHR usage changes as they gain experience over a 12-month period.

Methods Longitudinal descriptive study where participants were 622 resident physicians across postgraduate year cohorts (of 948 resident physicians at the institution, 65.6%) working in an ambulatory setting from July 2017 to June 2018. Time spent in the EHR per patient, patients records documented per day, and proportion of EHR time spent after-hours were the outcome, while the number of months of ambulatory care experience was the predictor.

Results Resident physicians spent an average of 45.6 minutes in the EHR per patient, with 13.5% of that time spent after-hours. Over 12 months of ambulatory experience, resident physicians reduced their EHR time per patient and saw more patients per day, but the proportion of EHR time after-hours did not change.

Conclusion Resident physicians spend a significant amount of time working in the EHR, both during and after clinic hours. While residents improve efficiency in reducing EHR time per patient, they do not reduce the proportion of EHR time spent after-hours. Concerns over the impact of EHRs on physician well-being should include recognition of the burden of EHR usage on early-career physicians.

Keywords

electronic health records - medical education - resident physicians

Background and Significance

Following the passage of the Health Information Technology for Economic and Clinical Health Act of 2009[1] and the widespread proliferation of electronic health records (EHRs),[2] [3] there has been growing concern over the impact of computers, information technology, and so-called “desktop medicine” on the health, well-being, and efficiency of physicians.[4] [5] [6] [7] [8] Recent studies have found strong associations between EHR use and burnout,[9] [10] especially time spent in the EHR after-hours (i.e., “pajama time”).[11] At the same time there has been growing concern over the number of hours worked by resident physicians during their training, resulting in the American College of Graduate Medical Education regulating the number of hours residents can work.[12] [13] While the impact of high weekly hours on quality of care is still debated, it is undeniable that heavy workloads take a physical, mental, and emotional toll on resident physicians.[14]

During their time as residents, physicians are developing clinical and socio-technical skills simultaneously.[15] They must learn not only the practice of medicine and how they fit within their organizational culture, but also how to navigate and work with the EHR—complex, specialized software infamous for poor usability and a steep learning curve.[16] Residents are taxed mentally on constant basis with demands on their time and attention coming from a variety of disparate sources.[17] Coupled with their high workload, concerns over EHR-driven burnout may be especially salient for resident physicians as they may be learning a new system. However, little is known about how resident physicians' EHR usage specifically, or how it changes as they gain experience.[18] [19] [20] [21] Residents may face a large EHR time burden as they attempt to quickly learn multiple systems at once, or it may be the case that young, technologically savvy residents are efficient in the EHR and are able to minimize their after-hours time.[22] While recent studies have shown variation across physician specialty,[23] there is little evidence to date about how physician EHR usage varies by experience or changes over time as they become more facile with the system. Given the high demands on residents' time and the increasingly strong base of evidence linking EHR usage to reduced physician job satisfaction and its concomitant burnout, it is critical to understand how this population of early-careerists interacts with the EHR.

In this study, we used detailed EHR audit log metadata following resident physicians in ambulatory settings to address three research questions. First, how much time do resident physicians spend working in the EHR? Second, what proportion of that time do resident physicians spend working in the EHR after-hours? And third, how does resident physician EHR usage change as they gain experience?

Data and Methods

Data

We used de-identified EHR audit log data collected from the Lights On Network reporting system, which tracks and reports user activity in detail from users of Cerner's Millennium EHR system (Cerner Corporation, North Kansas City, Missouri, United States). Lights On records user time spent in all clinical and nonclinical activities through the EHR system, and reports data aggregated at the encounter level. This system records all interactions with the EHR software including keystrokes, mouse movement, and clicks, and where the physician is within the system (such as documentation, orders, or chart review.) The system measures active time using prespecified, proprietary guidelines, which are detailed in-depth by Overhage and McCallie (2020).[23] All variables in the study come from the Lights On reporting system. The Lights On reporting system for this study included only ambulatory care EHR use, and excluded any inpatient or emergency EHR use.

Our study design was a longitudinal, descriptive study with two-way fixed effects that included Lights On audit log data for 622 resident physicians practicing in ambulatory care settings at the University of Alabama at Birmingham Health System, a large academic medical center located in Birmingham, Alabama, for the 12-month period between July 2017 and June 2018. This includes residents across multiple postgraduate year cohorts. This Lights On data reports only the amount of time spent using the ambulatory care portion of the EHR, and excludes any inpatient or emergency EHR time if a resident also spent time in those service areas during the month. We aggregated the data up to the physician-month level for our unit of analysis. We limited our sample to physician-month observations with at least five patients seen that month, as not all resident physicians practice in ambulatory care settings each month, and we wanted to exclude situations where a resident on inpatient or emergency service charted one or two follow-up encounters using the ambulatory EHR—this was our only exclusion criterion, and of the 948 residents at the institution, 622 (65.6%) met the inclusion criteria for our study. Our final analytic dataset consisted of 3,703 physician-month observations. Due to the de-identified nature of our data, we were unable to provide detailed demographic information for our study sample. The overall population of residents at the University of Alabama at Birmingham Health System is 34.4% female and 65.6% male, 5.8% Hispanic or Latinx, 18.1% Asian, 4.8% Black or African-American, 0.4% American Indian or Alaskan Native, and 76.6% White.

Our study was designated exempt by the Institutional Review Board of the University of Alabama at Birmingham Health System.

Measures

We defined three measures of EHR use and physician productivity as our primary outcome variables of interest. First was time spent in the EHR per patient encounter, which is the total amount of time a physician spends working in the EHR for a specific patient encounter, inclusive of all activities the physician performed. Our second measure was number of patients seen per day, a count of the number of unique patients seen each day where the physician signed a note in the EHR (resident physicians often sign notes, but this task may sometimes fall to the attending physician instead, which may result in our under-counting patient encounters). Our third measure was the percentage of EHR time spent after-hours. We defined after-hours time as any time spent working in the EHR, on any activity, between the hours of 6:00 p.m. and 6:00 a.m. These definitions of EHR time per patient, patient encounters per day, and after-hours time are consistent with existing studies on EHR audit log metadata, allowing us to make comparisons to the literature.[23]

Our primary predictor variable of interest was the number of months of ambulatory practice experience during the 12-month study period. Each month a physician in the sample had at least five encounters in an ambulatory setting was counted as a month of experience. Residents in the data were assigned a dummy identifier number so we could track them longitudinally in a de-identified setting.

Analysis

We first calculated descriptive statistics for our three outcome variables as well as our sample characteristics. We then plotted the mean of those three outcome variables by month of ambulatory care experience as line graphs. Finally, we created three ordinary least squares (OLS) regression models with each of our outcome variables as the dependent variable in a model, and number of months of ambulatory care experience as our independent variable of interest. All models were adjusted for physician and calendar month fixed effects to account for any time-invariant omitted variable bias from either resident physician characteristics such as age, gender postgraduate year, prior experience with the EHR, as well as potential seasonality, with robust standard errors clustered at the physician level. For a robustness test we also included models using cumulative patients seen during the study period as our independent variable of interest to measure ambulatory experience. We also estimate our measure of after-hours EHR time using raw minutes after-hours rather than percentage of time. Finally, we estimate our two-way fixed effects model on each individual component of EHR use (documentation, chart review, orders, and other). Full regression results are available in [Appendix Table A1]. All calculations were done using Stata version 16 (StataCorp, College Station, Texas, United States).

Appendix Table A1 Full regression results

	Patients seen per day			Time per patient in EHR (hours)			% EHR time after-hours
	Coef.	p-Value	[95% Confidence interval]	Coef.	p-Value	[95% Confidence interval]	Coef.	p-Value	[95% Confidence interval]
Months of ambulatory experience	0.009	0.389	−0.012 to 0.030	−0.7181524	<0.001	−0.019 to −1.154578	−0.2817266	0.592	−0.242 to 0.138

Note: Based on ordinary least squares regressions with individual and month fixed effects and robust standard errors clustered at the individual physician level. Includes controls for resident physician specialty not shown. N = 622 resident physicians from July 2017 to June 2018.

Results

In our sample, the mean number of months of physician ambulatory care experience during the 12-month study period was 4.8. Residents spent an average of 45.6 minutes in the EHR per patient. Most time was spent on documentation (20.2 minutes), chart review (13.8), and entering orders (6.5,) with the remaining time distributed between tasks such as patient discharge, medication reconciliation, and messaging with patients. Residents spent 13.55% of their EHR time after-hours. The mean number of patients seen per day was 2.97 ([Table 1]).

Table 1 Sample descriptive statistics of resident physician EHR use

	Mean	Standard deviation
Number of months of ambulatory care experience during the sample	4.78	3.09
Electronic health record time
EHR time per patient (min)	45.89	49.86
EHR time: documentation (min)	20.20	26.57
EHR time: chart review (min)	13.80	15.00
EHR time: orders (min)	6.50	6.69
EHR time: other functions (min)	5.39	8.40
Number of patients seen per day	2.97	1.61
Percentage of EHR time spent after hours (%)	13.55	14.90

Abbreviation: EHR, electronic health record.

Notes: Based on residents from July 2017 to June 2018.

When evaluating how residents learn through experience, we found EHR time per patient fell from 48.2 minutes in their first month to 40.9 minutes in their 12th, patients seen per day increased from 2.69 to 3.39, and percentage of EHR time after-hours increased from 15.67 to 16.51% ([Fig. 1]).

Fig. 1 Resident physician EHR usage by months of experience. EHR, electronic health record.

In our multivariate OLS regression models, we found that each month of experience was correlated with a significant reduction in minutes of EHR time per patient (β = −0.72 minutes, p < 0.001). The relationship between experience and patients seen per day and proportion of after-hours time was not significant ([Table 2]). Our robustness test using cumulative ambulatory encounters as an alternative measure of experience found similar results to our main specification ([Appendix Table A2]). Similarly, we find no effect of experience on after-hours time as measured by number of minutes ([Appendix Table A3]). Finally, in our models examining EHR time per patient across the components of EHR functions, we found each month of experience during the study period was associated with decreased time spent in documentation (β = −0.46 minutes, p < 0.001) and other functions such as messaging (β = −0.11 minutes, p < 0.001), but found no effect on chart review or orders. ([Appendix Table A4]).

Table 2 Association between months of experience and EHR use in resident physicians

	Time per Patient in EHR (min)		Patients seen per day		% EHR time after hours
	Coef.	p-Value	Coef.	p-Value	Coef.	p-Value
Months of ambulatory experience	−0.72	<0.001	0.01	0.389	−0.05	0.60

Abbreviation: EHR, electronic health record.

Notes: Based on ordinary least squares regressions with individual physician and calendar month fixed effects with robust standard errors clustered at the individual physician level. Includes controls for resident physician specialty not shown. N = 622 resident physicians from July 2017 to June 2018. Full regression results available in [Appendix Table A1].

Appendix Table A2 Construction of experience as cumulative ambulatory patients seen during study period

	Patients seen per day			Time per patient in EHR (min)			% of EHR time after-hours
	Coef.	p-Value	[95% Confidence interval]	Coef.	p-Value	[95% Confidence interval]	Coef.	p-Value	[95% Confidence interval]
Cumulative ambulatory encounters	0.0006	0.037	0.0000–0.0012	−0.017	<0.001	−0.025 to −0.011	−0.001	0.657	−0.007 to 0.004

Note: Based on ordinary least squares regressions with individual and month fixed effects and robust standard errors clustered at the individual physician level. Includes controls for resident physician specialty not shown. N = 622 resident physicians from July 2017 to June 2018.

Appendix Table A3 Association between resident physician months of experience during the study and after-hours EHR time per month

	After-hours EHR time (min)
	Coef.	p-Value	[95% Confidence interval]
Cumulative ambulatory encounters	−1.593	0.139	−3.70 to 0.518

Note: Based on ordinary least squares regressions with individual and month fixed effects and robust standard errors clustered at the individual physician level. Includes controls for resident physician specialty not shown. N = 622 resident physicians from July 2017 to June 2018.

Appendix Table A4 Association between resident physician months of experience during the study and EHR time per patient across component functions

	Independent variable: additional month of ambulatory experience
EHR function	Coef.	p-Value	[95% Confidence interval]
Documentation	−0.46	<0.001	−0.69 to −0.22
Chart review	−0.09	0.19	−0.22 to 0.04
Orders	−0.05	0.08	−0.11 to 0.01
Other functions	−0.11	<0.001	−0.18 to −0.04

Note: All coefficients represent minutes of EHR time per patient. Based on ordinary least squares regressions with individual and month fixed effects and robust standard errors clustered at the individual physician level. Includes controls for resident physician specialty not shown. N = 622 resident physicians from July 2017 to June 2018.

Discussion

Resident physicians clearly demonstrated a pattern of increased efficiency in EHR usage as they gained work experience. There are two major findings with regards to how resident physicians learn to use the EHR over time and how that translates into workloads. As might be expected of professionals working at the highest level, residents improved over a relatively short timeframe. However, the proportion of EHR time that occurred after-hours did not decrease, suggesting that residents did not seek to minimize their “pajama time” as they became more facile with the EHR.

Being able to learn, retain, and act on large amounts of information is a sine qua non of being a successful medical student and resident. Therefore, it is not surprising that this sample was able to significantly streamline the amount of time they used the EHR to carry out their clinical rotations. Resident physicians often have academic backgrounds that lend themselves to critical and analytic thinking. However, this experience may not be useful in mastering EHR technologies, and our results show that residents spend more time per patient in the EHR compared with previous studies of physicians more broadly.[23] [24] EHRs are notorious for poor usability and lack of intuitive navigability.[16] [25] The effect size of our results provides some context; however, while we find a statistically significant impact of months of experience on EHR time per patient, the effect size is not large—a reduction in 0.72 minutes per patient for each month of experience. Assuming three patients per day, the result is only 2.16 minutes of time savings per day. However, if that time savings translate to a larger volume of daily encounters, it could result in a significant quality-of-life improvement. This is a much smaller effect compared with the differences between physicians practicing in different national health systems or for attending physicians in surgical versus medical specialties or pediatric versus adult care.[23] [26] [27] [28]

Our models adjusting for physician and month fixed effects found no impact of experience on number of patients seen per day. Residents are strongly incentivized to see as wide a variety of conditions as possible to gain valuable experiences, and the level of patient volume they handle may not be fully within their control. While our unadjusted results show an increase in patients seen per day as residents gain experience, it may be that they are asked to do more by more senior residents or attending physicians, and EHR time per patient necessarily decreases as their clinical workload increases. This could in turn reflect a decrease in the quality of EHR documentation that we were unable to observe in our data.[29]

Finally, our results make it clear that despite residents' increasing competency with the EHR, the proportion of that time occurring after-hours remained constant. We also found no effect of additional months of experience on raw after-hours time. Despite research indicating that this after-hours work is draining and associated with physician exhaustion and burnout,[11] [30] resident physicians do not seem to be translating EHR efficiency gains into less time working at home. This may reflect a reality that after-hours EHR work is viewed as a necessary and normal part of the practice of medicine, and early-career physicians are not encouraged to reduce or avoid it. Given increasing rates of physician burnout and turnover that is at least partially attributable to the burden of EHR “pajama time,” this is a concerning result. A culture of medicine that necessitates extensive after-hours EHR time may have a more difficult time in improving clinician wellness even if EHR usability improves. However, it may simply be that residents are unable to reduce their after-hours time despite gaining more EHR proficiency. Future research should explore how resident physicians use the EHR outside of clinic hours in more depth.

This article has important limitations. First, data were drawn from a single academic medical center, and we are unable to guarantee that our data generalize to other sites, though we do not believe our study setting is meaningfully different from other teaching hospitals. Additionally, the data are from a single EHR vendor employing proprietary metrics,[31] [32] such as defining after-hours time as 6:00 p.m. to 6:00 a.m., which may not be appropriate for all clinical settings. EHR metadata are a powerful tool, but future studies should also supplement with methods including direct observation and qualitative data collection to capture a holistic view of physician EHR work. Second, as stated above, we observe changes in EHR usage but are unable to determine the causal reason for those changes, and individuals may differ greatly in the effectiveness of their EHR use. Third, we only have sufficient data to describe EHR usage patterns in ambulatory settings—it is possible that EHR use and resident physician learning patterns are different in inpatient settings. Fourth, due to the de-identified nature of our data, we do not have data on physician specialty or other characteristics. While our fixed effects models control for time-invariant omitted variable bias such as specialty, age, gender, etc., we are unable to examine variation across specialties or physician characteristics, and it may be possible that the relationship between experience providing ambulatory care and EHR use is moderated by those variables. It is also possible that other factors besides experience could have resulted in increases in productivity, such as EHR improvements. However, no significant changes were made to the University of Alabama at Birmingham Health System Cerner system during the study period, and we observed no sudden changes in EHR time or productivity during the study period, and calendar month fixed effects control for secular changes over time in our regression analyses. Fifth, our data do not allow us to disaggregate the type of work being done after-hours, though only time working in clinical activities (chart review, orders, and documentation) is counted in this metric. Additionally, our measure of ambulatory care experience does not capture other aspects of experience with both working within the health system and with patient care, and alternative measures such as time employed by the health system that were not available in our data may be preferable for future studies. Finally, one of our measures of physician productivity is patients seen per day, which is measured by the patient encounters where the physician signs the note. Because residents may not be the physician signing the note, this metric serves as a proxy measure of productivity and may not capture some patients who may be seen by resident physicians who do not sign the chart note.[33] Further, patients seen per day in an ambulatory setting may not be a good measure of resident productivity for a variety of reasons mentioned above, most specifically that resident physicians may not be in control of their workloads.

Our results have important implications for policy and practice. Educators and regulators considering resident physician workload requirements should be aware of the burden of EHR work, a noninsignificant portion of which is done at home outside of clinic hours. Policymakers seeking to address the issue of physician burnout and turnover should also be aware of the impact of EHR work on early career physicians, and should take note that in the current environment physicians do not appear to be minimizing their after-hours EHR work. Interventions targeted at reducing the burden of EHRs should therefore specifically target this after-hours work, as broad attempts to improve EHR usability in general may not be effective.

Conclusion

There is growing concern over the impact of EHRs on physician wellness, with burdensome documentation requirements coupled with poor usability culminating in a large amount of time dedicated to “desktop medicine.” Early-career physicians, such as residents, may be especially vulnerable to this as they are already subject to long hours and mentally taxed from learning both clinical and socio-technical skills. Our results show residents become more efficient in the EHR and reduce time spent per patient as they gain experience, and they increase the number of patients seen per day and do not reduce the proportion of EHR time spent after-hours. These data should be concerning to those seeking to improve the residency experience and physician well-being more broadly.

Clinical Relevance Statement

Resident physicians spend a significant amount of time working in the EHR. As they gain experience, they spend less time per patient and are able to see more patients, but the proportion of time they spend after-hours does not decrease. Practitioners seeking to address physician well-being should be aware of the burden that after-hours EHR time represents.

Multiple Choice Questions

1. Resident physicians spend most of their EHR time in this function:

   • Documentation.

   • Chart review.

   • Orders.

   • Other functions such as messaging.

   Correct Answer: The correct answer is option a.

2. Resident physicians show this type of improvement as they gain experience:

   • Spending less time in the EHR after-hours.

   • Spending less time per patient in the EHR.

   • Spending more time per patient in the EHR.

   • Seeing fewer patients per day.

   Correct Answer: The correct answer is option b.

Conflict of Interest

None declared.

Protection of Human and Animal Subjects

This study was designated not human subjects research and exempt by the Institutional Review Board at the University of Alabama at Birmingham.

Note

The views expressed in this article are of the authors and do not necessarily represent the views of the institution providing the data. All data used in the study are proprietary and belongs to the University of Alabama at Birmingham Health System. This study was previously presented at American Medical Informatics Association Annual Symposium 2020.

• References

• 1 Blumenthal D. Launching HITECH. N Engl J Med 2010; 362 (05) 382-385
• 2 Adler-Milstein J, Holmgren AJ, Kralovec P, Worzala C, Searcy T, Patel V. Electronic health record adoption in US hospitals: the emergence of a digital “advanced use” divide. J Am Med Inform Assoc 2017; 24 (06) 1142-1148
• 3 Adler-Milstein J, DesRoches CM, Kralovec P. et al. Electronic health record adoption in US hospitals: progress continues, but challenges persist. Health Aff (Millwood) 2015; 34 (12) 2174-2180
• 4 Tai-Seale M, Olson CW, Li J. et al. Electronic health record logs indicate that physicians split time evenly between seeing patients and desktop medicine. Health Aff (Millwood) 2017; 36 (04) 655-662
• 5 Shanafelt TD, Dyrbye LN, Sinsky C. et al. Relationship between clerical burden and characteristics of the electronic environment with physician burnout and professional satisfaction. Mayo Clin Proc 2016; 91 (07) 836-848
• 6 Ventres WB. Electronic health records: can we maximize their benefits and minimize their risks?. Acad Med 2012; 87 (11) 1456-1457
• 7 Sinsky C, Colligan L, Li L. et al. Allocation of physician time in ambulatory practice: a time and motion study in 4 specialties. Ann Intern Med 2016; 165 (11) 753-760
• 8 Bodenheimer T, Sinsky C. From triple to quadruple aim: care of the patient requires care of the provider. Ann Fam Med 2014; 12 (06) 573-576
• 9 Gardner RL, Cooper E, Haskell J. et al. Physician stress and burnout: the impact of health information technology. J Am Med Inform Assoc 2019; 26 (02) 106-114
• 10 Gregory ME, Russo E, Singh H. Electronic health record alert-related workload as a predictor of burnout in primary care providers. Appl Clin Inform 2017; 8 (03) 686-697
• 11 Adler-Milstein J, Zhao W, Willard-Grace R, Knox M, Grumbach K. Electronic health records and burnout: time spent on the electronic health record after hours and message volume associated with exhaustion but not with cynicism among primary care clinicians. J Am Med Inform Assoc 2020; 27 (04) 531-538
• 12 Philibert I, Friedmann P, Williams WT. ACGME Work Group on Resident Duty Hours. Accreditation Council for Graduate Medical Education. New requirements for resident duty hours. JAMA 2002; 288 (09) 1112-1114
• 13 Dziorny AC, Orenstein EW, Lindell RB, Hames NA, Washington N, Desai B. Pediatric trainees systematically under-report duty hour violations compared to electronic health record defined shifts. PLoS One 2019; 14 (12) e0226493
• 14 Bolster L, Rourke L. The effect of restricting residents' duty hours on patient safety, resident well-being, and resident education: an updated systematic review. J Grad Med Educ 2015; 7 (03) 349-363
• 15 McCray LW, Cronholm PF, Bogner HR, Gallo JJ, Neill RA. Resident physician burnout: is there hope?. Fam Med 2008; 40 (09) 626-632
• 16 Ratwani RM, Fairbanks RJ, Hettinger AZ, Benda NC. Electronic health record usability: analysis of the user-centered design processes of eleven electronic health record vendors. J Am Med Inform Assoc 2015; 22 (06) 1179-1182
• 17 Beckman TJ, Reed DA, Shanafelt TD, West CP. Resident physician well-being and assessments of their knowledge and clinical performance. J Gen Intern Med 2012; 27 (03) 325-330
• 18 Wang JK, Ouyang D, Hom J, Chi J, Chen JH. Characterizing electronic health record usage patterns of inpatient medicine residents using event log data. PLoS One 2019; 14 (02) e0205379
• 19 Hultman GM, Marquard JL, Lindemann E, Arsoniadis E, Pakhomov S, Melton GB. Challenges and opportunities to improve the clinician experience reviewing electronic progress notes. Appl Clin Inform 2019; 10 (03) 446-453
• 20 Khairat S, Burke G, Archambault H, Schwartz T, Larson J, Ratwani RM. Perceived burden of EHRs on physicians at different stages of their career. Appl Clin Inform 2018; 9 (02) 336-347
• 21 Zheng K, Padman R, Johnson MP, Diamond HS. Understanding technology adoption in clinical care: clinician adoption behavior of a point-of-care reminder system. Int J Med Inform 2005; 74 (7-8): 535-543
• 22 Chaiyachati KH, Shea JA, Asch DA. et al. Assessment of inpatient time allocation among first-year internal medicine residents using time-motion observations. JAMA Intern Med 2019; 179 (06) 760-767
• 23 Overhage JM, McCallie Jr D. Physician time spent using the electronic health record during outpatient encounters: a descriptive study. Ann Intern Med 2020; 172 (03) 169-174
• 24 Bennett S, Maton K. Beyond the ‘digital natives’ debate: Towards a more nuanced understanding of students' technology experiences. J Comput Assist Learn 2010; 26 (05) 321-331l
• 25 Ratwani RM, Reider J, Singh H. A decade of health information technology usability challenges and the path forward. JAMA 2019; 321 (08) 743-744
• 26 Holmgren AJ, Downing NL, Bates DW. et al. Assessment of electronic health record use between us and non-us health systems. JAMA Intern Med 2021; 181 (02) 251-259
• 27 Rotenstein LS, Holmgren AJ, Downing NL, Bates DW. Differences in total and after-hours electronic health record time across ambulatory specialties. JAMA Intern Med 2021; 181 (06) 863-865
• 28 Overhage JM, Johnson KB. Pediatrician electronic health record time use for outpatient encounters. Pediatrics 2020; 146 (06) e20194017
• 29 Berdahl CT, Moran GJ, McBride O, Santini AM, Verzhbinsky IA, Schriger DL. Concordance between electronic clinical documentation and physicians' observed behavior. JAMA Netw Open 2019; 2 (09) e1911390-e1911390l
• 30 Melnick ER, Dyrbye LN, Sinsky CA. et al. The association between perceived electronic health record usability and professional burnout among US physicians. Mayo Clin Proc 2020; 95 (03) 476-487
• 31 Sinsky CA, Rule A, Cohen G. et al. Metrics for assessing physician activity using electronic health record log data. J Am Med Inform Assoc 2020; 27 (04) 639-643
• 32 Baxter SL, Apathy NC, Cross DA, Sinsky C, Hribar MR. Measures of electronic health record use in outpatient settings across vendors. J Am Med Inform Assoc 2021; 28 (05) 955-959
• 33 Mai MV, Orenstein EW, Manning JD, Luberti AA, Dziorny AC. Attributing patients to pediatric residents using electronic health record features augmented with audit logs. Appl Clin Inform 2020; 11 (03) 442-451

Publication History

Received: 02 February 2021

Accepted: 21 June 2021

Article published online:
04 August 2021