A Comparison of Resident Procedure Logs to Data Generated from an Electronic Health Record

Abstract

Background Emergency medicine (EM) residents are required to report procedural competency, often in a database separate from the electronic health record (EHR), in a redundant and time-consuming manner. We hypothesize that, if documented in an appropriate manner, procedural reports generated from an EHR reliably exceed those required by the Accreditation Council for Graduate Medical Education (ACGME) as well as those self-reported by EM residents.

Objectives (1) To compare the number of medical resuscitations recorded in the EHR to the number documented by residents in a separate database. (2) To compare the number of medical resuscitations recorded in the EHR to the ACGME requirement for graduation.

Methods Self-reported numbers of adult medical resuscitation by each resident of the previous three graduating classes of one EM program were compared with those generated from the EHR (Epic Systems, Verona, Wisconsin). There is no discrete documentation of medical resuscitations in the EHR. The ACGME describes a resuscitation as “…patient care for which prolonged physician attention is needed,” and thus, a surrogate was determined to be any patient for which the attending physician documented critical care time.

Results Data generated from the EHR reliably exceeded reported (mean [M] = 165.78, standard deviation [SD] = 45.97) and required (M = 188.09, SD = 30.93) numbers for adult medical resuscitations for 100% of the residents of the past three graduating classes (n = 32).

Conclusion In an accredited EM residency program that utilizes a modern EHR with a validated reporting functionality, residents should not need to redundantly log the number of adult medical resuscitations performed. Each resident in this study performed significantly more adult medical resuscitations than they logged and more than required by the ACGME, and thus, the time spent documenting these in a separate database was superfluous. Furthermore, this process increases resident awareness of proper documentation and data stewardship, two skills certain to prevail throughout their careers as modern EM physicians.

Keywords

electronic health records and systems - databases - data creation and storage - emergency medicine - training and education requirements - data collection

Introduction

Medical residents in most specialties are required to keep a record of their procedures performed during residency per guidelines set by the Accreditation Council for Graduate Medical Education (ACGME). These procedure logs serve as a record of procedural competency required for residency graduation and often required for hiring as well. These procedure logs are often completed by individual residents in a database separate from the electronic health record (EHR) in a redundant and time-consuming manner. This dated process has not translated well to the data-heavy era of EHRs. We examined the procedure logs recorded by residents from an emergency medicine (EM) residency program with procedural reports generated by an EHR. In particular, we compared the number of medical resuscitations logged by residents with those captured by the EHR. We hypothesized that, if documented in an appropriate manner, procedural reports generated from an EHR reliably exceed those required by the ACGME as well as those self-reported by EM residents.

Prior studies have explored similar benefits of procedural reports generated from an EHR. Outcomes identified in a study by Seufert et al included an increase in daily mean number of procedures logged, while simultaneously being more detailed and complete than data self-reported by EM residents.[1]

Objectives

• To compare the number of medical resuscitations recorded in the EHR to the number documented by residents in a separate database.

• To compare the number of medical resuscitations recorded in the EHR to the ACGME requirement for graduation.

Methods

Self-reported numbers of adult medical resuscitations by each resident of the previous three graduating classes of one EM program were compared with those generated from the EHR (Epic Systems, Verona, Wisconsin; [Table 1]). Data were collected via the self-reporting tool SlicerDicer.

Table 1 Resident data from SlicerDicer

	Resident	NI	SD CC	> Required	> Reported
Class of 2021	Resident 1	53	206	161	153
	Resident 2	59	230	185	171
	Resident 3	50	213	168	163
	Resident 4	60	223	178	163
	Resident 5	59	184	139	125
	Resident 6	78	196	151	118
	Resident 7	52	221	176	169
	Resident 8	46	214	169	168
	Resident 9	63	228	183	165
	Resident 10	79	237	192	158
	Resident 11	48	239	194	191
	Resident 12	81	169	124	88
Class of 2020	Resident 13	59	207	162	148
	Resident 14	127	182	137	55
	Resident 15	46	257	212	211
	Resident 16	45	255	210	210
	Resident 17	47	230	185	183
	Resident 18	54	210	165	156
	Resident 19	48	245	200	197
	Resident 20	63	252	207	189
	Resident 21	54	276	231	222
	Resident 22	74	266	221	192
Class of 2019	Resident 23	60	291	246	231
	Resident 24	166	256	211	90
	Resident 25	102	236	191	134
	Resident 26	47	252	207	205
	Resident 27	117	240	195	123
	Resident 28	0	228	183	228
	Resident 29	55	258	213	203
	Resident 30	45	196	151	151
	Resident 31	49	300	255	251
	Resident 32	168	262	217	94

Abbreviations: NI, self-reported procedures from New Innovations; SD CC, SlicerDicer critical care instances.

Note: Columns 5 and 6 represent the number of procedures above required and reported, respectively.

Resident procedure logs were retrieved from the in-house residency management software (New Innovations, Uniontown, Ohio). These logs include approved real-life cases managed during residency training as well as simulated cases performed during procedural conference. The ACGME describes a resuscitation as “…patient care for which prolonged physician attention is needed,”[2] and thus, a surrogate was determined to be any patient for which the attending physician documented critical care time. Resident attribution was accomplished by identifying the “first resident assigned” as recorded in the EHR. This is a discrete data element captured during normal resident workflows. Each chief complaint was reviewed to separate medical resuscitations from trauma resuscitations. These were further delineated into adult and pediatric resuscitations by the patient's age at the time of visit.

Results

Data generated from the EHR reliably exceeded reported (mean [M] = 165.78, standard deviation [SD] = 45.97) numbers for adult medical resuscitations for 100% of the residents of the past three graduating classes (n = 32; [Table 2]). Data generated from the EHR required also exceeded the number required by the ACGME for procedural competency for 100% of the residents (M = 188.09, SD = 30.93; [Table 2]).

Table 2 Comparison of average reported versus retrieved adult medical resuscitations

	Reported	SlicerDicer	> Required	> Reported
Average	67.31	233.09	188.09	165.78
Standard deviation	34.80	30.93	30.93	45.97

Discussion

In the era of EHRs, a large amount of data are being collected in the background that sometimes precludes the need for separate databases that were previously necessary. In this instance of resident procedural logs, the number of medical resuscitations recorded by the EHR reliably exceeded both the number recorded by the residents as well as the number required by the ACGME for graduation for 100% of the residents.

One challenge in this study was developing a surrogate in the EHR for medical resuscitations as there is no discrete entry in the EHR that declares an intervention to be a medical resuscitation. In addition, residents, particularly interns, are often not aware of which interventions qualify as a medical resuscitation given the ambiguity in the ACGME's definition of medical resuscitation. This is in contrast to other procedures such as lumbar punctures and intubations, which are clearly defined. Based on the ACGME's definition of a resuscitation as “…patient care for which prolonged physician attention is needed,”[2] we used critical care time documented by the attending physician as a surrogate for medical resuscitation. This appears to be a fair surrogate as many of the same patient abnormalities, such as shock, and therapeutic interventions, such as intravenous vasopressors, recognized as part of critical care require prolonged physician attention. Thus, capturing a resuscitation is dependent on critical care time being documented for patients. Given this dependency on critical care time documentation, we are potentially capturing fewer than the actual number of resuscitations performed by residents if critical care time is not documented by the attending physician.

The necessity for developing a surrogate for medical resuscitations illustrates a principle of data literacy[3] that the type and quality of data output from the EHR is determined by the data input. As there was no discrete documentation of medical resuscitations in the EHR, it was not possible to obtain a direct log of medical resuscitations from the EHR.

There were several self-reported documentation numbers that either equaled or barely exceeded the number medical resuscitations required by the ACGME. This could be due to cessation of reporting after meeting the minimum standard for residency graduation. Hence, the logs obtained from the EHR are more accurate representations of the number of procedures the residents have actually performed.

Conclusion

In an accredited EM residency program that utilizes a modern EHR with a validated reporting functionality, residents should not need to redundantly log the number of adult medical resuscitations performed in a separate database. Each resident in this study performed significantly more adult medical resuscitations than they logged and more than required by the ACGME, and thus, the time spent documenting these in a separate database was superfluous. This method has the added advantage of being an accurate representation of real-life scenarios instead of simulated patient resuscitations. Furthermore, this process increases resident awareness of proper documentation and data stewardship, two skills certain to prevail throughout their careers as modern EM physicians. Further analysis is being conducted to determine if these findings apply to other procedural reporting required by the ACGME.

Clinical Relevance Statement

In an accredited EM residency program that utilizes a modern EHR with a validated reporting functionality, residents should not need to redundantly log the number of adult medical resuscitations performed in a separate database. The EHR in this case provided a log of medical resuscitations that was more accurate in capturing the number of medical resuscitations performed by residents than the separate database the residents used for self-reporting.

Conflict of Interest

None declared.

• References

• 1 Seufert TS, Mitchell PM, Wilcox AR. et al. An automated procedure logging system improves resident documentation compliance. Acad Emerg Med 2011; 18 (Suppl. 02) S54-S58
• 2 Review Committee for Emergency Medicine, Accreditation Council for Graduate Medical Education (ACGME). Frequently Asked Questions: Emergency Medicine. 2017 https://www.acgme.org/globalassets/pdfs/faq/110_emergency_medicine_faqs_2017-07-01.pdf
• 3 Wells BJ, Chagin KM, Nowacki AS, Kattan MW. Strategies for handling missing data in electronic health record derived data. EGEMS (Wash DC) 2013; 1 (03) 1035

Publication History

Received: 04 June 2023

Accepted: 07 September 2023

Article published online:
10 November 2023