Implementation of a Standardized Patient Safety Checklist in Ophthalmic Surgery

Abstract

Objective To develop and institute a standardized ophthalmology-specific operating room checklist, and to test adherence to the surgical checklist components after implementation.

Design An ophthalmology-specific operative room checklist was developed using recommendations from surgical preoperative checklists from other subspecialties and checklists specific to cataract surgery. This standardized checklist was then implemented into the operating rooms. Operative room staff was trained on its use. Adherence to the checklist was measured prospectively.

Setting Surgical centers at Kellogg Eye Center in Ann Arbor and Livonia, Michigan.

Participants Patients undergoing ophthalmic surgery at the Livonia and Ann Arbor surgical centers.

Main Outcome(s) and Measure(s) The primary outcome measures were surgical team adherence to each category of the operative checklist (preop, prebrief, preanesthesia verification, time-out, intraocular lens measurement/confirmation for cataract cases, debrief). Adherence to the subsections of the operative checklist was measured during three periods over the course of 1 year: baseline, postintervention (after introduction of the checklist in all operating rooms), and postadjustment (after additional training was provided to operating room staff and incentives to collect data were provided to personnel responsible for collecting it).

Results A total of 2,532 surgical cases were included in the analysis. Baseline adherence ranged from 87 to 90% across preop, prebrief, preanesthesia, and time-out verification sections of the operative checklist. After the institution of the intervention and adjustments, adherence to the surgical checklist improved significantly across all areas of the checklist to 97–98% (p < 0.01).

Conclusions and Relevance The development and implementation of an ophthalmology-specific operative room checklist increase adherence to recommended safety checks prior to the start of surgery. To maximize patient safety, the same or similar checklist is recommended for use in all ophthalmology operating rooms.

Keywords

surgical checklist - patient safety - time-out

Preventing and reducing medical errors is an increasing concern in health care. Particular attention has focused on the prevalence of preventable surgical errors, including wrong site, wrong patient, wrong implant, and wrong surgery. The landmark publication To Err is Human initially sheds light on medical errors in the United States.[1] Instead of blaming individual physicians for safety errors, the publication recommended minimizing errors through systematic changes to the health care system.[1] Using preoperative surgical checklists improves surgical morbidity and mortality.[2] The Joint Commission on Accreditation of Healthcare Organizations (JCAHO) published a “Universal Protocol” for all surgical procedures, which includes a preincision time-out policy that includes confirming patient identity, type of surgery to be performed, and site verification.[3]

Ophthalmic surgeons are at special risk of making surgical errors.[4] Cataract surgery is the most commonly performed surgery in the world, and it has the highest risk of wrong implant insertion of any specialty.[5] In 2009, the World Health Organization (WHO) developed a surgical safety checklist that was later adapted for cataract surgery.[6]

Despite the evidence supporting the use of operative checklists, no single standardized checklist has been widely implemented to apply to all types of ophthalmic surgery in the United States. At our institution, policy included marking the surgical site and preoperative time-outs, but these expectations were not routinely monitored or enforced.

Ophthalmologists are expected to assess and improve the care they provide to their patients. American Board of Ophthalmology mandated that diplomates who certified after 1991 participate in activities to improve the quality and safety of care to maintain certification.[7] Our institution has been approved by the American Board of Medical Specialties Multi-Specialty Portfolio Program to award Maintenance of Certification (MOC) credit to physicians participating in two or more linked cycles of data-guided improvement effort. The American Board of Ophthalmology accepts this participation as fulfilling the requirement for performing one practice performance module every 5 years, as part of its emphasis on patient safety through continuing medical education (CME). Similarly, the American Academy of Ophthalmology developed a performance improvement course dedicated to wrong site and wrong intraocular lens (IOL) surgery that qualifies for CME credit.[8]

The primary purpose of this study was to improve patient safety by developing a standardized ophthalmic operating room checklist, developing a method to measure performance in following the checklist, implementing routine use of the checklist, and determining whether improvement in performing preoperative safety checks occurred. A secondary purpose was to design the improvement effort so that participating ophthalmologists would meet their current MOC requirement for performing a practice performance module as part of their regular work at our institution.

Methods

Ethical Approval

This quality improvement (QI) effort translated known and recommended practices to the delivery of care for our ophthalmic patients. QI efforts are not regulated by Institutional Review Boards. This QI effort was performed with approval and oversight by clinical leaders for the units where improvement activities occurred.

Design

The overall plan is shown in [Table 1]. A prospective repeated pre-post design reflects two linked cycles of Plan-Do-Check-Act improvement effort: baseline measurement and planning, intervention, postintervention measurement and planning, adjustments (further interventions), and postadjustment measurement and planning. The unit of analysis is an operative case.

Table 1 Overview of activities by PDCA cycle stage

PDCA cycle stage	Activities
Plan	Develop preoperative safety checklist for ophthalmic operating rooms Develop method to measure adherence to preoperative checklist Collect baseline data (8/5/14–8/11/14) Review baseline data, identify underlying causes for nonadherence, and develop intervention plans
Do	Implement use of the preoperative safety checklist (9/1/14)
Check	Collect postintervention data (10/15/14–12/15/14)
Adjust/Replan	Review postintervention data, identify underlying causes for nonadherence, and develop plans to adjust interventions (12/16/14–12/21/14)
Redo	Implement adjustments (12/22/14)
Recheck	Collect postadjustment data (3/23/15–6/1/15)
Readjust/Replan	Review postadjustment data, identifying underlying causes for nonadherence, and develop further plans for improvement (8/1/15–8/31/15)

Abbreviation: PDCA, plan-do-check-adjust.

Population

This improvement effort included all patients undergoing ophthalmic surgery at the University of Michigan's Kellogg Eye Center (KEC) and Livonia Surgery Center operating rooms between August 5, 2014 and June 1, 2015.

Checklist Development

An ophthalmology-specific surgical checklist was designed utilizing recommendations and specifications from the WHO surgical checklist,[6] the WHO adaptation for cataract surgery, and adaptations from other specialties.[9] The checklist emphasized prevention of relatively common surgical errors and “near misses” in ophthalmic surgery, including laterality and IOL (or other) implant selection.[10] [11] The checklist contains 12 items grouped into 5 categories that apply to all ophthalmic surgeries and an additional 5 items and category that additionally apply to cataract surgery ([Table 2]).

Table 2 Ophthalmology surgery checklist

1. Debrief performed (form submitted)

2. Preop

Patient verification

Surgery site marked

3. Prebrief

Type of anesthesia

Special devices/instruments

Allergies

Confirm surgical marking visible under drape

Confirm microscope and phacoemulsification settings[a]

IOL power written by surgeon on board (axis for toric lenses)[a]

IOL measurements available[a]

Only primary IOL on counter[a]

Confirmation that selected IOL matches IOL power on board[a]

4. Preanesthesia verification

Name/side surgery/operation

IOL power[a]

Allergies

Antibiotics

Fire risk

5. Time-out (after draping)

Name/side surgery/operation

IOL power[a]

Allergies

Antibiotics

Fire risk

6. Confirmation of correct IOL from surgeon when opening IOL [a]

Abbreviations: IOL, intraocular lens implant.

^a Applicable to cataract surgery.

Measures and Data Collection

The outcome measures are surgeon adherence to each of the six categories on the checklist. While all physicians and operating room staff were responsible for conducting the checklist process, the attending surgeon was ultimately accountable for adhering to the checklist.

An online debriefing form was developed to record whether or not a category was addressed. Operating room circulating nurses were trained to complete a form for each surgery and were responsible for filling out and submitting the form at the conclusion of a surgery—see example form for noncataract surgery in [Fig. 1]. The debrief category was scored as adherent if the debrief form was submitted. Three categories on the checklist included more than one item. These categories were scored as adherent if any item in the category was completed. All ophthalmologic cases performed during each of the observation periods were included in the analysis.

Fig. 1 Operating room (OR) debrief form example for a noncataract ophthalmic surgery.

Intervention

Baseline data were collected during the week of August 5, 2014 to August 11, 2014 to measure checklist adherence. Surgical teams had been informed that the checklist project would occur and that baseline data were being collected. Surgeons individually received adherence rates for their cases and data aggregated across surgeons were provided to operative teams. Suggestions regarding underlying causes and ways to improve adherence rates were solicited from surgeons, nurses, and other members of the operative teams. The main interventions were (1) to post large versions of the checklists in an easily visible location in each operating room to provide an ongoing reminder and immediate reference to follow and (2) to educate operating room physicians and staff on the use of the standardized checklist and how adherence is documented.

Postintervention data were collected during the 2 months from October 15, 2014 to December 15, 2014. Again adherence rates were provided to surgeons individually and aggregate data were provided to operative teams. Physicians and staff provided further suggestions to improve performance. The main adjustment to the intervention for all personnel was additional education regarding the checklist, its importance, and how to perform the checks. An additional intervention focused on circulating nurses to increase the number and percentage of cases with debriefing forms completed and submitted during the next observation period. If a circulating nurse completed all of his/her assigned debrief forms during a week, the circulating nurse was given a $5.00 gift card and the nurse's 100% completion was recognized at a weekly nursing meeting.

Postadjustment data were collected during the 2 months and 1 week from March 23, 2015 to June 1, 2015. Again adherence rates were provided to surgeons individually and in aggregate to operative teams and further suggestions for improvements sought.

Statistical Analyses

Chi-squared tests were performed to compare adherence rates on checklist categories over time. The analyses were performed with Microsoft Excel software version 14.5.8 (Redmond, WA).

Results

A total of 2,532 surgical cases were included in the analysis. Time 1 (T1, baseline data collection) included all cases performed at KEC and Livonia sites over the course of 1 week, which comprised a total of 114 surgical cases by 28 surgeons (per surgeon, minimum = 1 case, median = 3 cases, and maximum = 13 cases). For these 114 cases, 90% had debrief forms submitted for analysis. Of the 103 submitted debrief forms, 87% of cases met criteria for preop completion and 90% of the cases met criteria for prebrief, preanesthesia, and time-out criteria. For the 55 debrief forms involving IOL cases, 1 case (2%) had an IOL selection confirmation.

Time 2 (T2, post-intervention) took place over 2 months and included 1,135 surgical cases by 51 surgeons (per surgeon, minimum = 1 case, median = 15 cases, and maximum = 85 cases). The rate of submitting debrief forms was lower (70%). For cases with debrief forms, rates of adherence to specific checks was lower (85% to 86%).

Time 3 (T3, postadjustment) occurred after re-education and after the initiation of small financial incentives to encourage the circulators to turn in debrief forms. Of the 1,283 surgical cases done in the two surgical centers by 55 surgeons (per surgeon, minimum = 1 case, median = 17 cases, and maximum = 126 cases) during the period of 2 months plus a week, 1,050 debrief forms (82%) were submitted. For cases with debrief forms, adherence was 97% for preop completion, prebrief, preanesthesia, and time-out, and 98% for IOL confirmation in the 581 cataract cases.

Looking at change in performance over time, [Table 3] shows that the rate of nurses completing debrief forms decreased somewhat from T1 to T3. However, the rate of surgeon's adherence to all aspects of the standardized checklist significantly increased between the baselines reported in T1 compared with T3. Overall increases were also noted between T2 and T3 ([Table 3]).

Table 3 Percent of preoperative cases with debrief forms submitted and percent of debrief forms submitted with preoperative check performed

	Time 1: baseline (8/5/14–8/11/14)	Time 2: Postintervention (10/15/14–12/15/14)	Time 3: Postadjustment (3/23/15–6/1/15)	p-Value T1 = T2	p-Value T2 = T3	p-Value T1 = T3
Number of preop cases	114	1,135	1,283
% with:
1. Debrief form	90%	71%	82%	< 0.01	< 0.01	0.03
Number of debrief forms	103	802	1,050
% with checks:
2. Preop	87%	85%	97%	0.66	< 0.01	< 0.01
3. Prebrief	90%	85%	97%	0.23	< 0.01	< 0.01
4. Preanesthesia	90%	86%	97%	0.27	< 0.01	< 0.01
5. Time-out	90%	86%	97%	0.23	< 0.01	< 0.01
Number of IOL cases with debrief forms	55	478	581
% (N) with:
6. IOL Confirmation	2%	86%	98%	<0.01	< 0.01	<0.01

Abbreviation: IOL, intraocular lens implant.

Adherence rates were compared for cases involving IOL implantation compared with non-IOL cases at T2 and at T3 ([Table 4]). At each time period and for each type of surgeon adherence (preop, prebrief, preanesthesia, and time-out), adherence rates were significantly higher for IOL cases than for non-IOL cases.

Table 4 Comparison of IOL versus non-IOL surgeries on checklist adherence rate at T2 and T3

Time period surgery type	Number of surgeries	Checklist adherence rate
		Preop	Prebrief	Preanesthesia	Time-out
T2
IOL	410	99%	99%	100%	100%
Non-IOL	392	70%	70%	71%	70%
p-Value	<0.0001	<0.0001	<0.0001	<0.0001	<0.0001
T3
IOL	570	100%	100%	100%	100%
Non-IOL	480	94%	94%	94%	94%
p-Value	<0.0001	<0.0001	<0.0001	<0.0001	<0.0001

Abbreviation: IOL, intraocular lens implant.

An analysis was performed to determine whether performance changed over time during the observation periods of at least 2 months at T2 and T3. The analysis of performance rates by week showed no trend for performance change over time during either T2 or T3.

The relationship between number of surgical cases performed and checklist adherence rates was examined for T2 and T3. Performing surgery more frequently is related to higher performance on each type of check. An additional analysis confirmed that the IOL implantation and higher surgical volume are independently related to checklist adherence. Cases involving IOL implantation had higher rates of checklist adherence regardless of the overall surgical volume of the IOL surgeons in T2 and T3.

A secondary aim of the QI effort was to meet requirements for participating ophthalmic surgeons to receive MOC credit for completing a performance improvement module. The project leader documented that the project met the requirement for cycles of improvement and verified the participation of individual ophthalmic surgeons in each main step of the effort. A total of 24 ophthalmic surgeons participating in MOC requested and received credit for completing a performance module, meeting this requirement for their 5-year MOC cycle.

Discussion

This project addressed previous findings that ophthalmologic surgery has significant rates of wrong-side and wrong-implant surgical errors, in large part because procedures vary widely and often include patient-specific implants.[4] To our knowledge, no ophthalmology-specific surgical checklist to reduce these surgical errors for all eye surgeries (not just cataract surgeries) has been widely instituted in the United States.

The project achieved its four aims. To reduce the likelihood of errors in ophthalmic surgery: (1) a standard preoperative checklist was developed, (2) a method to measure performance in following the checklist was developed and implemented, and (3) the checklist and measures were incorporated into interventions demonstrating statistical and meaningful improvement in performing preoperative checks from baseline (T1) to postadjustment (T3). Cycles of data-guided improvement were designed so that (4) participating ophthalmologists met requirements for completing a performance improvement module for their 5-year MOC cycle.

Five patterns in the performance results deserve further consideration. One pattern was that baseline IOL confirmation of measurements/selection rarely occurred (2%), while performance of other checks occurred frequently (87–90%). This initial difference reflects IOL confirmation being a new expectation with this project, while other checks were part of existing institutional recommendations (not previously measured or enforced). After the IOL confirmation recommendation was introduced during the initial intervention, its performance at T2 and T3 increased to levels similar to performance on other checks at those times.

A second pattern of results from baseline (T1) to postintervention (T2) was unexpected. Performance scores were expected to increase from baseline (T1) to postintervention (T2). However, completing debrief forms meaningfully decreased (19 percentage points) and performance on the four checks that were part of existing institutional recommendations decreased slightly (2–5 percentage points). A likely explanation is that the novelty of recording performance in operating rooms during the week of baseline data collection artificially increased the baseline performance rates. The novelty resulted in circulating nurses more systematically collecting data. Ophthalmic surgeons were aware of the cases being observed and the newly initiated observations resulted in recommended checks being performed more systematically. The baseline observation period was only 1 week while the subsequent observation periods were at least 2 months when observation was not new and occurred multiple times per ophthalmic surgeon. The later periods more likely reflect routine ongoing performance at those time periods. If baseline scores were inflated due to the novelty of observation, the increase from baseline (T1) to postadjustment (T3) underestimates the impact of the intervention.

A third pattern is that by postadjustment (T3), circulating nurses completed debrief forms for only 82% of the cases identified for observation. This is lower than the 90% completion rate at baseline (T1) when novelty and short duration likely enhanced completion rates. It is higher than the 71% rate at postintervention (T2), after which the $5.00 gift card and recognition at weekly meetings were initiated to increase completion of debriefing forms. Circulating nurses were asked why completion rate with these minor incentives was only 82%. They explained that already having to complete a variety of paperwork during and between relatively short ophthalmic surgeries did not always allow time to perform the additional task of recording checklist use. Further increases in data collection rates will likely require simplifying the task, increasing time for data collection, or identifying another approach to data collection.

A fourth pattern is that cataract surgery teams were more adherent to the checklist than noncataract surgery teams in both the postintervention (T2) and postadjustment (T3) periods. Cataract surgery is the most frequently performed surgery in ophthalmology and is more likely to be performed according to preset, uniform protocols than noncataract surgeries. The use of protocols in cataract surgery likely facilitates incorporating and following safety checklist procedures in performing cataract surgery. This pattern was unaffected by the surgical volume of the IOL surgeons.

A fifth pattern is that in both the postintervention (T2) and the postadjustment periods, surgeons who performed more cases had higher checklist adherence rates than surgeons who performed fewer cases. The repetition of performing cases more frequently likely helps ingrain adherence as a habit in performing surgery.[12]

As a study, the project has several methodological limitations. The demonstration involving a group of ophthalmic surgeons and patients at one institution may include methods of data collection and performance improvement that do not always apply to the context of practice elsewhere. However, the checklist itself can be used anywhere, the methods of data collection adapted, and interventions to improve performance customized locally. Performance data are missing for some patients identified for observation, but debrief forms were not completed. However, performance for those cases is unlikely to be sufficiently different from the observed cases to alter the conclusion that performing safety checks improved over the course of the project. Another weakness is that each main category of the debrief form had numerous subcategories. To simplify data collection for the circulators, not all items within a subcategory needed to be completed for full credit in that category. While more refined assessments of checklist adherence are needed, these results are sufficient to demonstrate that adherence can be increased. Furthermore, a member of the team evaluates compliance with checklist items; since scoring of items is subjective, it is possible that the team member may over- or underestimate compliance compared with an independent observer.

The positive results of this project suggest several advances to pursue. The checklist itself can be reviewed by other groups of ophthalmic surgeons for enhancement. Performance measurement can be expanded to include every subcategory in the checklist in addition to the major categories. Additional interventions to improve performance can be developed and shared, including possible implantation of penalties for not performing the items of the checklist. Performance can be monitored over time to determine the extent to which it is sustained when not being formally observed, and to observe adherence over time.

The development and implementation of a standardized ophthalmology-specific surgical checklist is important for all ophthalmic surgeons to avoid, or at least minimize, surgical errors in ophthalmological procedures. The standardized checklist reported here addresses items that have been shown to be the cause of surgical errors and near-misses in ophthalmic surgery.[11] Our project demonstrates that the use of a safety checklist can improve adherence to safety standards through appropriate surgical team education and support. Use of the same or similar checklist can be implemented in all ophthalmology operating rooms to support surgical safety. When deciding how to target efforts to increase checklist adherence, the greatest increase in performance is likely to be achieved by focusing on ophthalmic surgeons who perform noncataract surgery and those who perform fewer surgeries. Additionally, ophthalmologists can earn MOC credit for the American Board of Ophthalmology through local data-guided clinical improvement efforts such as implementing use of a surgical safety checklist.

Conflict of Interest

Dr. Lee was a Board Director of the American Board of Ophthalmology during this time. He was not involved in the approval of the institutional Maintenance of Certification (MOC) project described herein. The remaining authors have no conflict of interest.

Acknowledgments

The authors would like to thank Moshiur Rahman, PhD, from the University of Michigan who assisted with the statistical analysis of this data.

Meeting Presentation

None.

• References

• 1 Institute of Medicine (US) Committee on Quality of Health Care in America, Kohn LT, Corrigan JM, Donaldson MS. To Err is Human: Building a Safer Health System. 2000
• 2 Haynes AB, Weiser TG, Berry WR. , et al; Safe Surgery Saves Lives Study Group. A surgical safety checklist to reduce morbidity and mortality in a global population. N Engl J Med 2009; 360 (05) 491-499
• 3 Joint Commission on Accreditation of Healthcare Organizations. Universal protocol for preventing wrong site, wrong procedure, wrong person surgery. Available at: https://www.jointcommission.org/assets/1/18/UP_Poster1.PDF . Accessed January 1, 2018
• 4 Simon JW, Ngo Y, Khan S, Strogatz D. Surgical confusions in ophthalmology. Arch Ophthalmol 2007; 125 (11) 1515-1522
• 5 Loh HP, de Korne DF, Chee SP, Mathur R. Reducing wrong intraocular lens implants in cataract surgery. Int J Health Care Qual Assur 2017; 30 (06) 492-505
• 6 The World Health Organization. Surgical Safety Checklist 2009 . Available at: www.who.intpatientsafetysafesurgeryenindex.html . Accessed November 16, 2017
• 7 American Board of Ophthalmology. Maintaining Certification. Available at: https://abop.org/maintain-certification/ . Accessed November 16, 2017
• 8 American Academy of Ophthalmology. Wrong Site/Wrong IOL Surgery. Available at: https://www.aao.org/pi-cme/wrong-site-wrong-iol-surgery . Accessed January 1, 2018
• 9 Kelly SP, Steeples LR, Smith R, Azuara-Blanco A. Surgical checklist for cataract surgery: progress with the initiative by the Royal College of Ophthalmologists to improve patient safety. Eye (Lond) 2013; 27 (07) 878-882
• 10 Lum F, Schachat AP. The quest to eliminate “never events”. Ophthalmology 2009; 116 (06) 1021-1022
• 11 Schein OD, Banta JT, Chen TC, Pritzker S, Schachat AP. Lessons learned: wrong intraocular lens. Ophthalmology 2012; 119 (10) 2059-2064
• 12 Maruthappu M, Gilbert BJ, El-Harasis MA. , et al. The influence of volume and experience on individual surgical performance: a systematic review. Ann Surg 2015; 261 (04) 642-647