Development of the “Teamwork in Endoscopy Assessment Module for Endoscopic Non-Technical Skills” (TEAM-ENTS) behavioral marker system

Srivathsan Ravindran; Rebecca Cavilla; Hutan Ashrafian; Adam Haycock; Chris Healey; Mark Coleman; Steph Archer; Ara Darzi; Siwan Thomas-Gibson; on behalf of the TEAM-ENTS working group

doi:10.1055/a-1959-6123

Endoscopy, Table of Contents

Endoscopy 2023; 55(05): 403-412
DOI: 10.1055/a-1959-6123

Original article

Development of the “Teamwork in Endoscopy Assessment Module for Endoscopic Non-Technical Skills” (TEAM-ENTS) behavioral marker system

Srivathsan Ravindran

¹Joint Advisory Group on Gastrointestinal Endoscopy, Royal College of Physicians, London, UK

²Wolfson Unit for Endoscopy, St Mark’s Hospital, London, UK

³Department of Surgery and Cancer, Imperial College London, London, UK

,

Rebecca Cavilla

⁴NHS Improvement, NHS England, London, UK

,

Hutan Ashrafian

³Department of Surgery and Cancer, Imperial College London, London, UK

,

Adam Haycock

²Wolfson Unit for Endoscopy, St Mark’s Hospital, London, UK

,

Chris Healey

⁵Department of Gastroenterology, Airedale NHS Foundation Trust, Keighley, UK

,

Mark Coleman

¹Joint Advisory Group on Gastrointestinal Endoscopy, Royal College of Physicians, London, UK

⁶Department of Colorectal Surgery, University Hospitals Plymouth NHS Trust, Plymouth, UK

,

Steph Archer

³Department of Surgery and Cancer, Imperial College London, London, UK

⁷Department of Public Health and Primary Care, University of Cambridge, Cambridge, UK

⁸Department of Psychology, University of Cambridge, Cambridge, UK

,

Ara Darzi

³Department of Surgery and Cancer, Imperial College London, London, UK

,

Siwan Thomas-Gibson

²Wolfson Unit for Endoscopy, St Mark’s Hospital, London, UK

⁹Department of Metabolism, Digestion and Reproduction, Imperial College London, London, UK

,

on behalf of the TEAM-ENTS working group

› Author Affiliations

Abstract

Background Non-technical skills (NTS) are integral to team performance and subsequent quality and safety of care. Behavioral marker systems (BMSs) are now increasingly used in healthcare to support the training and assessment of team NTS. Within gastrointestinal endoscopy, this is an area of novel research. The aims of this study were to define the core relevant NTS for endoscopy teams and develop a preliminary framework for a team-based BMS known as TEAM-ENTS (Teamwork in Endoscopy Assessment Module for Endoscopic Non-Technical Skills).

Methods This study was conducted in two phases. In phase 1, a literature review of team-based BMSs was performed to inform an interview study of core endoscopy team members. Cognitive task analysis was used to break down the NTS relevant to endoscopy teams. Framework analysis generated the structure for the preliminary TEAM-ENTS framework. In phase 2, a modified Delphi process was undertaken to refine the items of the framework.

Results Seven consultant endoscopists and six nurses were interviewed. The final coding framework consisted of 88 codes grouped into five overarching categories. In total, 58 participants were recruited to the Delphi panel. In the first round, nine elements and 37 behavioral descriptors did not meet consensus. Following item adjustment, merging and deletion, all remaining items met consensus thresholds after the second round. The refined TEAM-ENTS BMS consists of five categories, 16 elements, and 47 behavioral descriptors.

Conclusions The refined TEAM-ENTS behavioral marker system was developed to reflect the core NTS relevant to endoscopy teams. Future studies will aim to fully validate this tool.

Full Text

References

References
1 Kohn L, Corrigan J, Donaldson M. To Err is Human: Building a safer health system. Washington DC: Institute of Medicine National Academies Press; 1999
2 Schmutz J, Manser T. Do team processes really have an effect on clinical performance? A systematic literature review. Br J Anaesth 2013; 110: 529-544
3 Dietz AS, Pronovost PJ, Benson KN. et al. A systematic review of behavioural marker systems in healthcare: what do we know about their attributes, validity and application?. BMJ Qual Saf 2014; 23: 1031
4 Higham H, Greig PR, Rutherford J. et al. Observer-based tools for non-technical skills assessment in simulated and real clinical environments in healthcare: a systematic review. BMJ Qual Saf 2019; 28: 672
5 Boet S, Etherington N, Larrigan S. et al. Measuring the teamwork performance of teams in crisis situations: a systematic review of assessment tools and their measurement properties. BMJ Qual Saf 2019; 28: 327
6 Wheelock A, Suliman A, Wharton R. et al. The impact of operating room distractions on stress, workload, and teamwork. Ann Surg 2015; 261: 1079-1084
7 Westli HK, Johnsen BH, Eid J. et al. Teamwork skills, shared mental models, and performance in simulated trauma teams: an independent group design. Scand J Trauma Resusc Emerg Med 2010; 18: 47
8 Doumouras AG, Hamidi M, Lung K. et al. Non-technical skills of surgeons and anaesthetists in simulated operating theatre crises. Br J Surg 2017; 104: 1028-1036
9 Ravindran S, Haycock A, Woolf K. et al. Development and impact of an endoscopic non-technical skills (ENTS) behavioural marker system. BMJ Simul Technol Enhanc Learn 2021; 7: 17-25
10 Joint Advisory Group on Gastrointestinal Endoscopy. Download Centre: Formative DOPS Colonoscopy & Flexible sigmoidoscopy. 2016 Available from (Accessed 19 October 2022): https://www.thejag.org.uk/Downloads/JAG/DOPS%20forms%20(international%20and%20reference%20use%20only)/Formative%20DOPS_Colonoscopy%20and%20Flexible%20sigmoidoscopy.pdf
11 Ravindran S, Thomas-Gibson S, Murray S. et al. Improving safety and reducing error in endoscopy: simulation training in human factors. Frontline Gastroenterol 2019; 10: 160-166
12 Joint Advisory Group on Gastrointestinal Endoscopy. Improving Safety and Reducing Error in Endoscopy (ISREE) Implementation strategy. 2018 Available from (Accessed 19 October 2022): www.thejag.org.uk/Downloads/JAG/General/Improving%20Safety%20and%20Reducing%20Error%20in%20Endoscopy%20(ISREE)%20Implementation%20strategy%20v1.0.pdf
13 Clark RE, Feldon DF, van Merriënboer J. et al. Cognitive task analysis. Spector JM, Merrill MD, van Merriënboer J. et al. Handbook of research on educational communications and technology. 3rd edn. Routledge, UK: Routledge; 2007
14 Fletcher G, Flin R, McGeorge P. et al. Anaesthetists' non-technical skills (ANTS): evaluation of a behavioural marker system. Br J Anaesth 2003; 90: 580-588
15 Gale NK, Heath G, Cameron E. et al. Using the framework method for the analysis of qualitative data in multi-disciplinary health research. BMC Med Res Methodol 2013; 13: 117
16 Trevelyan EG, Robinson PN. Delphi methodology in health research: how to do it?. Eur J Integr Med 2015; 7: 423-428
17 Clayton MJ. Delphi: A technique to harness expert opinion for critical decision-making tasks in education. Educ Psychol 1997; 17: 373-386
18 de Villiers MR, de Villiers PJT, Kent AP. The Delphi technique in health sciences education research. Med Teach 2005; 27: 639-643
19 Lawshe CH. A quantitative approach to content validity. Pers Psychol 1975; 28: 563-575
20 Polit DF, Beck CT. The content validity index: are you sure you know what's being reported? Critique and recommendations. Res Nurs Health 2006; 29: 489-497
21 Dietz AS, Rosen MA, Wyskiel R. et al. Development of a behavioral marker system to assess intensive care unit team performance. Proc Hum Factors Ergon Soc Annu Meet 2015; 59: 991-995
22 Thomas EJ, Sexton JB, Helmreich RL. Translating teamwork behaviours from aviation to healthcare: development of behavioural markers for neonatal resuscitation. Qual Saf Health Care 2004; 13 (Suppl. 01) i57-i64
23 Kiesewetter J, Fischer MR. The Teamwork Assessment Scale: a novel instrument to assess quality of undergraduate medical students' teamwork using the example of simulation-based ward-rounds. GMS Z Med Ausbild 2015; 32: Doc19
24 Frankel A, Gardner R, Maynard L. et al. Using the Communication and Teamwork Skills (CATS) Assessment to measure health care team performance. Jt Comm J Qual Patient Saf 2007; 33: 549-558
25 Undre S, Healey AN, Darzi A. et al. Observational assessment of surgical teamwork: a feasibility study. World J Surg 2006; 30: 1774-1783
26 Hull L, Arora S, Kassab E. et al. Observational teamwork assessment for surgery: content validation and tool refinement. J Am Coll Surg 2011; 212: 234-243.e1-e5
27 Mishra A, Catchpole K, McCulloch P. The Oxford NOTECHS System: reliability and validity of a tool for measuring teamwork behaviour in the operating theatre. Qual Saf Health Care 2009; 18: 104-108
28 Robertson ER, Hadi M, Morgan LJ. et al. Oxford NOTECHS II: a modified theatre team non-technical skills scoring system. PLoS One 2014; 9: e90320
29 Weller J, Frengley R, Torrie J. et al. Evaluation of an instrument to measure teamwork in multidisciplinary critical care teams. BMJ Qual Saf 2011; 20: 216-222
30 Kolbe M, Burtscher MJ, Manser T. Co-ACT--a framework for observing coordination behaviour in acute care teams. BMJ Qual Saf 2013; 22: 596-605
31 Malec JF, Torsher LC, Dunn WF. et al. The mayo high performance teamwork scale: reliability and validity for evaluating key crew resource management skills. Simul Healthc 2007; 2: 4-10
32 Lamb BW, Wong HWL, Vincent C. et al. Teamwork and team performance in multidisciplinary cancer teams: development and evaluation of an observational assessment tool. BMJ Quality & Safety 2011; 20: 849
33 Sutton G, Liao J, Jimmieson NL. et al. Measuring multidisciplinary team effectiveness in a ward-based healthcare setting: development of the team functioning assessment tool. J Healthc Qual 2011; 33: 10-23 quiz 23-24
34 Cook DA, Hatala R. Validation of educational assessments: a primer for simulation and beyond. Adv Simul (Lond) 2016; 1: 31
35 Ching H-L, Lau MS, Azmy IA. et al. Performance measures for the SACRED team-centered approach to advanced gastrointestinal endoscopy: European Society of Gastrointestinal Endoscopy (ESGE) Quality Improvement Initiative. Endoscopy 2022; 54: 712-722

Supplementary Material

Supplementary material