Download PDF
Endoscopy 2023; 55(05): 403-412
DOI: 10.1055/a-1959-6123
Original article

Development of the “Teamwork in Endoscopy Assessment Module for Endoscopic Non-Technical Skills” (TEAM-ENTS) behavioral marker system

Srivathsan Ravindran 
1   Joint Advisory Group on Gastrointestinal Endoscopy, Royal College of Physicians, London, UK
2   Wolfson Unit for Endoscopy, St Mark’s Hospital, London, UK
3   Department of Surgery and Cancer, Imperial College London, London, UK
,
Rebecca Cavilla
4   NHS Improvement, NHS England, London, UK
,
Hutan Ashrafian
3   Department of Surgery and Cancer, Imperial College London, London, UK
,
Adam Haycock 
2   Wolfson Unit for Endoscopy, St Mark’s Hospital, London, UK
,
Chris Healey
5   Department of Gastroenterology, Airedale NHS Foundation Trust, Keighley, UK
,
Mark Coleman
1   Joint Advisory Group on Gastrointestinal Endoscopy, Royal College of Physicians, London, UK
6   Department of Colorectal Surgery, University Hospitals Plymouth NHS Trust, Plymouth, UK
,
Steph Archer
3   Department of Surgery and Cancer, Imperial College London, London, UK
7   Department of Public Health and Primary Care, University of Cambridge, Cambridge, UK
8   Department of Psychology, University of Cambridge, Cambridge, UK
,
Ara Darzi
3   Department of Surgery and Cancer, Imperial College London, London, UK
,
Siwan Thomas-Gibson 
2   Wolfson Unit for Endoscopy, St Mark’s Hospital, London, UK
9   Department of Metabolism, Digestion and Reproduction, Imperial College London, London, UK
,
on behalf of the TEAM-ENTS working group
› Author Affiliations
› Further Information
Also available at
    Permissions and Reprints


Referred to by:

Creating and improving highly effective endoscopic teamsEndoscopy 2023; 55(05): 413-414
DOI: 10.1055/a-2013-2100

Abstract

Background Non-technical skills (NTS) are integral to team performance and subsequent quality and safety of care. Behavioral marker systems (BMSs) are now increasingly used in healthcare to support the training and assessment of team NTS. Within gastrointestinal endoscopy, this is an area of novel research. The aims of this study were to define the core relevant NTS for endoscopy teams and develop a preliminary framework for a team-based BMS known as TEAM-ENTS (Teamwork in Endoscopy Assessment Module for Endoscopic Non-Technical Skills).

Methods This study was conducted in two phases. In phase 1, a literature review of team-based BMSs was performed to inform an interview study of core endoscopy team members. Cognitive task analysis was used to break down the NTS relevant to endoscopy teams. Framework analysis generated the structure for the preliminary TEAM-ENTS framework. In phase 2, a modified Delphi process was undertaken to refine the items of the framework.

Results Seven consultant endoscopists and six nurses were interviewed. The final coding framework consisted of 88 codes grouped into five overarching categories. In total, 58 participants were recruited to the Delphi panel. In the first round, nine elements and 37 behavioral descriptors did not meet consensus. Following item adjustment, merging and deletion, all remaining items met consensus thresholds after the second round. The refined TEAM-ENTS BMS consists of five categories, 16 elements, and 47 behavioral descriptors.

Conclusions The refined TEAM-ENTS behavioral marker system was developed to reflect the core NTS relevant to endoscopy teams. Future studies will aim to fully validate this tool.

Supplementary material



Publication History

Received: 16 May 2022

Accepted after revision: 12 October 2022

Accepted Manuscript online:
12 October 2022

Article published online:
24 November 2022

© 2022. Thieme. All rights reserved.

Georg Thieme Verlag KG
Rüdigerstraße 14, 70469 Stuttgart, Germany

 

  • References

  • 1 Kohn L, Corrigan J, Donaldson M. To Err is Human: Building a safer health system. Washington DC: Institute of Medicine National Academies Press; 1999
    Google Scholar
  • 2 Schmutz J, Manser T. Do team processes really have an effect on clinical performance? A systematic literature review. Br J Anaesth 2013; 110: 529-544
    CrossrefPubMedGoogle Scholar
  • 3 Dietz AS, Pronovost PJ, Benson KN. et al. A systematic review of behavioural marker systems in healthcare: what do we know about their attributes, validity and application?. BMJ Qual Saf 2014; 23: 1031
    CrossrefPubMedGoogle Scholar
  • 4 Higham H, Greig PR, Rutherford J. et al. Observer-based tools for non-technical skills assessment in simulated and real clinical environments in healthcare: a systematic review. BMJ Qual Saf 2019; 28: 672
    CrossrefPubMedGoogle Scholar
  • 5 Boet S, Etherington N, Larrigan S. et al. Measuring the teamwork performance of teams in crisis situations: a systematic review of assessment tools and their measurement properties. BMJ Qual Saf 2019; 28: 327
    CrossrefPubMedGoogle Scholar
  • 6 Wheelock A, Suliman A, Wharton R. et al. The impact of operating room distractions on stress, workload, and teamwork. Ann Surg 2015; 261: 1079-1084
    CrossrefPubMedGoogle Scholar
  • 7 Westli HK, Johnsen BH, Eid J. et al. Teamwork skills, shared mental models, and performance in simulated trauma teams: an independent group design. Scand J Trauma Resusc Emerg Med 2010; 18: 47
    CrossrefPubMedGoogle Scholar
  • 8 Doumouras AG, Hamidi M, Lung K. et al. Non-technical skills of surgeons and anaesthetists in simulated operating theatre crises. Br J Surg 2017; 104: 1028-1036
    CrossrefPubMedGoogle Scholar
  • 9 Ravindran S, Haycock A, Woolf K. et al. Development and impact of an endoscopic non-technical skills (ENTS) behavioural marker system. BMJ Simul Technol Enhanc Learn 2021; 7: 17-25
    CrossrefPubMedGoogle Scholar
  • 10 Joint Advisory Group on Gastrointestinal Endoscopy. Download Centre: Formative DOPS Colonoscopy & Flexible sigmoidoscopy. 2016 Available from (Accessed 19 October 2022): https://www.thejag.org.uk/Downloads/JAG/DOPS%20forms%20(international%20and%20reference%20use%20only)/Formative%20DOPS_Colonoscopy%20and%20Flexible%20sigmoidoscopy.pdf
    PubMedGoogle Scholar
  • 11 Ravindran S, Thomas-Gibson S, Murray S. et al. Improving safety and reducing error in endoscopy: simulation training in human factors. Frontline Gastroenterol 2019; 10: 160-166
    CrossrefPubMedGoogle Scholar
  • 12 Joint Advisory Group on Gastrointestinal Endoscopy. Improving Safety and Reducing Error in Endoscopy (ISREE) Implementation strategy. 2018 Available from (Accessed 19 October 2022): www.thejag.org.uk/Downloads/JAG/General/Improving%20Safety%20and%20Reducing%20Error%20in%20Endoscopy%20(ISREE)%20Implementation%20strategy%20v1.0.pdf
    PubMedGoogle Scholar
  • 13 Clark RE, Feldon DF, van Merriënboer J. et al. Cognitive task analysis. Spector JM, Merrill MD, van Merriënboer J. et al. Handbook of research on educational communications and technology. 3rd edn. Routledge, UK: Routledge; 2007
    Google Scholar
  • 14 Fletcher G, Flin R, McGeorge P. et al. Anaesthetists' non-technical skills (ANTS): evaluation of a behavioural marker system. Br J Anaesth 2003; 90: 580-588
    PubMedGoogle Scholar
  • 15 Gale NK, Heath G, Cameron E. et al. Using the framework method for the analysis of qualitative data in multi-disciplinary health research. BMC Med Res Methodol 2013; 13: 117
    CrossrefPubMedGoogle Scholar
  • 16 Trevelyan EG, Robinson PN. Delphi methodology in health research: how to do it?. Eur J Integr Med 2015; 7: 423-428
    CrossrefPubMedGoogle Scholar
  • 17 Clayton MJ. Delphi: A technique to harness expert opinion for critical decision-making tasks in education. Educ Psychol 1997; 17: 373-386
    CrossrefPubMedGoogle Scholar
  • 18 de Villiers MR, de Villiers PJT, Kent AP. The Delphi technique in health sciences education research. Med Teach 2005; 27: 639-643
    CrossrefPubMedGoogle Scholar
  • 19 Lawshe CH. A quantitative approach to content validity. Pers Psychol 1975; 28: 563-575
    CrossrefPubMedGoogle Scholar
  • 20 Polit DF, Beck CT. The content validity index: are you sure you know what's being reported? Critique and recommendations. Res Nurs Health 2006; 29: 489-497
    CrossrefPubMedGoogle Scholar
  • 21 Dietz AS, Rosen MA, Wyskiel R. et al. Development of a behavioral marker system to assess intensive care unit team performance. Proc Hum Factors Ergon Soc Annu Meet 2015; 59: 991-995
    CrossrefPubMedGoogle Scholar
  • 22 Thomas EJ, Sexton JB, Helmreich RL. Translating teamwork behaviours from aviation to healthcare: development of behavioural markers for neonatal resuscitation. Qual Saf Health Care 2004; 13 (Suppl. 01) i57-i64
    CrossrefPubMedGoogle Scholar
  • 23 Kiesewetter J, Fischer MR. The Teamwork Assessment Scale: a novel instrument to assess quality of undergraduate medical students' teamwork using the example of simulation-based ward-rounds. GMS Z Med Ausbild 2015; 32: Doc19
    PubMedGoogle Scholar
  • 24 Frankel A, Gardner R, Maynard L. et al. Using the Communication and Teamwork Skills (CATS) Assessment to measure health care team performance. Jt Comm J Qual Patient Saf 2007; 33: 549-558
    PubMedGoogle Scholar
  • 25 Undre S, Healey AN, Darzi A. et al. Observational assessment of surgical teamwork: a feasibility study. World J Surg 2006; 30: 1774-1783
    CrossrefPubMedGoogle Scholar
  • 26 Hull L, Arora S, Kassab E. et al. Observational teamwork assessment for surgery: content validation and tool refinement. J Am Coll Surg 2011; 212: 234-243.e1-e5
    CrossrefPubMedGoogle Scholar
  • 27 Mishra A, Catchpole K, McCulloch P. The Oxford NOTECHS System: reliability and validity of a tool for measuring teamwork behaviour in the operating theatre. Qual Saf Health Care 2009; 18: 104-108
    CrossrefPubMedGoogle Scholar
  • 28 Robertson ER, Hadi M, Morgan LJ. et al. Oxford NOTECHS II: a modified theatre team non-technical skills scoring system. PLoS One 2014; 9: e90320
    CrossrefPubMedGoogle Scholar
  • 29 Weller J, Frengley R, Torrie J. et al. Evaluation of an instrument to measure teamwork in multidisciplinary critical care teams. BMJ Qual Saf 2011; 20: 216-222
    CrossrefPubMedGoogle Scholar
  • 30 Kolbe M, Burtscher MJ, Manser T. Co-ACT--a framework for observing coordination behaviour in acute care teams. BMJ Qual Saf 2013; 22: 596-605
    CrossrefPubMedGoogle Scholar
  • 31 Malec JF, Torsher LC, Dunn WF. et al. The mayo high performance teamwork scale: reliability and validity for evaluating key crew resource management skills. Simul Healthc 2007; 2: 4-10
    CrossrefPubMedGoogle Scholar
  • 32 Lamb BW, Wong HWL, Vincent C. et al. Teamwork and team performance in multidisciplinary cancer teams: development and evaluation of an observational assessment tool. BMJ Quality & Safety 2011; 20: 849
    CrossrefPubMedGoogle Scholar
  • 33 Sutton G, Liao J, Jimmieson NL. et al. Measuring multidisciplinary team effectiveness in a ward-based healthcare setting: development of the team functioning assessment tool. J Healthc Qual 2011; 33: 10-23 quiz 23-24
    CrossrefPubMedGoogle Scholar
  • 34 Cook DA, Hatala R. Validation of educational assessments: a primer for simulation and beyond. Adv Simul (Lond) 2016; 1: 31
    CrossrefPubMedGoogle Scholar
  • 35 Ching H-L, Lau MS, Azmy IA. et al. Performance measures for the SACRED team-centered approach to advanced gastrointestinal endoscopy: European Society of Gastrointestinal Endoscopy (ESGE) Quality Improvement Initiative. Endoscopy 2022; 54: 712-722
    Article in Thieme ConnectPubMedGoogle Scholar