Benchmarking definitions of false-positive alerts during computer-aided polyp detection in colonoscopy

Abstract

Background The occurrence of false-positive alerts is an important outcome measure in computer-aided colon polyp detection (CADe) studies. However, there is no consensus definition of a false positive in clinical trials evaluating CADe in colonoscopy. We aimed to study the diagnostic performance of CADe based on different threshold definitions for false-positive alerts.

Methods A previously validated CADe system was applied to screening/surveillance colonoscopy videos. Different thresholds for false-positive alerts were defined based on the time an alert box was continuously traced by the system. Primary outcomes were false-positive results and specificity using different threshold definitions of false positive.

Results 62 colonoscopies were analyzed. CADe specificity and accuracy were 93.2 % and 97.8 %, respectively, for a threshold definition of ≥ 0.5 seconds, 98.6 % and 99.5 % for a threshold definition of ≥ 1 second, and 99.8 % and 99.9 % for a threshold definition of ≥ 2 seconds.

Conclusion Our analysis demonstrated how different threshold definitions of false positive can impact the reported diagnostic performance of CADe for colon polyp detection.

Publication History

Received: 26 May 2020

Accepted: 02 November 2020

Accepted Manuscript online:
02 November 2020

Article published online:
18 January 2021

• References

• 1 Mori Y, Kudo SE, Misawa M. et al. Real-time use of artificial intelligence in identification of diminutive polyps during colonoscopy: a prospective study. Ann Intern Med 2018; 169: 357-366
• 2 Urban G, Tripathi P, Alkayali T. et al. Deep learning localizes and identifies polyps in real time with 96% accuracy in screening colonoscopy. Gastroenterology 2018; 155: 1069-1078
• 3 Wang P, Berzin TM, Glissen Brown JR. et al. Real-time automatic detection system increases colonoscopic polyp and adenoma detection rates: a prospective randomized controlled study. Gut 2019; 68: 1813-1819
• 4 Repici A, Badalamenti M, Maselli R. et al. Efficacy of real-time computer-aided detection of colorectal neoplasia in a randomized trial. Gastroenterology 2020; 159: 512-520
• 5 Misawa M, Kudo SE, Mori Y. et al. Artificial intelligence-assisted polyp detection for colonoscopy: initial experience. Gastroenterology 2018; 154: 2027-2029
• 6 Wang P, Liu X, Berzin TM. et al. Effect of a deep-learning computer-aided detection system on adenoma detection during colonoscopy (CADe-DB trial): a double-blind randomised study. Lancet Gastroenterol Hepatol 2020; 5: 343-351
• 7 Wang Z, Liang Z, Li L. et al. Reduction of false positives by internal features for polyp detection in CT-based virtual colonoscopy. Med Phys 2005; 32: 3602-3616
• 8 Wang P, Xiao X, Glissen Brown JR. et al. Development and validation of a deep-learning algorithm for the detection of polyps during colonoscopy. Nat Biomed Eng 2018; 2: 741-748
• 9 Vinsard DG, Mori Y, Misawa M. et al. Quality assurance of computer-aided detection and diagnosis in colonoscopy. Gastrointest Endosc 2019; 90: 55-63
• 10 Alagappan M, Brown JRG, Mori Y. et al. Artificial intelligence in gastrointestinal endoscopy: the future is almost here. World J Gastrointest Endosc 2018; 10: 239-249