Real-time carotid plaque recognition from dynamic ultrasound videos based on artificial neural network

Abstract

Purpose Carotid ultrasound allows noninvasive assessment of vascular anatomy and function with real-time display. Based on the transfer learning method, a series of research results have been obtained on the optimal image recognition and analysis of static images. However, for carotid plaque recognition, there are high requirements for self-developed algorithms in real-time ultrasound detection. This study aims to establish an automatic recognition system, Be Easy to Use (BETU), for the real-time and synchronous diagnosis of carotid plaque from ultrasound videos based on an artificial neural network.

Materials and Methods 445 participants (mean age, 54.6±7.8 years; 227 men) were evaluated. Radiologists labeled a total of 3259 segmented ultrasound images from 445 videos with the diagnosis of carotid plaque, 2725 images were collected as a training dataset, and 554 images as a testing dataset. The automatic plaque recognition system BETU was established based on an artificial neural network, and remote application on a 5G environment was performed to test its diagnostic performance.

Results The diagnostic accuracy of BETU (98.5%) was consistent with the radiologist’s (Kappa = 0.967, P < 0.001). Remote diagnostic feedback based on BETU-processed ultrasound videos could be obtained in 150ms across a distance of 1023 km between the ultrasound/BETU station and the consultation workstation.

Conclusion Based on the good performance of BETU in real-time plaque recognition from ultrasound videos, 5G plus Artificial intelligence (AI)-assisted ultrasound real-time carotid plaque screening was achieved, and the diagnosis was made.

Zusammenfassung

Hintergrund Der Karotis-Ultraschall ermöglicht eine nicht invasive Beurteilung der Anatomie und Funktion von Gefäßen in Echtzeit. Auf der Grundlage der Transfer-Learning-Methode wurden viele Forschungsergebnisse zur optimalen Bilderkennung und Analyse statischer Bilder gewonnen. Für die Erkennung von Karotisplaques bestehen jedoch hohe Anforderungen an selbstentwickelte Algorithmen für die Echtzeit-Ultraschall-Erkennung. Ziel der Studie ist es, ein automatisches Erkennungssystem – Be-Easy-to-Use (BETU) – für die Echtzeit- und Synchrondiagnose von Karotisplaques aus Ultraschallvideos auf Basis eines künstlichen neuronalen Netzes zu entwickeln.

Zu Material und Methoden 445 Teilnehmer (Durchschnittsalter: 54,6 ±7,8 Jahre; 227 davon Männer) wurden untersucht. Radiologen stellten bei insgesamt 3259 segmentierten Ultraschallbildern aus 445 Videos die Diagnose „Karotisplaque“; 2725 Bilder wurden als Trainingsdatensatz und 554 Bilder als Testdatensatz gesammelt. Das automatische Plaque-Erkennungssystem BETU wurde auf Basis eines künstlichen neuronalen Netzes etabliert, und dessen diagnostische Leistung wurde durch eine Remote-Anwendung in einer 5G-Umgebung getestet.

Ergebnisse Die diagnostische Genauigkeit von BETU (98,5%) stimmte mit der des Radiologen überein (Kappa = 0,967; p < 0,001). Ein auf den BETU-prozessierten Ultraschallvideos basierendes Ferndiagnose-Feedback konnte in 150 ms über eine Entfernung von 1023 km zwischen dem Ultraschall-/BETU-System und dem Konsultations-Bildschirm erhalten werden.

Schlussfolgerung Basierend auf der guten Leistung von BETU bei der Echtzeit-Plaque-Erkennung aus Ultraschallvideos wurde ein 5G- plus durch künstliche Intelligenz (KI) gestütztes Echtzeit-Ultraschall-Screening von Karotisplaques durchgeführt und die Diagnose gestellt.

Keywords

Artificial intelligence - Carotid plaque - METHODS & TECHNIQUES - ultrasound - YOLOv4 neural network

Publication History

Received: 06 June 2023

Accepted after revision: 15 September 2023

Article published online:
19 December 2023

• References

• 1 Hodis HN, Mack WJ, LaBree L. et al. The Role of Carotid Arterial Intima-Media Thickness in Predicting Clinical Coronary Events. Ann Intern Med 1998; 128 (04) 262-269
• 2 O’Leary D, Polak J, Kronmal R. et al. Carotid-artery intima and media thickness as a risk factor for myocardial infarction and stroke in older adults. N Engl J Med 1999; 340 (01) 14-22
• 3 Clarke R, Du H, Kurmi O. et al. Burden of carotid artery atherosclerosis in Chinese adults: Implications for future risk of cardiovascular diseases. Eur J Prev Cardiol 2017; 24 (06) 647-656
• 4 Clinical advisory: carotid endarterectomy for patients with asymptomatic internal carotid artery stenosis. Stroke 1994; 25 (12) 2523-2524
• 5 Go AS, Mozaffarian D, Roger VL. et al. Heart disease and stroke statistics-2013 update: a report from the American Heart Association. Circulation 2013; 127 (01) e6-e245
• 6 Rothwell PM, Gibson RJ, Slattery J. et al. Prognostic value and reproducibility of measurements of carotid stenosis. A comparison of three methods on 1001 angiograms. European carotid surgery trialists’ collaborative group. Stroke 1994; 25 (12) 2440-2444
• 7 Loizou CP. A review of ultrasound common carotid artery image and video segmentation techniques. Med Biol Eng Comput 2014; 52 (12) 1073-1093
• 8 Brinjikji W, Huston 3rd J, Rabinstein AA. et al. Contemporary carotid imaging: from degree of stenosis to plaque vulnerability. J Neurosurg 2016; 124 (01) 27-42
• 9 Roy-Cardinal MH, Destrempes F, Soulez G. et al. Assessment of carotid artery plaque components with machine learning classification using homodyned-K parametric maps and elastograms. IEEE Trans Ultrason Ferroelectr Freq Control 2019; 66 (03) 493-504
• 10 Wei Y, Wang M, Gui Y. et al. Carotid artery stiffness in rural adult Chinese: a cross- sectional analysis of the community- based China stroke cohort study. BMJ Open 2020; 10 (10) e036398
• 11 Greenland P, Alpert JS, Beller GA. et al. 2010 ACCF/AHA guideline for assessment of cardiovascular risk in asymptomatic adults: a report of the American College of Cardiology Foundation/American Heart Association Task Force on Practice Guidelines. J Am Coll Cardiol 2010; 56 (25) e50-103
• 12 Brott TG, Halperin JL, Abbara S. et al. 2011 ASA/ACCF/AHA/AANN/AANS/ACR/ASNR/CNS/SAIP/SCAI/SIR/SNIS/SVM/SVS guideline on the management of patients with extracranial carotid and vertebral artery disease. A report of the American College of Cardiology Foundation/American Heart Association Task Force on Practice Guidelines, and the American Stroke Association, American Association of Neuroscience Nurses, American Association of Neurological Surgeons, American College of Radiology, American Society of Neuroradiology, Congress of Neurological Surgeons, Society of Atherosclerosis Imaging and Prevention, Society for Cardiovascular Angiography and Interventions, Society of Interventional Radiology, Society of NeuroInterventional Surgery, Society for Vascular Medicine, and Society for Vascular Surgery. Circulation 2011; 124 (04) e54-130
• 13 Sun W, Zheng B, Huang X. et al. Balance the nodule shape and surroundings: a new multichannel image based convolutional neural network scheme on lung nodule diagnosis. Proc SPIE Medical Imaging 2017; 10134: 101343L
• 14 Pedro LM, Pedro MM, Gonçalves I. et al. Computer-assisted carotid plaque analysis: characteristics of plaques associated with cerebrovascular symptoms and cerebral infarction. Eur J Vasc Endovasc Surg 2000; 19 (02) 118-123
• 15 Menchón-Lara RM, Sancho-Gómez JL, Bueno-Crespo A. et al. Early-stage atherosclerosis detection using deep learning over carotid ultrasound images. Applied Soft Computing 2016; 49: 616-628
• 16 Sun X, Wie WU, Peng WU. et al. Recognition of Carotid Plaque in Ultrasonic Images Based on Deep Convolutional Neural Network. China Medical Device Information 2016; 22 (05) 4-8
• 17 Fang Lu, Fa Wu, Peijun Hu. et al. Automatic 3D liver location and segmentation via convolutional neural network and graph cut. Int J Comput Assist Radiol Surg 2017; 12 (02) 171-182
• 18 Wang BS, Liu J B, Zhu Z. et al. Artificial Intelligence in Ultrasound Imaging: Current Research and Applications. Advanced Ultrasound in Diagnosis and Therapy 2019; 3: 53-61
• 19 Wang L, Yang S, Yang S. et al. Automatic thyroid nodule recognition and diagnosis in ultrasound imaging with the YOLOv2 neural network. World J Surg Oncol 2019; 17 (01) 12
• 20 Xue LY, Jiang ZY, Fu TT. et al. Transfer learning radiomics based on multimodal ultrasound imaging for staging liver fibrosis. Eur Radiol 2020; 30 (05) 2973-2983
• 21 Zhuang Z, Kang Y, Raj ANJ. et al. Breast ultrasound lesion classification based on image decomposition and transfer learning. Med Phys 2020; 47 (12) 6257-6269
• 22 Zhou H, Wang K, Tian J. Online Transfer Learning for Differential Diagnosis of Benign and Malignant Thyroid Nodules with Ultrasound Images. IEEE Trans Biomed Eng 2020; 67 (10) 2773-2780
• 23 Standards for Carotid Ultrasound Examination in Healthy Subjects in China. Chin J Health Manage 2015; 9 (04) 254-260
• 24 Touboul PJ, Hennerici MG, Meairs MG. et al. Mannheim carotid intima-media thickness and plaque consensus (2004–2006–2011). An update on behalf of the advisory board of the 3rd, 4th and 5th watching the risk symposia, at the 13th, 15th and 20th European Stroke Conferences, Mannheim, Germany, 2004, Brussels, Belgium, 2006, and Hamburg, Germany, 2011. Cerebrovasc Dis 2012; 34 (04) 290-296
• 25 Akkus Z, Cai J, Boonrod A. et al. A Survey of Deep-Learning Applications in Ultrasound: Artificial Intelligence-Powered Ultrasound for Improving Clinical Workflow. J Am Coll Radiol 2019; 16: 1318-1328
• 26 Bonanno L, Sottile F, Ciurleo R. et al. Automatic Algorithm for Segmentation of Atherosclerotic Carotid Plaque. J Stroke Cerebrovasc Dis 2017; 26 (02) 411-416
• 27 Lekadir K, Galimzianova A, Betriu A. et al. A Convolutional Neural Network for Automatic Characterization of Plaque Composition in Carotid Ultrasound. IEEE J Biomed Health Inform 2017; 21 (01) 48-55
• 28 Chaudhry A, Hassan M, Khan A. et al. Automatic active contour-based segmentation and classification of carotid artery ultrasound images. J Digit Imaging 2013; 26 (06) 1071-1081
• 29 Redmon J, Divvala S, Girshick R. et al. You Only Look Once: Unified, Real-Time Object Detection. IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2016; 779-788