The Use of Artificial Intelligence in Automation in the Fields of Gynaecology and Obstetrics – an Assessment of the State of Play

Jan Weichert; Amrei Welp; Jann Lennard Scharf; Christoph Dracopoulos; Wolf-Henning Becker; Michael Gembicki

doi:10.1055/a-1522-3029

Geburtshilfe und Frauenheilkunde, Table of Contents

CC BY-NC-ND 4.0 · Geburtshilfe Frauenheilkd 2021; 81(11): 1203-1216
DOI: 10.1055/a-1522-3029

GebFra Science

Review/Übersicht

The Use of Artificial Intelligence in Automation in the Fields of Gynaecology and Obstetrics – an Assessment of the State of Play

Article in several languages: English | deutsch

Jan Weichert

¹Klinik für Frauenheilkunde und Geburtshilfe, Bereich Pränatalmedizin und Spezielle Geburtshilfe, Universitätsklinikum Schleswig-Holstein, Campus Lübeck, Lübeck, Germany

²Zentrum für Pränatalmedizin an der Elbe, Hamburg, Germany

,

Amrei Welp

¹Klinik für Frauenheilkunde und Geburtshilfe, Bereich Pränatalmedizin und Spezielle Geburtshilfe, Universitätsklinikum Schleswig-Holstein, Campus Lübeck, Lübeck, Germany

,

Jann Lennard Scharf

¹Klinik für Frauenheilkunde und Geburtshilfe, Bereich Pränatalmedizin und Spezielle Geburtshilfe, Universitätsklinikum Schleswig-Holstein, Campus Lübeck, Lübeck, Germany

,

Christoph Dracopoulos

¹Klinik für Frauenheilkunde und Geburtshilfe, Bereich Pränatalmedizin und Spezielle Geburtshilfe, Universitätsklinikum Schleswig-Holstein, Campus Lübeck, Lübeck, Germany

,

Wolf-Henning Becker

²Zentrum für Pränatalmedizin an der Elbe, Hamburg, Germany

,

Michael Gembicki

¹Klinik für Frauenheilkunde und Geburtshilfe, Bereich Pränatalmedizin und Spezielle Geburtshilfe, Universitätsklinikum Schleswig-Holstein, Campus Lübeck, Lübeck, Germany

› Author Affiliations

Abstract

The long-awaited progress in digitalisation is generating huge amounts of medical data every day, and manual analysis and targeted, patient-oriented evaluation of this data is becoming increasingly difficult or even infeasible. This state of affairs and the associated, increasingly complex requirements for individualised precision medicine underline the need for modern software solutions and algorithms across the entire healthcare system. The utilisation of state-of-the-art equipment and techniques in almost all areas of medicine over the past few years has now indeed enabled automation processes to enter – at least in part – into routine clinical practice. Such systems utilise a wide variety of artificial intelligence (AI) techniques, the majority of which have been developed to optimise medical image reconstruction, noise reduction, quality assurance, triage, segmentation, computer-aided detection and classification and, as an emerging field of research, radiogenomics. Tasks handled by AI are completed significantly faster and more precisely, clearly demonstrated by now in the annual findings of the ImageNet Large-Scale Visual Recognition Challenge (ILSVCR), first conducted in 2015, with error rates well below those of humans. This review article will discuss the potential capabilities and currently available applications of AI in gynaecological-obstetric diagnostics. The article will focus, in particular, on automated techniques in prenatal sonographic diagnostics.

Key words

pregnancy - sonography - gynecology - malformation

Full Text

References

References/Literatur
1 Drukker L, Droste R, Chatelain P. et al. Expected-value bias in routine third-trimester growth scans. Ultrasound Obstet Gynecol 2020; 55: 375-382
2 Deng J, Dong W, Socher R. et al. ImageNet: A large-scale hierarchical image database. Paper presented at: 2009 IEEE Conference on Computer Vision and Pattern Recognition; 20 – 25 June 2009. 2009
3 Russakovsky O, Deng J, Su H. et al. ImageNet Large Scale Visual Recognition Challenge. Int J Comput Vis 2015; 115: 211-252
4 Turing AM. I – Computing Machinery and Intelligence. Mind 1950; LIX: 433-460
5 Deloitte. State of AI in the enterprise – 3rd ed. Deloitte, 2020. Accessed September 30, 2021 at: http://www2.deloitte.com/content/dam/Deloitte/de/Documents/technology-media-telecommunications/DELO-6418_State of AI 2020_KS4.pdf
6 Anthes E. Alexa, do I have COVID-19?. Nature 2020; 586: 22-25
7 Huang Z, Epps J, Joachim D. Investigation of Speech Landmark Patterns for Depression Detection. IEEE Transactions on Affective Computing 2019;
8 Bodalal Z, Trebeschi S, Nguyen-Kim TDL. et al. Radiogenomics: bridging imaging and genomics. Abdom Radiol (NY) 2019; 44: 1960-1984
9 Allen B, Dreyer K, McGinty GB. Integrating Artificial Intelligence Into Radiologic Practice: A Look to the Future. J Am Coll Radiol 2020; 17: 280-283
10 Purohit K. Growing Interest in Radiology Despite AI Fears. Acad Radiol 2019; 26: e75
11 Richardson ML, Garwood ER, Lee Y. et al. Noninterpretive Uses of Artificial Intelligence in Radiology. Acad Radiol 2021; 28: 1225-1235
12 Bennani-Baiti B, Baltzer PAT. Künstliche Intelligenz in der Mammadiagnostik. Radiologe 2020; 60: 56-63
13 Chan H-P, Samala RK, Hadjiiski LM. CAD and AI for breast cancer–recent development and challenges. Br J Radiol 2019; 93: 20190580
14 Fujita H. AI-based computer-aided diagnosis (AI-CAD): the latest review to read first. Radiol Phys Technol 2020; 13: 6-19
15 McKinney SM, Sieniek M, Godbole V. et al. International evaluation of an AI system for breast cancer screening. Nature 2020; 577: 89-94
16 Rodriguez-Ruiz A, Lång K, Gubern-Merida A. et al. Can we reduce the workload of mammographic screening by automatic identification of normal exams with artificial intelligence? A feasibility study. Eur Radiol 2019; 29: 4825-4832
17 OʼConnell AM, Bartolotta TV, Orlando A. et al. Diagnostic Performance of An Artificial Intelligence System in Breast Ultrasound. J Ultrasound Med 2021;
18 Cho BJ, Choi YJ, Lee MJ. et al. Classification of cervical neoplasms on colposcopic photography using deep learning. Sci Rep 2020; 10: 13652
19 Shanthi PB, Faruqi F, Hareesha KS. et al. Deep Convolution Neural Network for Malignancy Detection and Classification in Microscopic Uterine Cervix Cell Images. Asian Pac J Cancer Prev 2019; 20: 3447-3456
20 Försch S, Klauschen F, Hufnagl P. et al. Künstliche Intelligenz in der Pathologie. Dtsch Arztebl 2021; 118: 199-204
21 Chang PJ. Moving Artificial Intelligence from Feasible to Real: Time to Drill for Gas and Build Roads. Radiology 2020; 294: 432-433
22 Tran D, Cooke S, Illingworth PJ. et al. Deep learning as a predictive tool for fetal heart pregnancy following time-lapse incubation and blastocyst transfer. Hum Reprod 2019; 34: 1011-1018
23 Zaninovic N, Rosenwaks Z. Artificial intelligence in human in vitro fertilization and embryology. Fertil Steril 2020; 114: 914-920
24 Bori L, Paya E, Alegre L. et al. Novel and conventional embryo parameters as input data for artificial neural networks: an artificial intelligence model applied for prediction of the implantation potential. Fertil Steril 2020; 114: 1232-1241
25 DEGUM. Pressemitteilungen. DEGUM, 2017. Updated 29.11.2017. Accessed September 30, 2021 at: http://www.degum.de/aktuelles/presse-medien/pressemitteilungen/im-detail/news/zu-viele-kindliche-fehlbildungen-bleiben-unentdeckt.html
26 Murugesu S, Galazis N, Jones BP. et al. Evaluating the use of telemedicine in gynaecological practice: a systematic review. BMJ Open 2020; 10: e039457
27 Benacerraf BR, Minton KK, Benson CB. et al. Proceedings: Beyond Ultrasound First Forum on Improving the Quality of Ultrasound Imaging in Obstetrics and Gynecology. J Ultrasound Med 2018; 37: 7-18
28 Timmerman D, Verrelst H, Bourne TH. et al. Artificial neural network models for the preoperative discrimination between malignant and benign adnexal masses. Ultrasound Obstet Gynecol 1999; 13: 17-25
29 Froyman W, Timmerman D. Methods of Assessing Ovarian Masses: International Ovarian Tumor Analysis Approach. Obstet Gynecol Clin North Am 2019; 46: 625-641
30 Van Calster B, Van Hoorde K, Valentin L. et al. Evaluating the risk of ovarian cancer before surgery using the ADNEX model to differentiate between benign, borderline, early and advanced stage invasive, and secondary metastatic tumours: prospective multicentre diagnostic study. BMJ 2014; 349: g5920
31 Vázquez-Manjarrez SE, Rico-Rodriguez OC, Guzman-Martinez N. et al. Imaging and diagnostic approach of the adnexal mass: what the oncologist should know. Chin Clin Oncol 2020; 9: 69
32 Andreotti RF, Timmerman D, Strachowski LM. et al. O-RADS US Risk Stratification and Management System: A Consensus Guideline from the ACR Ovarian-Adnexal Reporting and Data System Committee. Radiology 2020; 294: 168-185
33 Christiansen F, Epstein EL, Smedberg E. et al. Ultrasound image analysis using deep neural networks for discriminating between benign and malignant ovarian tumors: comparison with expert subjective assessment. Ultrasound Obstet Gynecol 2021; 57: 155-163
34 Acharya UR, Mookiah MR, Vinitha Sree S. et al. Evolutionary algorithm-based classifier parameter tuning for automatic ovarian cancer tissue characterization and classification. Ultraschall Med 2014; 35: 237-245
35 Akazawa M, Hashimoto K. Artificial Intelligence in Ovarian Cancer Diagnosis. Anticancer Res 2020; 40: 4795-4800
36 Aramendia-Vidaurreta V, Cabeza R, Villanueva A. et al. Ultrasound Image Discrimination between Benign and Malignant Adnexal Masses Based on a Neural Network Approach. Ultrasound Med Biol 2016; 42: 742-752
37 Khazendar S, Sayasneh A, Al-Assam H. et al. Automated characterisation of ultrasound images of ovarian tumours: the diagnostic accuracy of a support vector machine and image processing with a local binary pattern operator. Facts Views Vis Obgyn 2015; 7: 7-15
38 Zhou J, Zeng ZY, Li L. Progress of Artificial Intelligence in Gynecological Malignant Tumors. Cancer Manag Res 2020; 12: 12823-12840
39 Al-Karawi D, Al-Assam H, Du H. et al. An Evaluation of the Effectiveness of Image-based Texture Features Extracted from Static B-mode Ultrasound Images in Distinguishing between Benign and Malignant Ovarian Masses. Ultrason Imaging 2021; 43: 124-138
40 Bakker MK, Bergman JEH, Krikov S. et al. Prenatal diagnosis and prevalence of critical congenital heart defects: an international retrospective cohort study. BMJ Open 2019; 9: e028139
41 van Nisselrooij AEL, Teunissen AKK, Clur SA. et al. Why are congenital heart defects being missed?. Ultrasound Obstet Gynecol 2020; 55: 747-757
42 Knackstedt C, Bekkers SC, Schummers G. et al. Fully Automated Versus Standard Tracking of Left Ventricular Ejection Fraction and Longitudinal Strain: The FAST-EFs Multicenter Study. J Am Coll Cardiol 2015; 66: 1456-1466
43 Tsang W, Salgo IS, Medvedofsky D. et al. Transthoracic 3D Echocardiographic Left Heart Chamber Quantification Using an Automated Adaptive Analytics Algorithm. JACC Cardiovasc Imaging 2016; 9: 769-782
44 Kusunose K. Steps to use artificial intelligence in echocardiography. J Echocardiogr 2021; 19: 21-27
45 Zhang J, Gajjala S, Agrawal P. et al. Fully Automated Echocardiogram Interpretation in Clinical Practice. Circulation 2018; 138: 1623-1635
46 Harari YN. Homo sapiens verliert die Kontrolle. Die Große Entkopplung. In: Homo Deus – Eine Geschichte von Morgen. 16. Aufl.. München: C. H. Beck; 2020
47 Gandhi S, Mosleh W, Shen J. et al. Automation, machine learning, and artificial intelligence in echocardiography: A brave new world. Echocardiography 2018; 35: 1402-1418
48 Arnaout R, Curran L, Zhao Y. et al. Expert-level prenatal detection of complex congenital heart disease from screening ultrasound using deep learning. medRxiv 2020;
49 Le TK, Truong V, Nguyen-Vo TH. et al. Application of machine learning in screening of congenital heart diseases using fetal echocardiography. J Am Coll Cardiol 2020; 75: 648
50 Dong J, Liu S, Liao Y. et al. A Generic Quality Control Framework for Fetal Ultrasound Cardiac Four-Chamber Planes. IEEE J Biomed Health Inform 2020; 24: 931-942
51 Hinton GE. To recognize shapes, first learn to generate images. Prog Brain Res 2007; 165: 535-547
52 Voelker R. Cardiac Ultrasound Uses Artificial Intelligence to Produce Images. JAMA 2020; 323: 1034
53 Yeo L, Romero R. Optical ultrasound simulation-based training in obstetric sonography. J Matern Fetal Neonatal Med 2020;
54 Steinhard J, Dammeme Debbih A, Laser KT. et al. Randomised controlled study on the use of systematic simulator-based training (OPUS Fetal Heart Trainer) for learning the standard heart planes in fetal echocardiography. Ultrasound Obstet Gynecol 2019; 54 (S1): 28-29
55 Day TG, Kainz B, Hajnal J. et al. Artificial intelligence, fetal echocardiography, and congenital heart disease. Prenat Diagn 2021; 41: 733-742
56 Garcia-Canadilla P, Sanchez-Martinez S, Crispi F. et al. Machine Learning in Fetal Cardiology: What to Expect. Fetal Diagn Ther 2020; 47: 363-372
57 Meiburger KM, Acharya UR, Molinari F. Automated localization and segmentation techniques for B-mode ultrasound images: A review. Comput Biol Med 2018; 92: 210-235
58 Rawat V, Jain A, Shrimali V. Automated Techniques for the Interpretation of Fetal Abnormalities: A Review. Appl Bionics Biomech 2018; 2018: 6452050
59 Yeo L, Luewan S, Romero R. Fetal Intelligent Navigation Echocardiography (FINE) Detects 98 % of Congenital Heart Disease. J Ultrasound Med 2018; 37: 2577-2593
60 Gembicki M, Hartge DR, Dracopoulos C. et al. Semiautomatic Fetal Intelligent Navigation Echocardiography Has the Potential to Aid Cardiac Evaluations Even in Less Experienced Hands. J Ultrasound Med 2020; 39: 301-309
61 Weichert J, Weichert A. A “holistic” sonographic view on congenital heart disease: How automatic reconstruction using fetal intelligent navigation echocardiography eases unveiling of abnormal cardiac anatomy part II-Left heart anomalies. Echocardiography 2021; 38: 777-789
62 DeVore GR, Klas B, Satou G. et al. Longitudinal Annular Systolic Displacement Compared to Global Strain in Normal Fetal Hearts and Those With Cardiac Abnormalities. J Ultrasound Med 2018; 37: 1159-1171
63 DeVore GR, Klas B, Satou G. et al. 24-segment sphericity index: a new technique to evaluate fetal cardiac diastolic shape. Ultrasound Obstet Gynecol 2018; 51: 650-658
64 DeVore GR, Polanco B, Satou G. et al. Two-Dimensional Speckle Tracking of the Fetal Heart: A Practical Step-by-Step Approach for the Fetal Sonologist. J Ultrasound Med 2016; 35: 1765-1781
65 Lee M, Won H. Novel technique for measurement of fetal right myocardial performance index using synchronised images of right ventricular inflow and outflow. Ultrasound Obstet Gynecol 2019; 54 (S1): 178-179
66 Leung V, Avnet H, Henry A. et al. Automation of the Fetal Right Myocardial Performance Index to Optimise Repeatability. Fetal Diagn Ther 2018; 44: 28-35
67 Rizzo G, Aiello E, Pietrolucci ME. et al. The feasibility of using 5D CNS software in obtaining standard fetal head measurements from volumes acquired by three-dimensional ultrasonography: comparison with two-dimensional ultrasound. J Matern Fetal Neonatal Med 2016; 29: 2217-2222
68 Welp A, Gembicki M, Rody A. et al. Validation of a semiautomated volumetric approach for fetal neurosonography using 5DCNS+ in clinical data from > 1100 consecutive pregnancies. Childs Nerv Syst 2020; 36: 2989-2995
69 Pluym ID, Afshar Y, Holliman K. et al. Accuracy of three-dimensional automated ultrasound imaging of biometric measurements of the fetal brain. Ultrasound Obstet Gynecol 2021; 57: 798-803
70 Ambroise Grandjean G, Hossu G, Bertholdt C. et al. Artificial intelligence assistance for fetal head biometry: Assessment of automated measurement software. Diagn Interv Imaging 2018; 99: 709-716
71 Huang R, Xie W, Alison Noble J. VP-Nets: Efficient automatic localization of key brain structures in 3D fetal neurosonography. Med Image Anal 2018; 47: 127-139
72 Xie HN, Wang N, He M. et al. Using deep-learning algorithms to classify fetal brain ultrasound images as normal or abnormal. Ultrasound Obstet Gynecol 2020; 56: 579-587
73 Cerrolaza JJ, Li Y, Biffi C. et al. Fetal Skull Reconstruction via Deep Convolutional Autoencoders. Annu Int Conf IEEE Eng Med Biol Soc 2018; 2018: 887-890
74 Ghesu FC, Georgescu B, Grbic S. et al. Towards intelligent robust detection of anatomical structures in incomplete volumetric data. Med Image Anal 2018; 48: 203-213
75 Cai Y, Droste R, Sharma H. et al. Spatio-temporal visual attention modelling of standard biometry plane-finding navigation. Medical Image Analysis 2020; 65: 101762
76 Baumgartner CF, Kamnitsas K, Matthew J. et al. SonoNet: Real-Time Detection and Localisation of Fetal Standard Scan Planes in Freehand Ultrasound. IEEE Trans Med Imaging 2017; 36: 2204-2215
77 Yaqub M, Kelly B, Noble JA. et al. An AI system to support sonologists during fetal ultrasound anomaly screening. Ultrasound Obstet Gynecol 2018; 52 (S1): 9-10
78 Yaqub M, Sleep N, Syme S. et al. ScanNav^® audit: an AI-powered screening assistant for fetal anatomical ultrasound. Am J Obstet Gynecol 2021; 224 (Suppl.) S312
79 Sharma H, Drukker L, Chatelain P. et al. Knowledge representation and learning of operator clinical workflow from full-length routine fetal ultrasound scan videos. Med Image Anal 2021; 69: 101973
80 Droste R, Drukker L, Papageorghiou AT. et al. Automatic Probe Movement Guidance for Freehand Obstetric Ultrasound. Med Image Comput Comput Assist Interv 2020; 12263: 583-592
81 Alsharid M, Sharma H, Drukker L. et al. Captioning Ultrasound Images Automatically. In: Shen D. et al., eds. Medical Image Computing and Computer Assisted Intervention – MICCAI 2019. MICCAI 2019. (Lecture Notes in Computer Science, vol 11767; ). Cham: Springer; 2019
82 Lee W, Deter RL, Ebersole JD. et al. Birth weight prediction by three-dimensional ultrasonography: fractional limb volume. J Ultrasound Med 2001; 20: 1283-1292
83 Corrêa VM, Araujo Júnior E, Braga A. et al. Prediction of birth weight in twin pregnancies using fractional limb volumes by three-dimensional ultrasonography. J Matern Fetal Neonatal Med 2020; 33: 3652-3657
84 Gembicki M, Offerman DR, Weichert J. Semiautomatic Assessment of Fetal Fractional Limb Volume for Weight Prediction in Clinical Praxis: How Does It Perform in Routine Use?. J Ultrasound Med 2021;
85 Mack LM, Kim SY, Lee S. et al. Automated Fractional Limb Volume Measurements Improve the Precision of Birth Weight Predictions in Late Third-Trimester Fetuses. J Ultrasound Med 2017; 36: 1649-1655
86 Youssef A, Salsi G, Montaguti E. et al. Automated Measurement of the Angle of Progression in Labor: A Feasibility and Reliability Study. Fetal Diagn Ther 2017; 41: 293-299
87 Brocklehurst P, Field D, Greene K. et al. Computerised interpretation of fetal heart rate during labour (INFANT): a randomised controlled trial. Lancet 2017; 389: 1719-1729
88 Keith R. The INFANT study-a flawed design foreseen. Lancet 2017; 389: 1697-1698
89 Silver RM. Computerising the intrapartum continuous cardiotocography does not add to its predictive value: FOR: Computer analysis does not add to intrapartum continuous cardiotocography predictive value. BJOG 2019; 126: 1363
90 Gyllencreutz E, Lu K, Lindecrantz K. et al. Validation of a computerized algorithm to quantify fetal heart rate deceleration area. Acta Obstet Gynecol Scand 2018; 97: 1137-1147
91 Fung R, Villar J, Dashti A. et al. International Fetal and Newborn Growth Consortium for the 21st Century (INTERGROWTH-21st). Achieving accurate estimates of fetal gestational age and personalised predictions of fetal growth based on data from an international prospective cohort study: a population-based machine learning study. Lancet Digit Health 2020; 2: e368-e375
92 Lee KS, Ahn KH. Application of Artificial Intelligence in Early Diagnosis of Spontaneous Preterm Labor and Birth. Diagnostics (Basel) 2020; 10: 733
93 Maassen O, Fritsch S, Palm J. et al. Future Medical Artificial Intelligence Application Requirements and Expectations of Physicians in German University Hospitals: Web-Based Survey. J Med Internet Res 2021; 23: e26646
94 Littmann M, Selig K, Cohen-Lavi L. et al. Validity of machine learning in biology and medicine increased through collaborations across fields of expertise. Nature Machine Intelligence 2020; 2: 18-24
95 Norgeot B, Glicksberg BS, Butte AJ. A call for deep-learning healthcare. Nat Med 2019; 25: 14-15
96 Borck C. Communicating the Modern Body: Fritz Kahnʼs Popular Images of Human Physiology as an Industrialized World. Canadian Journal of Communication 2007; 32: 495-520
97 Jachertz N. Populärmedizin: Der Mensch ist eine Maschine, die vom Menschen bedient wird. Dtsch Arztebl 2010; 107: A-391-393
98 Frey CB, Osborne MA. The future of employment: How susceptible are jobs to computerisation?. Technological Forecasting and Social Change 2017; 114: 254-280
99 Gartner H, Stüber H. Strukturwandel am Arbeitsmarkt seit den 70er Jahren: Arbeitsplatzverluste werden durch neue Arbeitsplätze immer wieder ausgeglichen. 16.7.2019. Nürnberg: Institut für Arbeitsmarkt- und Berufsforschung; 2019
100 Bartoli A, Quarello E, Voznyuk I. et al. Intelligence artificielle et imagerie en médecine fœtale: de quoi parle-t-on? [Artificial intelligence and fetal imaging: What are we talking about?]. Gynecol Obstet Fertil Senol 2019; 47: 765-768
101 Allen jr. B, Seltzer SE, Langlotz CP. et al. A Road Map for Translational Research on Artificial Intelligence in Medical Imaging: From the 2018 National Institutes of Health/RSNA/ACR/The Academy Workshop. J Am Coll Radiol 2019; 16 (9 Pt A): 1179-1189
102 Langlotz CP, Allen B, Erickson BJ. et al. A Roadmap for Foundational Research on Artificial Intelligence in Medical Imaging: From the 2018 NIH/RSNA/ACR/The Academy Workshop. Radiology 2019; 291: 781-791
103 Tolsgaard MG, Svendsen MBS, Thybo JK. et al. Does artificial intelligence for classifying ultrasound imaging generalize between different populations and contexts?. Ultrasound Obstet Gynecol 2021; 57: 342-343
104 Davenport T, Kalakota R. The potential for artificial intelligence in healthcare. Future Healthc J 2019; 6: 94-98