Reproducibility and Explainability of Deep Learning in Mammography: A Systematic Review of Literature

Abstract

Background Although abundant literature is currently available on the use of deep learning for breast cancer detection in mammography, the quality of such literature is widely variable.

Purpose To evaluate published literature on breast cancer detection in mammography for reproducibility and to ascertain best practices for model design.

Methods The PubMed and Scopus databases were searched to identify records that described the use of deep learning to detect lesions or classify images into cancer or noncancer. A modification of Quality Assessment of Diagnostic Accuracy Studies (mQUADAS-2) tool was developed for this review and was applied to the included studies. Results of reported studies (area under curve [AUC] of receiver operator curve [ROC] curve, sensitivity, specificity) were recorded.

Results A total of 12,123 records were screened, of which 107 fit the inclusion criteria. Training and test datasets, key idea behind model architecture, and results were recorded for these studies. Based on mQUADAS-2 assessment, 103 studies had high risk of bias due to nonrepresentative patient selection. Four studies were of adequate quality, of which three trained their own model, and one used a commercial network. Ensemble models were used in two of these. Common strategies used for model training included patch classifiers, image classification networks (ResNet in 67%), and object detection networks (RetinaNet in 67%). The highest reported AUC was 0.927 ± 0.008 on a screening dataset, while it reached 0.945 (0.919–0.968) on an enriched subset. Higher values of AUC (0.955) and specificity (98.5%) were reached when combined radiologist and Artificial Intelligence readings were used than either of them alone. None of the studies provided explainability beyond localization accuracy. None of the studies have studied interaction between AI and radiologist in a real world setting.

Conclusion While deep learning holds much promise in mammography interpretation, evaluation in a reproducible clinical setting and explainable networks are the need of the hour.

Keywords

artificial intelligence - breast cancer - deep learning - mammography - neural networks - systematic review

Publikationsverlauf

Artikel online veröffentlicht:
10. Oktober 2023

• References

• 1 Warren Burhenne LJ, Wood SA, D'Orsi CJ. et al. Potential contribution of computer-aided detection to the sensitivity of screening mammography. Radiology 2000; 215 (02) 554-562
• 2 Birdwell RL, Ikeda DM, O'Shaughnessy KF, Sickles EA. Mammographic characteristics of 115 missed cancers later detected with screening mammography and the potential utility of computer-aided detection. Radiology 2001; 219 (01) 192-202
• 3 Birdwell RL, Bandodkar P, Ikeda DM. Computer-aided detection with screening mammography in a university hospital setting. Radiology 2005; 236 (02) 451-457
• 4 Brem RF, Baum J, Lechner M. et al. Improvement in sensitivity of screening mammography with computer-aided detection: a multiinstitutional trial. Am J Roentgenol 2003; 181 (03) 687-693
• 5 Freeman K, Geppert J, Stinton C. et al. Use of artificial intelligence for image analysis in breast cancer screening programmes: systematic review of test accuracy. BMJ 2021; 374: n1872
• 6 Wang X, Liang G, Zhang Y, Blanton H, Bessinger Z, Jacobs N. Inconsistent performance of deep learning models on mammogram classification. J Am Coll Radiol 2020; 17 (06) 796-803
• 7 miccai reproducibility - Google Search. Accessed May 16, 2022 at: https://www.google.com/search?q=miccai+reproducibility&oq=miccai+&aqs=chrome.1.69i57j35i39j0i512j0i20i263i512j0i512l6.4510j0j4&sourceid=chrome&ie=UTF-8
• 8 Mongan J, Moy L, Kahn Jr CE. Checklist for Artificial Intelligence in Medical Imaging (CLAIM): a guide for authors and reviewers. Radiol Artif Intell 2020; 2 (02) e200029
• 9 Li X, Xiong H, Li X. et al. Interpretable deep learning: Interpretation, interpretability, trustworthiness, and beyond. Knowl Inf Syst 2022; 64 (12) 3197-3234
• 10 McInnes MDF, Moher D, Thombs BD. et al; and the PRISMA-DTA Group. Preferred Reporting Items for a Systematic Review and Meta-analysis of Diagnostic Test Accuracy Studies: the PRISMA-DTA statement. JAMA 2018; 319 (04) 388-396
• 11 Whiting PF, Rutjes AWS, Westwood ME. et al; QUADAS-2 Group. QUADAS-2: a revised tool for the quality assessment of diagnostic accuracy studies. Ann Intern Med 2011; 155 (08) 529-536
• 12 Kooi T, Litjens G, van Ginneken B. et al. Large scale deep learning for computer aided detection of mammographic lesions. Med Image Anal 2017; 35: 303-312
• 13 Ribli D, Horváth A, Unger Z, Pollner P, Csabai I. Detecting and classifying lesions in mammograms with Deep Learning. Sci Rep 2018; 8 (01) 4165
• 14 Dhungel N, Carneiro G, Bradley AP. A deep learning approach for the analysis of masses in mammograms with minimal user intervention. Med Image Anal 2017; 37: 114-128
• 15 Al-Antari MA, Al-Masni MA, Choi MT, Han SM, Kim TS. A fully integrated computer-aided diagnosis system for digital X-ray mammograms via deep learning detection, segmentation, and classification. Int J Med Inform 2018; 117: 44-54
• 16 Al-Masni MA, Al-Antari MA, Park JM. et al. Detection and classification of the breast abnormalities in digital mammograms via regional Convolutional Neural Network. Annu Int Conf IEEE Eng Med Biol Soc 2017; 2017: 1230-1233
• 17 Li C, Zhang D, Tian Z, Du S, Qu Y. Few-shot learning with deformable convolution for multiscale lesion detection in mammography. Med Phys 2020; 47 (07) 2970-2985
• 18 Agarwal R, Díaz O, Yap MH, Lladó X, Martí R. Deep learning for mass detection in full field digital mammograms. Comput Biol Med 2020; 121: 103774
• 19 Al-Masni MA, Al-Antari MA, Park JM. et al. Simultaneous detection and classification of breast masses in digital mammograms via a deep learning YOLO-based CAD system. Comput Methods Programs Biomed 2018; 157: 85-94
• 20 Bandeira Diniz JO, Bandeira Diniz PH, Azevedo Valente TL, Corrêa Silva A, de Paiva AC, Gattass M. Detection of mass regions in mammograms by bilateral analysis adapted to breast density using similarity indexes and convolutional neural networks. Comput Methods Programs Biomed 2018; 156: 191-207
• 21 Jung H, Kim B, Lee I. et al. Detection of masses in mammograms using a one-stage object detector based on a deep convolutional neural network. PLoS One 2018; 13 (09) e0203355
• 22 Savelli B, Bria A, Molinara M, Marrocco C, Tortorella F. A multi-context CNN ensemble for small lesion detection. Artif Intell Med 2020; 103: 101749
• 23 Valvano G, Santini G, Martini N. et al. Convolutional neural networks for the segmentation of microcalcification in mammography imaging. J Healthc Eng 2019; 2019: 9360941
• 24 Sarath CK, Chakravarty A, Ghosh N, Sarkar T, Sethuraman R, Sheet D. A two-stage multiple instance learning framework for the detection of breast cancer in mammograms. Annu Int Conf IEEE Eng Med Biol Soc 2020; 2020: 1128-1131
• 25 Wang H, Feng J, Bu Q. et al. Breast mass detection in digital mammogram based on gestalt psychology. J Healthc Eng 2018; 2018: 4015613
• 26 Cha KH, Petrick N, Pezeshk A. et al. Evaluation of data augmentation via synthetic images for improved breast mass detection on mammograms using deep learning. J Med Imaging (Bellingham) 2020; 7 (01) 012703
• 27 Wang J, Nishikawa RM, Yang Y. Global detection approach for clustered microcalcifications in mammograms using a deep learning network. J Med Imaging (Bellingham) 2017; 4 (02) 024501
• 28 Shin SY, Lee S, Yun ID. et al. A novel cascade classifier for automatic microcalcification detection. PLoS One 2015; 10 (12) e0143725
• 29 Wang J, Yang Y. A context-sensitive deep learning approach for microcalcification detection in mammograms. Pattern Recognit 2018; 78: 12-22
• 30 Fathy W, Ghoneim A. A deep learning approach for breast cancer mass detection. Int J Adv Comput Sci Appl 2019; 10: 175-182
• 31 Li H, Ye J, Liu H. et al. Application of deep learning in the detection of breast lesions with four different breast densities. Cancer Med 2021; 10 (14) 4994-5000
• 32 Yi PH, Singh D, Harvey SC, Hager GD, Mullen LA. DeepCAT: deep computer-aided triage of screening mammography. J Digit Imaging 2021; 34 (01) 27-35
• 33 Sun L, Sun H, Wang J, Wu S, Zhao Y, Xu Y. Breast mass detection in mammography based on image template matching and CNN. Sensors (Basel) 2021; 21 (08) 2855
• 34 Aly GH, Marey M, El-Sayed SA, Tolba MF. YOLO based breast masses detection and classification in full-field digital mammograms. Comput Methods Programs Biomed 2021; 200: 105823
• 35 Isfahani ZN, Jannat-Dastjerdi I, Eskandari F, Ghoushchi SJ, Pourasad Y. Presentation of novel hybrid algorithm for detection and classification of breast cancer using growth region method and probabilistic neural network. Comput Intell Neurosci 2021; 2021: 5863496
• 36 Sarangi S, Rath NP, Sahoo HK. Mammogram mass segmentation and detection using Legendre neural network-based optimal threshold. Med Biol Eng Comput 2021; 59 (04) 947-955
• 37 Liu Y, Zhou C, Zhang F. et al. Compare and contrast: detecting mammographic soft-tissue lesions with C²-Net. Med Image Anal 2021; 71: 101999
• 38 Shen L, Margolies LR, Rothstein JH, Fluder E, McBride R, Sieh W. Deep learning to improve breast cancer detection on screening mammography. Sci Rep 2019; 9 (01) 12495
• 39 Aboutalib SS, Mohamed AA, Berg WA, Zuley ML, Sumkin JH, Wu S. Deep learning to distinguish recalled but benign mammography images in breast cancer screening. Clin Cancer Res 2018; 24 (23) 5902-5909
• 40 Becker AS, Marcon M, Ghafoor S, Wurnig MC, Frauenfelder T, Boss A. Deep learning in mammography: diagnostic accuracy of a multipurpose image analysis software in the detection of breast cancer. Invest Radiol 2017; 52 (07) 434-440
• 41 Wang J, Yang X, Cai H, Tan W, Jin C, Li L. Discrimination of breast cancer with microcalcifications on mammography by deep learning. Sci Rep 2016; 6: 27327
• 42 Arevalo J, González FA, Ramos-Pollán R, Oliveira JL, Guevara Lopez MA. Representation learning for mammography mass lesion classification with convolutional neural networks. Comput Methods Programs Biomed 2016; 127: 248-257
• 43 Yala A, Schuster T, Miles R, Barzilay R, Lehman C. A deep learning model to triage screening mammograms: a simulation study. Radiology 2019; 293 (01) 38-46
• 44 Samala RK, Chan HP, Hadjiiski LM, Helvie MA, Cha KH, Richter CD. Multi-task transfer learning deep convolutional neural network: application to computer-aided diagnosis of breast cancer on mammograms. Phys Med Biol 2017; 62 (23) 8894-8908
• 45 He T, Puppala M, Ezeana CF. et al. A deep learning-based decision support tool for precision risk assessment of breast cancer. JCO Clin Cancer Inform 2019; 3: 1-12
• 46 Kim ST, Lee JH, Lee H, Ro YM. Visually interpretable deep network for diagnosis of breast masses on mammograms. Phys Med Biol 2018; 63 (23) 235025
• 47 Carneiro G, Nascimento J, Bradley AP. Automated analysis of unregistered multi-view mammograms with deep learning. IEEE Trans Med Imaging 2017; 36 (11) 2355-2365
• 48 Chougrad H, Zouaki H, Alheyane O. Deep convolutional neural networks for breast cancer screening. Comput Methods Programs Biomed 2018; 157: 19-30
• 49 Cai H, Huang Q, Rong W. et al. Breast microcalcification diagnosis using deep convolutional neural network from digital mammograms. Comput Math Methods Med 2019; 2019: 2717454
• 50 Bruno A, Ardizzone E, Vitabile S, Midiri M. A novel solution based on scale invariant feature transform descriptors and deep learning for the detection of suspicious regions in mammogram images. J Med Signals Sens 2020; 10 (03) 158-173
• 51 Arora R, Rai PK, Raman B. Deep feature-based automatic classification of mammograms. Med Biol Eng Comput 2020; 58 (06) 1199-1211
• 52 Zhang X, Zhang Y, Han EY. et al. Classification of whole mammogram and tomosynthesis images using deep convolutional neural networks. IEEE Trans Nanobiosci 2018; 17 (03) 237-242
• 53 Sun W, Tseng TB, Zhang J, Qian W. Enhancing deep convolutional neural network scheme for breast cancer diagnosis with unlabeled data. Comput Med Imaging Graph 2017; 57: 4-9
• 54 Muramatsu C, Nishio M, Goto T. et al. Improving breast mass classification by shared data with domain transformation using a generative adversarial network. Comput Biol Med 2020; 119: 103698
• 55 Shen Y, Wu N, Phang J. et al. An interpretable classifier for high-resolution breast cancer screening images utilizing weakly supervised localization. Med Image Anal 2021; 68: 101908
• 56 Guan S, Loew M. Breast cancer detection using synthetic mammograms from generative adversarial networks in convolutional neural networks. J Med Imaging (Bellingham) 2019; 6 (03) 031411
• 57 Antropova N, Huynh BQ, Giger ML. A deep feature fusion methodology for breast cancer diagnosis demonstrated on three imaging modality datasets. Med Phys 2017; 44 (10) 5162-5171
• 58 Agnes SA, Anitha J, Pandian SIA, Peter JD. Classification of mammogram images using multiscale all convolutional neural network (MA-CNN). J Med Syst 2019; 44 (01) 30
• 59 Jadoon MM, Zhang Q, Haq IU, Butt S, Jadoon A. Three-class mammogram classification based on descriptive CNN features. BioMed Res Int 2017; 2017: 3640901
• 60 Zhang C, Zhao J, Niu J, Li D. New convolutional neural network model for screening and diagnosis of mammograms. PLoS One 2020; 15 (08) e0237674
• 61 Shu X, Zhang L, Wang Z, Lv Q, Yi Z. Deep neural networks with region-based pooling structures for mammographic image classification. IEEE Trans Med Imaging 2020; 39 (06) 2246-2255
• 62 Lång K, Dustler M, Dahlblom V, Åkesson A, Andersson I, Zackrisson S. Identifying normal mammograms in a large screening population using artificial intelligence. Eur Radiol 2021; 31 (03) 1687-1692
• 63 Kooi T, Karssemeijer N. Classifying symmetrical differences and temporal change for the detection of malignant masses in mammography using deep neural networks. J Med Imaging (Bellingham) 2017; 4 (04) 044501
• 64 Samala RK, Chan HP, Hadjiiski L, Helvie MA. Risks of feature leakage and sample size dependencies in deep feature extraction for breast mass classification. Med Phys 2021; 48 (06) 2827-2837
• 65 Duggento A, Aiello M, Cavaliere C. et al. An ad hoc random initialization deep neural network architecture for discriminating malignant breast cancer lesions in mammographic images. Contrast Media Mol Imaging 2019; 2019: 5982834
• 66 Sawyer Lee R, Dunnmon JA, He A, Tang S, Ré C, Rubin DL. Comparison of segmentation-free and segmentation-dependent computer-aided diagnosis of breast masses on a public mammography dataset. J Biomed Inform 2021; 113: 103656
• 67 Cogan T, Cogan M, Tamil L. RAMS: remote and automatic mammogram screening. Comput Biol Med 2019; 107: 18-29
• 68 Ragab DA, Sharkas M, Marshall S, Ren J. Breast cancer detection using deep convolutional neural networks and support vector machines. PeerJ 2019; 7: e6201
• 69 Shen Y, Wu N, Phang J. et al. Globally-aware multiple instance classifier for breast cancer screening. Mach Learn Med Imaging 2019; 11861: 18-26
• 70 Qiu Y, Yan S, Gundreddy RR. et al. A new approach to develop computer-aided diagnosis scheme of breast mass classification using deep learning technology. J XRay Sci Technol 2017; 25 (05) 751-763
• 71 Teare P, Fishman M, Benzaquen O, Toledano E, Elnekave E. Malignancy detection on mammography using dual deep convolutional neural networks and genetically discovered false color input enhancement. J Digit Imaging 2017; 30 (04) 499-505
• 72 Guan Y, Wang X, Li H. et al. Detecting asymmetric patterns and localizing cancers on mammograms. Patterns (N Y) 2020; 1 (07) 100106
• 73 Rodriguez-Ruiz A, Lång K, Gubern-Merida A. et al. Stand-alone artificial intelligence for breast cancer detection in mammography: comparison with 101 radiologists. J Natl Cancer Inst 2019; 111 (09) 916-922
• 74 Huynh BQ, Li H, Giger ML. Digital mammographic tumor classification using transfer learning from deep convolutional neural networks. J Med Imaging (Bellingham) 2016; 3 (03) 034501
• 75 Rodríguez-Ruiz A, Krupinski E, Mordang JJ. et al. Detection of breast cancer with mammography: effect of an artificial intelligence support system. Radiology 2019; 290 (02) 305-314
• 76 Schaffter T, Buist DSM, Lee CI. et al; and the DM DREAM Consortium. Evaluation of combined artificial intelligence and radiologist assessment to interpret screening mammograms. JAMA Netw Open 2020; 3 (03) e200265
• 77 Sasaki M, Tozaki M, Rodríguez-Ruiz A. et al. Artificial intelligence for breast cancer detection in mammography: experience of use of the ScreenPoint Medical Transpara system in 310 Japanese women. Breast Cancer 2020; 27 (04) 642-651
• 78 P S, R T. Aiding the digital mammogram for detecting the breast cancer using shearlet transform and neural network. Asian Pac J Cancer Prev 2018; 19 (09) 2665-2671
• 79 Sepandi M, Taghdir M, Rezaianzadeh A, Rahimikazerooni S. Assessing breast cancer risk with an artificial neural network. Asian Pac J Cancer Prev 2018; 19 (04) 1017-1019
• 80 Rodriguez-Ruiz A, Lång K, Gubern-Merida A. et al. Can we reduce the workload of mammographic screening by automatic identification of normal exams with artificial intelligence? A feasibility study. Eur Radiol 2019; 29 (09) 4825-4832
• 81 Kim HE, Kim HH, Han BK. et al. Changes in cancer detection and false-positive recall in mammography using artificial intelligence: a retrospective, multireader study. Lancet Digit Health 2020; 2 (03) e138-e148
• 82 Mednikov Y, Nehemia S, Zheng B, Benzaquen O, Lederman D. Transfer representation learning using inception-V3 for the detection of masses in mammography. Annu Int Conf IEEE Eng Med Biol Soc 2018; 2018: 2587-2590
• 83 Melekoodappattu JG, Subbian PS. A hybridized ELM for automatic micro calcification detection in mammogram images based on multi-scale features. J Med Syst 2019; 43 (07) 183
• 84 Chen X, Zargari A, Hollingsworth AB, Liu H, Zheng B, Qiu Y. Applying a new quantitative image analysis scheme based on global mammographic features to assist diagnosis of breast cancer. Comput Methods Programs Biomed 2019; 179: 104995
• 85 Arevalo J, Gonzalez FA, Ramos-Pollan R, Oliveira JL, Guevara Lopez MA. Convolutional neural networks for mammography mass lesion classification. Annu Int Conf IEEE Eng Med Biol Soc 2015; 2015: 797-800
• 86 Taghanaki SA, Kawahara J, Miles B, Hamarneh G. Pareto-optimal multi-objective dimensionality reduction deep auto-encoder for mammography classification. Comput Methods Programs Biomed 2017; 145: 85-93
• 87 Sert E, Ertekin S, Halici U. Ensemble of convolutional neural networks for classification of breast microcalcification from mammograms. Annu Int Conf IEEE Eng Med Biol Soc 2017; 2017: 689-692
• 88 Tan M, Qian W, Pu J, Liu H, Zheng B. A new approach to develop computer-aided detection schemes of digital mammograms. Phys Med Biol 2015; 60 (11) 4413-4427
• 89 Saraswathi D, Srinivasan E. A CAD system to analyse mammogram images using fully complex-valued relaxation neural network ensembled classifier. J Med Eng Technol 2014; 38 (07) 359-366
• 90 Suh YJ, Jung J, Cho BJ. Automated breast cancer detection in digital mammograms of various densities via deep learning. J Pers Med 2020; 10 (04) E211
• 91 Jiao Z, Gao X, Wang Y, Li J. A deep feature based framework for breast masses classification. Neurocomputing 2016; 197: 221-231
• 92 Kooi T, van Ginneken B, Karssemeijer N, den Heeten A. Discriminating solitary cysts from soft tissue lesions in mammography using a pretrained deep convolutional neural network. Med Phys 2017; 44 (03) 1017-1027
• 93 Duraisamy S, Emperumal S. Computer-aided mammogram diagnosis system using deep learning convolutional fully complex-valued relaxation neural network classifier. IET Comput Vis 2017; 11 (08) 656-662
• 94 Kurek J, Swiderski B, Osowski S, Kruk M, Barhoumi W. Deep learning versus classical neural approach to mammogram recognition. Bull Pol Acad Sci Tech Sci 2018; 66: 831-840
• 95 Jiao Z, Gao X, Wang Y, Li J. A parasitic metric learning net for breast mass classification based on mammography. Pattern Recognit 2018; 75: 292-301
• 96 Al-antari MA, Al-masni MA, Park SU. et al. An automatic computer-aided diagnosis system for breast cancer in digital mammograms via deep belief network. J Med Biol Eng 2018; 38 (03) 443-456
• 97 Niu J, Li H, Zhang C, Li D. Multi-scale attention-based convolutional neural network for classification of breast masses in mammograms. Med Phys 2021; 48 (07) 3878-3892
• 98 Rajathi GM. Optimized radial basis neural network for classification of breast cancer images. Curr Med Imaging 2021; 17 (01) 97-108
• 99 Khan HN, Shahid AR, Raza B, Dar AH, Alquhayz H. Multi-view feature fusion based four views model for mammogram classification using convolutional neural network. IEEE Access 2019; 7: 165724-165733
• 100 Medjeded M, Saïd M, Chenine A, Chikh M. A new triplet convolutional neural network for classification of lesions on mammograms. Rev Intell Artif 2019; 33: 213-217
• 101 Sun L, Wang J, Hu Z, Xu Y, Cui Z. Multi-view convolutional neural networks for mammographic image classification. IEEE Access 2019; 7: 126273-126282
• 102 Yu S, Liu L, Wang Z, Dai G, Xie Y. Transferring deep neural networks for the differentiation of mammographic breast lesions. Sci China Technol Sci 2019; 62 (03) 441-447
• 103 Tavakoli N, Karimi M, Norouzi A, Karimi N, Samavi S. Soroushmehr SMR. Detection of abnormalities in mammograms using deep features. J Ambient Intell Humaniz Comput 2019; 14 (05) 5355-5367
• 104 Jo YY, Choi YS, Park HW. et al. Impact of image compression on deep learning-based mammogram classification. Sci Rep 2021; 11 (01) 7924
• 105 Cui Y, Li Y, Xing D, Bai T, Dong J, Zhu J. Improving the prediction of benign or malignant breast masses using a combination of image biomarkers and clinical parameters. Front Oncol 2021; 11: 629321
• 106 Yang J, Li H, Shi N, Zhang Q, Liu Y. Microscopic tumour classification by digital mammography. J Healthc Eng 2021; 2021: 6635947
• 107 Chouhan N, Khan A, Shah JZ, Hussnain M, Khan MW. Deep convolutional neural network and emotional learning based breast cancer detection using digital mammography. Comput Biol Med 2021; 132: 104318
• 108 Ragab DA, Attallah O, Sharkas M, Ren J, Marshall S. A framework for breast cancer classification using Multi-DCNNs. Comput Biol Med 2021; 131: 104245
• 109 Tsochatzidis L, Koutla P, Costaridou L, Pratikakis I. Integrating segmentation information into CNN for breast cancer diagnosis of mammographic masses. Comput Methods Programs Biomed 2021; 200: 105913
• 110 Zhang X, Liang C, Zeng D. et al. Pattern classification for breast lesion on FFDM by integration of radiomics and deep features. Comput Med Imaging Graph 2021; 90: 101922
• 111 Rehman KU, Li J, Pei Y, Yasin A, Ali S, Mahmood T. Computer vision-based microcalcification detection in digital mammograms using fully connected depthwise separable convolutional neural network. Sensors (Basel) 2021; 21 (14) 4854
• 112 Salim M, Wåhlin E, Dembrower K. et al. External evaluation of 3 commercial artificial intelligence algorithms for independent assessment of screening mammograms. JAMA Oncol 2020; 6 (10) 1581-1588
• 113 McKinney SM, Sieniek M, Godbole V. et al. International evaluation of an AI system for breast cancer screening. Nature 2020; 577 (7788) 89-94
• 114 Akselrod-Ballin A, Chorev M, Shoshan Y. et al. Predicting breast cancer by applying deep learning to linked health records and mammograms. Radiology 2019; 292 (02) 331-342
• 115 Pacilè S, Lopez J, Chone P, Bertinotti T, Grouin JM, Fillard P. Improving breast cancer detection accuracy of mammography with the concurrent use of an artificial intelligence tool. Radiol Artif Intell 2020; 2 (06) e190208
• 116 Watanabe AT, Lim V, Vu HX. et al. Improved cancer detection using artificial intelligence: a retrospective evaluation of missed cancers on mammography. J Digit Imaging 2019; 32 (04) 625-637
• 117 Lotter W, Diab AR, Haslam B. et al. Robust breast cancer detection in mammography and digital breast tomosynthesis using an annotation-efficient deep learning approach. Nat Med 2021; 27 (02) 244-249
• 118 Tsochatzidis L, Costaridou L, Pratikakis I. Deep learning for breast cancer diagnosis from mammograms-a comparative study. J Imaging 2019; 5 (03) 37
• 119 Chakraborty DP, Breatnach ES, Yester MV, Soto B, Barnes GT, Fraser RG. Digital and conventional chest imaging: a modified ROC study of observer performance using simulated nodules. Radiology 1986; 158 (01) 35-39
• 120 Uzun Ozsahin D, Ikechukwu Emegano D, Uzun B, Ozsahin I. The systematic review of artificial intelligence applications in breast cancer diagnosis. Diagnostics (Basel) 2022; 13 (01) 45