Artificial Intelligence-powered automatic volume calculation in medical images – available tools, performance and challenges for nuclear medicine

Thomas Wendler; Michael C. Kreissl; Benedikt Schemmer; Julian Manuel Michael Rogasch; Francesca De Benetti

doi:10.1055/a-2200-2145

Nuklearmedizin - NuclearMedicine, Table of Contents

CC BY-NC-ND 4.0 · Nuklearmedizin 2023; 62(06): 343-353
DOI: 10.1055/a-2200-2145

Review

Artificial Intelligence-powered automatic volume calculation in medical images – available tools, performance and challenges for nuclear medicine

Automatische Volumenberechnung mithilfe künstlicher Intelligenz in der medizinischen Bildgebung – verfügbare Werkzeuge, Performance und Herausforderungen für die Nuklearmedizin

Thomas Wendler

¹Clinical Computational Medical Imaging Research, Department of Diagnostic and Interventional Radiology and Neuroradiology, University Hospital Augsburg, Germany

²Institute of Digital Medicine, Universitätsklinikum Augsburg, Germany

³Computer-Aided Medical Procedures and Augmented Reality School of Computation, Information and Technology, Technical University of Munich, Munich, Germany

,

Michael C. Kreissl

⁴Abteilung für Nuklearmedizin, Universitätsklinikum Magdeburg, Germany

,

Benedikt Schemmer

⁵Department of Nuclear Medicine, Universitätsklinikum Bonn, Germany

,

Julian Manuel Michael Rogasch

⁶Department of Nuclear Medicine, Charité – Universitätsmedizin Berlin, corporate member of Freie Universität Berlin and Humboldt-Universität zu Berlin,Germany

,

Francesca De Benetti

³Computer-Aided Medical Procedures and Augmented Reality School of Computation, Information and Technology, Technical University of Munich, Munich, Germany

› Author Affiliations

Abstract

Volumetry is crucial in oncology and endocrinology, for diagnosis, treatment planning, and evaluating response to therapy for several diseases. The integration of Artificial Intelligence (AI) and Deep Learning (DL) has significantly accelerated the automatization of volumetric calculations, enhancing accuracy and reducing variability and labor. In this review, we show that a high correlation has been observed between Machine Learning (ML) methods and expert assessments in tumor volumetry; Yet, it is recognized as more challenging than organ volumetry. Liver volumetry has shown progression in accuracy with a decrease in error. If a relative error below 10 % is acceptable, ML-based liver volumetry can be considered reliable for standardized imaging protocols if used in patients without major anomalies. Similarly, ML-supported automatic kidney volumetry has also shown consistency and reliability in volumetric calculations. In contrast, AI-supported thyroid volumetry has not been extensively developed, despite initial works in 3D ultrasound showing promising results in terms of accuracy and reproducibility. Despite the advancements presented in the reviewed literature, the lack of standardization limits the generalizability of ML methods across diverse scenarios. The domain gap, i. e., the difference in probability distribution of training and inference data, is of paramount importance before clinical deployment of AI, to maintain accuracy and reliability in patient care. The increasing availability of improved segmentation tools is expected to further incorporate AI methods into routine workflows where volumetry will play a more prominent role in radionuclide therapy planning and quantitative follow-up of disease evolution.

Key words

artificial intelligence - machine learning - deep learning - volumetry - volume estimation - volume calculation

Full Text

References

References
1 Riveira-Martin M, Akhavanallaf A, Mansouri Z. et al. Predictive value of 99mTc-MAA-based dosimetry in personalized 90Y-SIRT planning for liver malignancies. EJNMMI Res 2023; 13 (01) 63
2 Krönke M, Eilers C, Dimova D. et al. Tracked 3D ultrasound and deep neural network-based thyroid segmentation reduce interobserver variability in thyroid volumetry. PLOS ONE 2022; 17 (07) e0268550
3 Sunoqrot MRS, Selnæs KM, Sandsmark E. et al. The Reproducibility of Deep Learning-Based Segmentation of the Prostate Gland and Zones on T2-Weighted MR Images. Diagn Basel Switz 2021; 11 (09) 1690
4 Ding J, Cao P, Chang HC. et al. Deep learning-based thigh muscle segmentation for reproducible fat fraction quantification using fat–water decomposition MRI. Insights Imaging 2020; 11 (01) 128
5 Seifert P, Ullrich SL, Kühnel C. et al. Optimization of Thyroid Volume Determination by Stitched 3D-Ultrasound Data Sets in Patients with Structural Thyroid Disease. Biomedicines 2023; 11 (02) 381
6 Brunn J, Block U, Ruf G. et al. [Volumetric analysis of thyroid lobes by real-time ultrasound (author’s transl)]. Dtsch Med Wochenschr 1946 1981; 106 (41) 1338-1340
7 Campennì A, Avram AM, Verburg FA. et al. The EANM guideline on radioiodine therapy of benign thyroid disease. Eur J Nucl Med Mol Imaging [Internet] 2023;
8 Muggli D, Müller MA, Karlo C. et al. A simple method to approximate liver size on cross-sectional images using living liver models. Clin Radiol 2009; 64 (07) 682-689
9 Hussain MA, Hamarneh G, Garbi R. Cascaded Regression Neural Nets for Kidney Localization and Segmentation-free Volume Estimation. IEEE Trans Med Imaging 2021; 40 (06) 1555-1567
10 Weber M, Telli T, Kersting D. et al. Prognostic Implications of PET-Derived Tumor Volume and Uptake in Patients with Neuroendocrine Tumors. Cancers 2023; 15 (14) 3581
11 Mikhaeel NG, Heymans MW, Eertink JJ. et al. Proposed New Dynamic Prognostic Index for Diffuse Large B-Cell Lymphoma: International Metabolic Prognostic Index. J Clin Oncol Off J Am Soc Clin Oncol 2022; 40 (21) 2352-2360
12 Gafita A, Calais J, Grogan TR. et al. Nomograms to predict outcomes after 177Lu-PSMA therapy in men with metastatic castration-resistant prostate cancer: an international, multicentre, retrospective study. Lancet Oncol 2021; 22 (08) 1115-1125
13 Seifert R, Herrmann K, Kleesiek J. et al. Semiautomatically Quantified Tumor Volume Using 68Ga-PSMA-11 PET as a Biomarker for Survival in Patients with Advanced Prostate Cancer. J Nucl Med Off Publ Soc Nucl Med 2020; 61 (12) 1786-1792
14 Hutchings M, Barrington SF. PET/CT for therapy response assessment in lymphoma. J Nucl Med Off Publ Soc Nucl Med 2009; 50 (Suppl. 01) 21S-30S
15 Isensee F, Jaeger PF, Kohl SAA. et al. nnU-Net: a self-configuring method for deep learning-based biomedical image segmentation. Nat Methods 2021; 18 (02) 203-211
16 Gutsche R, Lowis C, Ziemons K. et al. Automated Brain Tumor Detection and Segmentation for Treatment Response Assessment Using Amino Acid PET. J Nucl Med Off Publ Soc Nucl Med 2023;
17 Vivanti R, Szeskin A, Lev-Cohain N. et al. Automatic detection of new tumors and tumor burden evaluation in longitudinal liver CT scan studies. Int J Comput Assist Radiol Surg 2017; 12 (11) 1945-1957
18 Ronneberger O, Fischer P, Brox T. U-Net: Convolutional Networks for Biomedical Image Segmentation. In: Navab N, Hornegger J, Wells WM. et al. Medical Image Computing and Computer-Assisted Intervention – MICCAI. Cham: Springer International Publishing; 2015: 234-241
19 Shapey J, Wang G, Dorent R. et al. An artificial intelligence framework for automatic segmentation and volumetry of vestibular schwannomas from contrast-enhanced T1-weighted and high-resolution T2-weighted MRI. J Neurosurg 2019; 134 (01) 171-179
20 Kang H, Witanto JN, Pratama K. et al. Fully Automated MRI Segmentation and Volumetric Measurement of Intracranial Meningioma Using Deep Learning. J Magn Reson Imaging JMRI 2023; 57 (03) 871-881
21 Oktay O, Schlemper J, Folgoc LL. Attention U-Net: Learning Where to Look for the Pancreas. In: Conference Book [Internet]. Amsterdam: Radboud University Medical Center; 2018 [cited 2023 Sep 21]. 15 Available from: https://openreview.net/pdf?id=Skft7cijM
22 Toyonaga T, Shao D, Shi L. et al. Deep learning-based attenuation correction for whole-body PET – a multi-tracer study with 18F-FDG, 68 Ga-DOTATATE, and 18F-Fluciclovine. Eur J Nucl Med Mol Imaging 2022; 49 (09) 3086-3097
23 Hirata K, Furuya S, Huang SC. et al. A semi-automated method to separate tumor from physiological uptakes on FDG PET-CT for efficient generation of training data targeting deep learning. J Nucl Med 2019; 60 (Suppl. 01) 1213-1213
24 Van Ginneken B, Heimann T, Styner M. 3D segmentation in the clinic: A grand challenge. In: MICCAI workshop on 3D segmentation in the clinic: a grand challenge. 2007: 7-15
25 Soler L, Hostettler A, Agnus V. et al 3D image reconstruction for comparison of algorithm database. 2010 Available from: https://www.ircad.fr/research/data-sets/liver-segmentation-3d-ircadb-01
26 CodaLab – Competition [Internet]. [cited 2023 Sep 25]. Available from: https://competitions.codalab.org/competitions/17094#results
27 Takamoto T, Ban D, Nara S. et al. Automated Three-Dimensional Liver Reconstruction with Artificial Intelligence for Virtual Hepatectomy. J Gastrointest Surg Off J Soc Surg Aliment Tract 2022; 26 (10) 2119-2127
28 Lu F, Wu F, Hu P. et al. Automatic 3D liver location and segmentation via convolutional neural network and graph cut. Int J Comput Assist Radiol Surg 2017; 12 (02) 171-182
29 Marinelli B, Kang M, Martini M. et al. Combination of Active Transfer Learning and Natural Language Processing to Improve Liver Volumetry Using Surrogate Metrics with Deep Learning. Radiol Artif Intell 2019; 1 (01) e180019
30 Ibtehaz N, Rahman MS. MultiResUNet : Rethinking the U-Net architecture for multimodal biomedical image segmentation. Neural Netw Off J Int Neural Netw Soc 2020; 121: 74-87
31 Koitka S, Gudlin P, Theysohn JM. et al. Fully automated preoperative liver volumetry incorporating the anatomical location of the central hepatic vein. Sci Rep 2022; 12 (01) 16479
32 Shin TY, Kim H, Lee JH. et al. Expert-level segmentation using deep learning for volumetry of polycystic kidney and liver. Investig Clin Urol 2020; 61 (06) 555-564
33 Milletari F, Navab N, Ahmadi SA. V-net: Fully convolutional neural networks for volumetric medical image segmentation. In IEEE;. 2016: 565-571
34 Cayot B, Milot L, Nempont O. et al. Polycystic liver: automatic segmentation using deep learning on CT is faster and as accurate compared to manual segmentation. Eur Radiol 2022; 32 (07) 4780-4790
35 Ng YS, Xi Y, Qian Y. et al. Use of Spectral Detector Computed Tomography to Improve Liver Segmentation and Volumetry. J Comput Assist Tomogr 2020; 44 (02) 197-203
36 Chlebus G, Meine H, Thoduka S. et al. Reducing inter-observer variability and interaction time of MR liver volumetry by combining automatic CNN-based liver segmentation and manual corrections. PloS One 2019; 14 (05) e0217228
37 Saunders SL, Clark JM, Rudser K. et al. Comparison of automatic liver volumetry performance using different types of magnetic resonance images. Magn Reson Imaging 2022; 91: 16-23
38 Wang K, Mamidipalli A, Retson T. et al. Automated CT and MRI Liver Segmentation and Biometry Using a Generalized Convolutional Neural Network. Radiol Artif Intell 2019; 1 (02) 180022
39 Bilic P, Christ P, Li HB. et al. The Liver Tumor Segmentation Benchmark (LiTS). Med Image Anal 2023; 84: 102680
40 Sundlöv A, Sjögreen-Gleisner K, Svensson J. et al. Individualised 177Lu-DOTATATE treatment of neuroendocrine tumours based on kidney dosimetry. Eur J Nucl Med Mol Imaging 2017; 44 (09) 1480-1489
41 Hsiao CH, Sun TL, Lin PC. et al. A deep learning-based precision volume calculation approach for kidney and tumor segmentation on computed tomography images. Comput Methods Programs Biomed 2022; 221: 106861
42 Heller N, Sathianathen N, Kalapara A. et al The KiTS19 Challenge Data: 300 Kidney Tumor Cases with Clinical Context, CT Semantic Segmentations, and Surgical Outcomes [Internet]. arXiv; 2020 [cited 2023 Sep 21]. Available from: http://arxiv.org/abs/1904.00445
43 Licht K, Darr A, Opfermann T. et al. 3D ultrasonography is as accurate as low-dose CT in thyroid volumetry. Nukl Nucl Med 2014; 53 (03) 99-104
44 Reinartz P, Sabri O, Zimny M. et al. Thyroid volume measurement in patients prior to radioiodine therapy: comparison between three-dimensional magnetic resonance imaging and ultrasonography. Thyroid Off J Am Thyroid Assoc 2002; 12 (08) 713-717
45 Chang CY, Lei YF, Tseng CH. et al. Thyroid segmentation and volume estimation in ultrasound images. IEEE Trans Biomed Eng 2010; 57 (06) 1348-1357
46 Ham FM, Kostanic I. Principles of Neurocomputing for Science and Engineering. McGraw Hill 2000; 680
47 Kennedy J, Eberhart R. Particle swarm optimization. In: Proceedings of ICNN’95 – International Conference on Neural Networks [Internet]. Perth, WA, Australia: IEEE; 1995 [cited 2023 Sep 25]. 1942-1948 Available from: http://ieeexplore.ieee.org/document/488968/
48 Eberhart R, Kennedy J. A new optimizer using particle swarm theory. MHS95 Proc Sixth Int Symp Micro Mach Hum Sci [Internet]. 1995: 39-43 Available from: http://ieeexplore.ieee.org/document/494215/
49 Kumar V, Webb J, Gregory A. et al. Automated Segmentation of Thyroid Nodule, Gland, and Cystic Components From Ultrasound Images Using Deep Learning. IEEE Access Pract Innov Open Solut 2020; 8: 63482-63496
50 Yu F, Koltun V. Multi-Scale Context Aggregation by Dilated Convolutions [Internet]. arXiv; 2016 [cited 2023 Sep 25]. Available from: http://arxiv.org/abs/1511.07122
51 Chan TF, Vese LA. Active contours without edges. IEEE Trans Image Process [Internet]. 2001; 10 (02) 266-277 http://ieeexplore.ieee.org/document/902291/
52 Rother C, Kolmogorov V, Blake A. ‘GrabCut’: interactive foreground extraction using iterated graph cuts. ACM Trans Graph [Internet]. 2004; 23 (03) 309-314
53 Breiman L. Random Forests. Mach Learn [Internet]. 2001; 45 (01) 5-32
54 Criminisi A, Shotton J. Decision Forests for Computer Vision and Medical Image Analysis. Springer Science & Business Media 2013; 367
55 Krönke M, Eilers C, Dimova D. et al. Tracked 3D ultrasound and deep neural network-based thyroid segmentation reduce interobserver variability in thyroid volumetry. PloS One 2022; 17 (07) e0268550
56 Dietlein M, Grünwald F, Schmidt M. et al. Radioiodtherapie bei benignen Schilddrüsenerkrankungen (Version 5)*. Nukl – Nucl [Internet]. 2016; 55 (06) 213-220
57 Guha RoyA, Conjeti S, Navab N. et al. QuickNAT: A fully convolutional network for quick and accurate segmentation of neuroanatomy. NeuroImage [Internet] 2019; 186: 713-727 https://www.sciencedirect.com/science/article/pii/S1053811918321232
58 Zielke J, Eilers C, Busam B. et al. RSV: Robotic Sonography for Thyroid Volumetry. IEEE Robot Autom Lett 2022; 7 (02) 3342-3348
59 Yu SH, Choi SJ, Noh H. et al. Comparison of CT Volumetry and RECIST to Predict the Treatment Response and Overall Survival in Gastric Cancer Liver Metastases. Taehan Yongsang Uihakhoe Chi 2021; 82 (04) 876-888
60 Hofmann FO, Heinemann V, D’Anastasi M. et al. Standard diametric versus volumetric early tumor shrinkage as a predictor of survival in metastatic colorectal cancer: subgroup findings of the randomized, open-label phase III trial FIRE-3/AIO KRK-0306. Eur Radiol 2023; 33 (02) 1174-1184
61 Siegel MJ, Ippolito JE, Wahl RL. et al. Discrepant Assessments of Progressive Disease in Clinical Trials between Routine Clinical Reads and Formal RECIST 1.1 Interpretations. Radiol Imaging Cancer 2023; 5 (05) e230001
62 Garin E, Tselikas L, Guiu B. et al. Personalised versus standard dosimetry approach of selective internal radiation therapy in patients with locally advanced hepatocellular carcinoma (DOSISPHERE-01): a randomised, multicentre, open-label phase 2 trial. Lancet Gastroenterol Hepatol 2021; 6 (01) 17-29
63 Pacilio M, Conte M, Frantellizzi V. et al. Personalized Dosimetry in the Context of Radioiodine Therapy for Differentiated Thyroid Cancer. Diagn Basel Switz 2022; 12 (07) 1763
64 Mariani G, Tonacchera M, Grosso M. et al. The Role of Nuclear Medicine in the Clinical Management of Benign Thyroid Disorders, Part 1: Hyperthyroidism. J Nucl Med Off Publ Soc Nucl Med 2021; 62 (03) 304-312
65 van Heek L, Weindler J, Gorniak C. et al. Prognostic value of baseline metabolic tumor volume (MTV) for forecasting chemotherapy outcome in early-stage unfavorable Hodgkin lymphoma: Data from the phase III HD17 trial. Eur J Haematol 2023; 111 (06) 881-887