Quantification in Musculoskeletal Imaging Using Computational Analysis and Machine Learning: Segmentation and Radiomics

Meritxell Bach Cuadra; Julien Favre; Patrick Omoumi

doi:10.1055/s-0039-3400268

Seminars in Musculoskeletal Radiology, Table of Contents

Semin Musculoskelet Radiol 2020; 24(01): 50-64
DOI: 10.1055/s-0039-3400268

Review Article

Quantification in Musculoskeletal Imaging Using Computational Analysis and Machine Learning: Segmentation and Radiomics

Meritxell Bach Cuadra

¹Department of Radiology, Lausanne University Hospital and University of Lausanne (UNIL), Lausanne, Switzerland

²Centre d'Imagerie BioMédicale (CIBM), Lausanne University Hospital (CHUV) and University of Lausanne (UNIL), Lausanne, Switzerland

³Signal Processing Laboratory (LTS5), École Polytechnique Fédérale de Lausanne (EPFL), Lausanne, Switzerland

,

Julien Favre

⁴Swiss BioMotion Lab, Department of Musculoskeletal Medicine, Lausanne University Hospital and University of Lausanne, Lausanne, Switzerland

,

Patrick Omoumi

¹Department of Radiology, Lausanne University Hospital and University of Lausanne (UNIL), Lausanne, Switzerland

⁴Swiss BioMotion Lab, Department of Musculoskeletal Medicine, Lausanne University Hospital and University of Lausanne, Lausanne, Switzerland

› Author Affiliations

Abstract

Although still limited in clinical practice, quantitative analysis is expected to increase the value of musculoskeletal (MSK) imaging. Segmentation aims at isolating the tissues and/or regions of interest in the image and is crucial to the extraction of quantitative features such as size, signal intensity, or image texture. These features may serve to support the diagnosis and monitoring of disease. Radiomics refers to the process of extracting large amounts of features from radiologic images and combining them with clinical, biological, genetic, or any other type of complementary data to build diagnostic, prognostic, or predictive models. The advent of machine learning offers promising prospects for automatic segmentation and integration of large amounts of data. We present commonly used segmentation methods and describe the radiomics pipeline, highlighting the challenges to overcome for adoption in clinical practice. We provide some examples of applications from the MSK literature.

Keywords

artificial intelligence - radiomics - musculoskeletal imaging - radiology

Full Text

References

References
1 Buckler AJ, Bresolin L, Dunnick NR, Sullivan DC. Group. A collaborative enterprise for multi-stakeholder participation in the advancement of quantitative imaging. Radiology 2011; 258 (03) 906-914
2 Mansoor A, Bagci U, Foster B. et al. Segmentation and image analysis of abnormal lungs at CT: current approaches, challenges, and future trends. Radiographics 2015; 35 (04) 1056-1076
3 Traverso A, Wee L, Dekker A, Gillies R. Repeatability and reproducibility of radiomic features: a systematic review. Int J Radiat Oncol Biol Phys 2018; 102 (04) 1143-1158
4 Gillies RJ, Anderson AR, Gatenby RA, Morse DL. The biology underlying molecular imaging in oncology: from genome to anatome and back again. Clin Radiol 2010; 65 (07) 517-521
5 Limkin EJ, Sun R, Dercle L. et al. Promises and challenges for the implementation of computational medical imaging (radiomics) in oncology. Ann Oncol 2017; 28 (06) 1191-1206
6 Gillies RJ, Kinahan PE, Hricak H. Radiomics: images are more than pictures, they are data. Radiology 2016; 278 (02) 563-577
7 Lee G, Lee HY, Park H. et al. Radiomics and its emerging role in lung cancer research, imaging biomarkers and clinical management: state of the art. Eur J Radiol 2017; 86: 297-307
8 Lambin P, Leijenaar RTH, Deist TM. et al. Radiomics: the bridge between medical imaging and personalized medicine. Nat Rev Clin Oncol 2017; 14 (12) 749-762
9 Bodalal Z, Trebeschi S, Beets-Tan R. Radiomics: a critical step towards integrated healthcare. Insights Imaging 2018; 9 (06) 911-914
10 Bodalal Z, Trebeschi S, Nguyen-Kim TDL, Schats W, Beets-Tan R. Radiogenomics: bridging imaging and genomics. Abdom Radiol (NY) 2019; 44 (06) 1960-1984
11 Afshar P, Mohammadi A, Plataniotis KN, Oikonomou A, Benali H. From hand-crafted to deep learning-based cancer radiomics: challenges and opportunities. IEEE Signal Process Mag 2019; 36 (04) 132-160
12 Aerts HJWL, Velazquez ER, Leijenaar RTH. et al. Decoding tumour phenotype by noninvasive imaging using a quantitative radiomics approach. Nat Commun 2014; 5 (01) 4006
13 Aerts HJWL. The potential of radiomic-based phenotyping in precision medicine: a review. JAMA Oncol 2016; 2 (12) 1636-1642
14 Ma Z, Tavares JMRS, Jorge RN, Mascarenhas T. A review of algorithms for medical image segmentation and their applications to the female pelvic cavity. Comput Methods Biomech Biomed Engin 2010; 13 (02) 235-246
15 Pedoia V, Majumdar S, Link TM. Segmentation of joint and musculoskeletal tissue in the study of arthritis. MAGMA 2016; 29 (02) 207-221
16 Aprovitola A, Gallo L. Knee bone segmentation from MRI: a classification and literature review. Biocybern Biomed Eng 2016; 36 (02) 437-449
17 Zhang B, Zhang Y, Cheng HD. et al. Computer-aided knee joint magnetic resonance image segmentation—a survey. 2018 . Abstract Accessed August 16, 2019 at: http://arxiv.org/abs/1802.04894
18 Ryzhkov MD. Knee cartilage segmentation algorithms: a critical literature review. Accessed August 16, 2019 at: http://dspace.library.uu.nl/handle/1874/308831 2015
19 Litjens G, Kooi T, Bejnordi BE. et al. A survey on deep learning in medical image analysis. Med Image Anal 2017; 42: 60-88
20 John NW. Segmentation of radiological images. In: Neri E, Caramella D, Bartolozzi C. eds. Image Processing in Radiology. Berlin, Germany: Springer; 2008: 45-54
21 Dalvi R, Abugharbieh R, Wilson D, Wilson DR. Multi-contrast MR for enhanced bone imaging and segmentation. Conf Proc IEEE Eng Med Biol Soc 2007; 2007: 5620-5623
22 Li X, Ma BC, Bolbos RI. et al. Quantitative assessment of bone marrow edema-like lesion and overlying cartilage in knees with osteoarthritis and anterior cruciate ligament tear using MR imaging and spectroscopic imaging at 3 Tesla. J Magn Reson Imaging 2008; 28 (02) 453-461
23 Li W, Abram F, Pelletier JP. et al. Fully automated system for the quantification of human osteoarthritic knee joint effusion volume using magnetic resonance imaging. Arthritis Res Ther 2010; 12 (05) R173
24 Kim S, Lee D, Park S, Oh KS, Chung SW, Kim Y. Automatic segmentation of supraspinatus from MRI by internal shape fitting and autocorrection. Comput Methods Programs Biomed 2017; 140: 165-174
25 Hacihaliloglu I. Ultrasound imaging and segmentation of bone surfaces: a review. Technology (Singap World Sci) 2017; 5 (02) 74-80
26 Zhang J, Yan CH, Chui CK, Ong SH. Fast segmentation of bone in CT images using 3D adaptive thresholding. Comput Biol Med 2010; 40 (02) 231-236
27 Jena S, Sahu B, Jagadev AK. Analysis of medical X-ray bone images using image segmentation. In: Jain LC, Patnaik S, Ichalkaranje N. eds. Intelligent Computing, Communication and Devices. Advances in Intelligent Systems and Computing. New Delhi, India: Springer India; 2015: 787-794
28 van Eijnatten M, van Dijk R, Dobbe J, Streekstra G, Koivisto J, Wolff J. CT image segmentation methods for bone used in medical additive manufacturing. Med Eng Phys 2018; 51: 6-16
29 Silva G, Oliveira L, Pithon M. Automatic segmenting teeth in X-ray images: Trends, a novel data set, benchmarking and future perspectives. Expert Syst Appl 2018; 107: 15-31
30 Szeliski R. Computer Vision: Algorithms and Applications. London, UK: Springer; 2011. . Accessed August 16, 2019 at: https://www.springer.com/gp/book/9781848829343
31 Serra J. Introduction to mathematical morphology. Comput Vis Graph Image Process 1986; 35 (03) 283-305
32 Lee JS, Chung YN. Integrating edge detection and thresholding approaches to segmenting femora and patellae from magnetic resonance images. Biomed Eng Appl Basis Commun 2005; 17 (01) 1-11
33 Carballido-Gamio J, Bauer JS, Stahl R. et al. Inter-subject comparison of MRI knee cartilage thickness. Med Image Anal 2008; 12 (02) 120-135
34 Duryea J, Neumann G, Brem MH. et al. Novel fast semi-automated software to segment cartilage for knee MR acquisitions. Osteoarthritis Cartilage 2007; 15 (05) 487-492
35 Iranpour-Boroujeni T, Watanabe A, Bashtar R, Yoshioka H, Duryea J. Quantification of cartilage loss in local regions of knee joints using semi-automated segmentation software: analysis of longitudinal data from the Osteoarthritis Initiative (OAI). Osteoarthritis Cartilage 2011; 19 (03) 309-314
36 Brem MH, Lang PK, Neumann G. et al. Magnetic resonance image segmentation using semi-automated software for quantification of knee articular cartilage---initial evaluation of a technique for paired scans. Skeletal Radiol 2009; 38 (05) 505-511
37 Adams R, Bischof L. Seeded region growing. IEEE Trans Pattern Anal Mach Intell 1994; 16 (06) 641-647
38 Chang YL, Li X. Adaptive image region-growing. IEEE Trans Image Process 1994; 3 (06) 868-872
39 Waterton JC, Solloway S, Foster JE. et al. Diurnal variation in the femoral articular cartilage of the knee in young adult humans. Magn Reson Med 2000; 43 (01) 126-132
40 Digabel H, Lantuejoul Ch. Iterative Algorithms, Special Issues of Practical Metallography. Vol. 8. Stuttgart, Germany: Riederer; 1977
41 Meyer F. Topographic distance and watershed lines. Signal Processing 1994; 38 (01) 113-125
42 Kornilov A, Safonov I. An overview of watershed algorithm implementations in open source libraries. J Imaging 2018; 4: 123
43 Grau V, Mewes AUJ, Alcañiz M, Kikinis R, Warfield SK. Improved watershed transform for medical image segmentation using prior information. IEEE Trans Med Imaging 2004; 23 (04) 447-458
44 Włodarczyk J, Czaplicka K, Tabor Z, Wojciechowski W, Urbanik A. Segmentation of bones in magnetic resonance images of the wrist. Int J CARS 2015; 10 (04) 419-431
45 Desai P, Hacihaliloglu I. Knee-cartilage segmentation and thickness measurement from 2D ultrasound. J Imaging 2019; 5: 43
46 Caselles V, Kimmel R, Sapiro G. Geodesic active contours. Int J Comput Vis 1997; 22 (01) 61-79
47 Malladi R, Sethian JA, Vemuri BC. Shape modeling with front propagation: a level set approach. IEEE Trans Pattern Anal Mach Intell 1995; 17: 158-175
48 Osher S, Sethian JA. Fronts propagating with curvature-dependent speed: Algorithms based on Hamilton-Jacobi formulations. J Comput Phys 1988; 79 (01) 12-49
49 Sethian JA. Level Set Methods and Fast Marching Methods: Evolving Interfaces in Computational Geometry, Fluid Mechanics, Computer Vision, and Materials Science. 2nd ed. Cambridge, UK: Cambridge University Press; 1999
50 Chan TF, Vese LA. Active contours without edges. IEEE Trans Image Process 2001; 10 (02) 266-277
51 Li W, Abram F, Beaudoin G, Berthiaume MJ, Pelletier JP, Martel-Pelletier J. Human hip joint cartilage: MRI quantitative thickness and volume measurements discriminating acetabulum and femoral head. IEEE Trans Biomed Eng 2008; 55 (12) 2731-2740
52 Cootes TF, Taylor CJ, Cooper DH, Graham J. Active shape models-their training and application. Comput Vis Image Underst 1995; 61: 38-59
53 Cootes TF, Edwards GJ, Taylor CJ. Comparing active shape models with active appearance models. In: Pridmore T. , Elliman, eds. Proceedings of the British Machine Vision Conference. Durham, UK: BMVA Press; 1999: 18.1-18.10
54 Davies RH, Twining CJ, Cootes TF, Waterton JC, Taylor CJ. A minimum description length approach to statistical shape modeling. IEEE Trans Med Imaging 2002; 21 (05) 525-537
55 Heimann T, Meinzer H-P. Statistical shape models for 3D medical image segmentation: a review. Med Image Anal 2009; 13 (04) 543-563
56 Solloway S, Hutchinson CE, Waterton JC, Taylor CJ. The use of active shape models for making thickness measurements of articular cartilage from MR images. Magn Reson Med 1997; 37 (06) 943-952
57 Fripp J, Crozier S, Warfield SK, Ourselin S. Automatic segmentation and quantitative analysis of the articular cartilages from magnetic resonance images of the knee. IEEE Trans Med Imaging 2010; 29 (01) 55-64
58 Fripp J, Crozier S, Warfield SK, Ourselin S. Automatic segmentation of the bone and extraction of the bone-cartilage interface from magnetic resonance images of the knee. Phys Med Biol 2007; 52 (06) 1617-1631
59 Schmid J, Kim J, Magnenat-Thalmann N. Robust statistical shape models for MRI bone segmentation in presence of small field of view. Med Image Anal 2011; 15 (01) 155-168
60 Bach Cuadra M, Duay V, Thiran J-Ph. Atlas-based segmentation. In: Paragios N, Ayache N, Duncan J. eds. Handbook of Biomedical Imaging: Methodologies and Clinical Research. New York, NY: Springer; 2015: 221-244
61 Iglesias JE, Sabuncu MR. Multi-atlas segmentation of biomedical images: A survey. Med Image Anal 2015; 24 (01) 205-219
62 Kapur T, Beardsley P, Frisken S, Eric W, Grimson L, Wells W. Model-based segmentation of clinical knee MRI. Paper presented at: IEEE International Workshop on Model-Based 3D Image Analysis; 1998 Bombay, India:
63 Lee JG, Gumus S, Moon CH, Kwoh CK, Bae KT. Fully automated segmentation of cartilage from the MR images of knee using a multi-atlas and local structural analysis method. Med Phys 2014; 41 (09) 092303
64 Chen X, Pan L. A Survey of graph cuts/graph search based medical image segmentation. IEEE Rev Biomed Eng 2018; 11: 112-124
65 Boykov Y, Veksler O, Zabih R. Fast approximate energy minimization via graph cuts. Accessed October 18, 2019 at: http://www.cs.cornell.edu/rdz/Papers/BVZ-iccv99.pdf
66 Shim H, Chang S, Tao C, Wang JH, Kwoh CK, Bae KT. Knee cartilage: efficient and reproducible segmentation on high-spatial-resolution MR images with the semiautomated graph-cut algorithm method. Radiology 2009; 251 (02) 548-556
67 Bishop CM. Pattern Recognition and Machine Learning. New York, NY: Springer; 2006
68 Zhang DQ, Chen SC. A novel kernelized fuzzy C-means algorithm with application in medical image segmentation. Artif Intell Med 2004; 32 (01) 37-50
69 Bezdek JC. Pattern recognition with fuzzy objective function algorithms. Springer US; 1981. . Accessed August 26, 2019 at: https://www.springer.com/gp/book/9781475704525
70 Carballido-Gamio J, Phan C, Link TM, Majumdar S. Characterization of trabecular bone structure from high-resolution magnetic resonance images using fuzzy logic. Magn Reson Imaging 2006; 24 (08) 1023-1029
71 Folkesson J, Carballido-Gamio J, Eckstein F, Link TM, Majumdar S. Local bone enhancement fuzzy clustering for segmentation of MR trabecular bone images. Med Phys 2010; 37 (01) 295-302
72 Folkesson J, Dam EB, Olsen OF, Pettersen PC, Christiansen C. Segmenting articular cartilage automatically using a voxel classification approach. IEEE Trans Med Imaging 2007; 26 (01) 106-115
73 Zhang K, Lu W, Marziliano P. Automatic knee cartilage segmentation from multi-contrast MR images using support vector machine classification with spatial dependencies. Magn Reson Imaging 2013; 31 (10) 1731-1743
74 Goodfellow I, Bengio Y, Courville A. Deep Learning. Cambridge, MA: MIT Press; 2016
75 Hirschmann A, Cyriac J, Stieltjes B, Kober T, Richiardi J, Omoumi P. Artificial intelligence in musculoskeletal imaging: review of current literature, challenges, and trends. Semin Musculoskelet Radiol 2019; 23 (03) 304-311
76 Prasoon A, Petersen K, Igel C, Lauze F, Dam E, Nielsen M. Deep feature learning for knee cartilage segmentation using a triplanar convolutional neural network. In: Salinesi C, Norrie MC, Pastor Ó. eds. Advanced Information Systems Engineering. Vol 7908. Berlin, Germany: Springer; 2013: 246-253
77 Liu F, Zhou Z, Jang H, Samsonov A, Zhao G, Kijowski R. Deep convolutional neural network and 3D deformable approach for tissue segmentation in musculoskeletal magnetic resonance imaging. Magn Reson Med 2018; 79 (04) 2379-2391
78 Heimann T, Morrison BJ, Styner MA, Niethammer M, Warfield SK. Segmentation of knee images: a grand challenge. Accessed October 18, 2019 at: http://www.ski10.org/ski10.pdf
79 Omoumi P, Becce F, Ott JG, Racine D, Verdun FR. Optimization of radiation dose and image quality in musculoskeletal CT: emphasis on iterative reconstruction techniques (Part 1). Semin Musculoskelet Radiol 2015; 19 (05) 415-421
80 Omoumi P, Verdun FR, Becce F. Optimization of radiation dose and image quality in musculoskeletal CT: emphasis on iterative reconstruction techniques (Part 2). Semin Musculoskelet Radiol 2015; 19 (05) 422-430
81 Rizzo S, Botta F, Raimondi S. et al. Radiomics: the facts and the challenges of image analysis. Eur Radiol Exp 2018; 2 (01) 36
82 Depeursinge A, Foncubierta-Rodriguez A, Van De Ville D, Müller H. Three-dimensional solid texture analysis in biomedical imaging: review and opportunities. Med Image Anal 2014; 18 (01) 176-196
83 Miles KA, Ganeshan B, Hayball MP. CT texture analysis using the filtration-histogram method: what do the measurements mean?. Cancer Imaging 2013; 13 (03) 400-406
84 Erickson BJ, Korfiatis P, Akkus Z, Kline TL. Machine learning for medical imaging. Radiographics 2017; 37 (02) 505-515
85 Yin P, Mao N, Zhao C, Wu J, Chen L, Hong N. A triple-classification radiomics model for the differentiation of primary chordoma, giant cell tumor, and metastatic tumor of sacrum based on T2-weighted and contrast-enhanced T1-weighted MRI. J Magn Reson Imaging 2019; 49 (03) 752-759
86 Park JE, Kim D, Kim HS. et al. Quality of science and reporting of radiomics in oncologic studies: room for improvement according to radiomics quality score and TRIPOD statement. Eur Radiol 2019; July 26 (Epub ahead of print)
87 Collins GS, Reitsma JB, Altman DG, Moons KGM. Transparent Reporting of a multivariable prediction model for Individual Prognosis or Diagnosis (TRIPOD): the TRIPOD statement. Ann Intern Med 2015; 162 (01) 55-63
88 Zwanenburg A, Leger S, Vallières M, Löck S. Image biomarker standardisation initiative. 2016 . Abstract Accessed September 1, 2019 at: http://arxiv.org/abs/1612.07003
89 Zhang Y, Zhu Y, Shi X. et al. Soft tissue sarcomas: preoperative predictive histopathological grading based on radiomics of MRI. Acad Radiol 2019; 26 (09) 1262-1268
90 Hayano K, Tian F, Kambadakone AR. et al. Texture analysis of non-contrast enhanced computed tomography for assessing angiogenesis and survival of soft tissue sarcoma. J Comp Assist Tomogr 2015; 39 (04) 607-612
91 Yi J, Lee YH, Kim SK. et al. Response evaluation of giant-cell tumor of bone treated by denosumab: histogram and texture analysis of CT images. J Orthop Sci 2018; 23 (03) 570-577
92 Ferizi U, Honig S, Chang G. Artificial intelligence, osteoporosis and fragility fractures. Curr Opin Rheumatol 2019; 31 (04) 368-375
93 Ho-Le TP, Center JR, Eisman JA, Nguyen TV, Nguyen HT. Prediction of hip fracture in post-menopausal women using artificial neural network approach. Conf Proc IEEE Eng Med Biol Soc 2017; 2017: 4207-4210
94 Kruse C, Eiken P, Vestergaard P. Clinical fracture risk evaluated by hierarchical agglomerative clustering. Osteoporos Int 2017; 28 (03) 819-832
95 Kruse C, Eiken P, Vestergaard P. Machine learning principles can improve hip fracture prediction. Calcif Tissue Int 2017; 100 (04) 348-360
96 Areeckal AS, Jayasheelan N, Kamath J, Zawadynski S, Kocher M, David S S. Early diagnosis of osteoporosis using radiogrammetry and texture analysis from hand and wrist radiographs in Indian population. Osteoporos Int 2018; 29 (03) 665-673
97 Singh A, Dutta MK, Jennane R, Lespessailles E. Classification of the trabecular bone structure of osteoporotic patients using machine vision. Comput Biol Med 2017; 91: 148-158
98 Valentinitsch A, Trebeschi S, Kaesmacher J. et al. Opportunistic osteoporosis screening in multi-detector CT images via local classification of textures. Osteoporos Int 2019; 30 (06) 1275-1285
99 Muehlematter UJ, Mannil M, Becker AS. et al. Vertebral body insufficiency fractures: detection of vertebrae at risk on standard CT images using texture analysis and machine learning. Eur Radiol 2019; 29 (05) 2207-2217
100 Dimai HP, Ljuhar R, Ljuhar D. et al. Assessing the effects of long-term osteoporosis treatment by using conventional spine radiographs: results from a pilot study in a sub-cohort of a large randomized controlled trial. Skeletal Radiol 2019; 48 (07) 1023-1032
101 Carballido-Gamio J, Majumdar S. Atlas-based knee cartilage assessment. Magn Reson Med 2011; 66 (02) 574-583
102 Ashinsky BG, Bouhrara M, Coletta CE. et al. Predicting early symptomatic osteoarthritis in the human knee using machine learning classification of magnetic resonance images from the osteoarthritis initiative. J Orthop Res 2017; 35 (10) 2243-2250
103 Pedoia V, Li X, Su F, Calixto N, Majumdar S. Fully automatic analysis of the knee articular cartilage T1ρ relaxation time using voxel-based relaxometry. J Magn Reson Imaging 2016; 43 (04) 970-980
104 Favre J, Erhart-Hledik JC, Blazek K, Fasel B, Gold GE, Andriacchi TP. Anatomically standardized maps reveal distinct patterns of cartilage thickness with increasing severity of medial compartment knee osteoarthritis. J Orthop Res 2017; 35 (11) 2442-2451
105 Edd SN, Omoumi P, Andriacchi TP, Jolles BM, Favre J. Modeling knee osteoarthritis pathophysiology using an integrated joint system (IJS): a systematic review of relationships among cartilage thickness, gait mechanics, and subchondral bone mineral density. Osteoarthritis Cartilage 2018; 26 (11) 1425-1437
106 Andriacchi TP, Favre J, Erhart-Hledik JC, Chu CR. A systems view of risk factors for knee osteoarthritis reveals insights into the pathogenesis of the disease. Ann Biomed Eng 2015; 43 (02) 376-387
107 Yushkevich PA, Piven J, Hazlett HC. et al. User-guided 3D active contour segmentation of anatomical structures: significantly improved efficiency and reliability. Neuroimage 2006; 31 (03) 1116-1128