Quantification in Musculoskeletal Imaging Using Computational Analysis and Machine Learning: Segmentation and Radiomics

Abstract

Although still limited in clinical practice, quantitative analysis is expected to increase the value of musculoskeletal (MSK) imaging. Segmentation aims at isolating the tissues and/or regions of interest in the image and is crucial to the extraction of quantitative features such as size, signal intensity, or image texture. These features may serve to support the diagnosis and monitoring of disease. Radiomics refers to the process of extracting large amounts of features from radiologic images and combining them with clinical, biological, genetic, or any other type of complementary data to build diagnostic, prognostic, or predictive models. The advent of machine learning offers promising prospects for automatic segmentation and integration of large amounts of data. We present commonly used segmentation methods and describe the radiomics pipeline, highlighting the challenges to overcome for adoption in clinical practice. We provide some examples of applications from the MSK literature.

Keywords

artificial intelligence - radiomics - musculoskeletal imaging - radiology

Publication History

Article published online:
28 January 2020

• References

• 1 Buckler AJ, Bresolin L, Dunnick NR, Sullivan DC. Group. A collaborative enterprise for multi-stakeholder participation in the advancement of quantitative imaging. Radiology 2011; 258 (03) 906-914
• 2 Mansoor A, Bagci U, Foster B. et al. Segmentation and image analysis of abnormal lungs at CT: current approaches, challenges, and future trends. Radiographics 2015; 35 (04) 1056-1076
• 3 Traverso A, Wee L, Dekker A, Gillies R. Repeatability and reproducibility of radiomic features: a systematic review. Int J Radiat Oncol Biol Phys 2018; 102 (04) 1143-1158
• 4 Gillies RJ, Anderson AR, Gatenby RA, Morse DL. The biology underlying molecular imaging in oncology: from genome to anatome and back again. Clin Radiol 2010; 65 (07) 517-521
• 5 Limkin EJ, Sun R, Dercle L. et al. Promises and challenges for the implementation of computational medical imaging (radiomics) in oncology. Ann Oncol 2017; 28 (06) 1191-1206
• 6 Gillies RJ, Kinahan PE, Hricak H. Radiomics: images are more than pictures, they are data. Radiology 2016; 278 (02) 563-577
• 7 Lee G, Lee HY, Park H. et al. Radiomics and its emerging role in lung cancer research, imaging biomarkers and clinical management: state of the art. Eur J Radiol 2017; 86: 297-307
• 8 Lambin P, Leijenaar RTH, Deist TM. et al. Radiomics: the bridge between medical imaging and personalized medicine. Nat Rev Clin Oncol 2017; 14 (12) 749-762
• 9 Bodalal Z, Trebeschi S, Beets-Tan R. Radiomics: a critical step towards integrated healthcare. Insights Imaging 2018; 9 (06) 911-914
• 10 Bodalal Z, Trebeschi S, Nguyen-Kim TDL, Schats W, Beets-Tan R. Radiogenomics: bridging imaging and genomics. Abdom Radiol (NY) 2019; 44 (06) 1960-1984
• 11 Afshar P, Mohammadi A, Plataniotis KN, Oikonomou A, Benali H. From hand-crafted to deep learning-based cancer radiomics: challenges and opportunities. IEEE Signal Process Mag 2019; 36 (04) 132-160
• 12 Aerts HJWL, Velazquez ER, Leijenaar RTH. et al. Decoding tumour phenotype by noninvasive imaging using a quantitative radiomics approach. Nat Commun 2014; 5 (01) 4006
• 13 Aerts HJWL. The potential of radiomic-based phenotyping in precision medicine: a review. JAMA Oncol 2016; 2 (12) 1636-1642
• 14 Ma Z, Tavares JMRS, Jorge RN, Mascarenhas T. A review of algorithms for medical image segmentation and their applications to the female pelvic cavity. Comput Methods Biomech Biomed Engin 2010; 13 (02) 235-246
• 15 Pedoia V, Majumdar S, Link TM. Segmentation of joint and musculoskeletal tissue in the study of arthritis. MAGMA 2016; 29 (02) 207-221
• 16 Aprovitola A, Gallo L. Knee bone segmentation from MRI: a classification and literature review. Biocybern Biomed Eng 2016; 36 (02) 437-449
• 17 Zhang B, Zhang Y, Cheng HD. et al. Computer-aided knee joint magnetic resonance image segmentation—a survey. 2018 . Abstract Accessed August 16, 2019 at: http://arxiv.org/abs/1802.04894
• 18 Ryzhkov MD. Knee cartilage segmentation algorithms: a critical literature review. Accessed August 16, 2019 at: http://dspace.library.uu.nl/handle/1874/308831 2015
• 19 Litjens G, Kooi T, Bejnordi BE. et al. A survey on deep learning in medical image analysis. Med Image Anal 2017; 42: 60-88
• 20 John NW. Segmentation of radiological images. In: Neri E, Caramella D, Bartolozzi C. eds. Image Processing in Radiology. Berlin, Germany: Springer; 2008: 45-54
• 21 Dalvi R, Abugharbieh R, Wilson D, Wilson DR. Multi-contrast MR for enhanced bone imaging and segmentation. Conf Proc IEEE Eng Med Biol Soc 2007; 2007: 5620-5623
• 22 Li X, Ma BC, Bolbos RI. et al. Quantitative assessment of bone marrow edema-like lesion and overlying cartilage in knees with osteoarthritis and anterior cruciate ligament tear using MR imaging and spectroscopic imaging at 3 Tesla. J Magn Reson Imaging 2008; 28 (02) 453-461
• 23 Li W, Abram F, Pelletier JP. et al. Fully automated system for the quantification of human osteoarthritic knee joint effusion volume using magnetic resonance imaging. Arthritis Res Ther 2010; 12 (05) R173
• 24 Kim S, Lee D, Park S, Oh KS, Chung SW, Kim Y. Automatic segmentation of supraspinatus from MRI by internal shape fitting and autocorrection. Comput Methods Programs Biomed 2017; 140: 165-174
• 25 Hacihaliloglu I. Ultrasound imaging and segmentation of bone surfaces: a review. Technology (Singap World Sci) 2017; 5 (02) 74-80
• 26 Zhang J, Yan CH, Chui CK, Ong SH. Fast segmentation of bone in CT images using 3D adaptive thresholding. Comput Biol Med 2010; 40 (02) 231-236
• 27 Jena S, Sahu B, Jagadev AK. Analysis of medical X-ray bone images using image segmentation. In: Jain LC, Patnaik S, Ichalkaranje N. eds. Intelligent Computing, Communication and Devices. Advances in Intelligent Systems and Computing. New Delhi, India: Springer India; 2015: 787-794
• 28 van Eijnatten M, van Dijk R, Dobbe J, Streekstra G, Koivisto J, Wolff J. CT image segmentation methods for bone used in medical additive manufacturing. Med Eng Phys 2018; 51: 6-16
• 29 Silva G, Oliveira L, Pithon M. Automatic segmenting teeth in X-ray images: Trends, a novel data set, benchmarking and future perspectives. Expert Syst Appl 2018; 107: 15-31
• 30 Szeliski R. Computer Vision: Algorithms and Applications. London, UK: Springer; 2011. . Accessed August 16, 2019 at: https://www.springer.com/gp/book/9781848829343
• 31 Serra J. Introduction to mathematical morphology. Comput Vis Graph Image Process 1986; 35 (03) 283-305
• 32 Lee JS, Chung YN. Integrating edge detection and thresholding approaches to segmenting femora and patellae from magnetic resonance images. Biomed Eng Appl Basis Commun 2005; 17 (01) 1-11
• 33 Carballido-Gamio J, Bauer JS, Stahl R. et al. Inter-subject comparison of MRI knee cartilage thickness. Med Image Anal 2008; 12 (02) 120-135
• 34 Duryea J, Neumann G, Brem MH. et al. Novel fast semi-automated software to segment cartilage for knee MR acquisitions. Osteoarthritis Cartilage 2007; 15 (05) 487-492
• 35 Iranpour-Boroujeni T, Watanabe A, Bashtar R, Yoshioka H, Duryea J. Quantification of cartilage loss in local regions of knee joints using semi-automated segmentation software: analysis of longitudinal data from the Osteoarthritis Initiative (OAI). Osteoarthritis Cartilage 2011; 19 (03) 309-314
• 36 Brem MH, Lang PK, Neumann G. et al. Magnetic resonance image segmentation using semi-automated software for quantification of knee articular cartilage---initial evaluation of a technique for paired scans. Skeletal Radiol 2009; 38 (05) 505-511
• 37 Adams R, Bischof L. Seeded region growing. IEEE Trans Pattern Anal Mach Intell 1994; 16 (06) 641-647
• 38 Chang YL, Li X. Adaptive image region-growing. IEEE Trans Image Process 1994; 3 (06) 868-872
• 39 Waterton JC, Solloway S, Foster JE. et al. Diurnal variation in the femoral articular cartilage of the knee in young adult humans. Magn Reson Med 2000; 43 (01) 126-132
• 40 Digabel H, Lantuejoul Ch. Iterative Algorithms, Special Issues of Practical Metallography. Vol. 8. Stuttgart, Germany: Riederer; 1977
• 41 Meyer F. Topographic distance and watershed lines. Signal Processing 1994; 38 (01) 113-125
• 42 Kornilov A, Safonov I. An overview of watershed algorithm implementations in open source libraries. J Imaging 2018; 4: 123
• 43 Grau V, Mewes AUJ, Alcañiz M, Kikinis R, Warfield SK. Improved watershed transform for medical image segmentation using prior information. IEEE Trans Med Imaging 2004; 23 (04) 447-458
• 44 Włodarczyk J, Czaplicka K, Tabor Z, Wojciechowski W, Urbanik A. Segmentation of bones in magnetic resonance images of the wrist. Int J CARS 2015; 10 (04) 419-431
• 45 Desai P, Hacihaliloglu I. Knee-cartilage segmentation and thickness measurement from 2D ultrasound. J Imaging 2019; 5: 43
• 46 Caselles V, Kimmel R, Sapiro G. Geodesic active contours. Int J Comput Vis 1997; 22 (01) 61-79
• 47 Malladi R, Sethian JA, Vemuri BC. Shape modeling with front propagation: a level set approach. IEEE Trans Pattern Anal Mach Intell 1995; 17: 158-175
• 48 Osher S, Sethian JA. Fronts propagating with curvature-dependent speed: Algorithms based on Hamilton-Jacobi formulations. J Comput Phys 1988; 79 (01) 12-49
• 49 Sethian JA. Level Set Methods and Fast Marching Methods: Evolving Interfaces in Computational Geometry, Fluid Mechanics, Computer Vision, and Materials Science. 2nd ed. Cambridge, UK: Cambridge University Press; 1999
• 50 Chan TF, Vese LA. Active contours without edges. IEEE Trans Image Process 2001; 10 (02) 266-277
• 51 Li W, Abram F, Beaudoin G, Berthiaume MJ, Pelletier JP, Martel-Pelletier J. Human hip joint cartilage: MRI quantitative thickness and volume measurements discriminating acetabulum and femoral head. IEEE Trans Biomed Eng 2008; 55 (12) 2731-2740
• 52 Cootes TF, Taylor CJ, Cooper DH, Graham J. Active shape models-their training and application. Comput Vis Image Underst 1995; 61: 38-59
• 53 Cootes TF, Edwards GJ, Taylor CJ. Comparing active shape models with active appearance models. In: Pridmore T. , Elliman, eds. Proceedings of the British Machine Vision Conference. Durham, UK: BMVA Press; 1999: 18.1-18.10
• 54 Davies RH, Twining CJ, Cootes TF, Waterton JC, Taylor CJ. A minimum description length approach to statistical shape modeling. IEEE Trans Med Imaging 2002; 21 (05) 525-537
• 55 Heimann T, Meinzer H-P. Statistical shape models for 3D medical image segmentation: a review. Med Image Anal 2009; 13 (04) 543-563
• 56 Solloway S, Hutchinson CE, Waterton JC, Taylor CJ. The use of active shape models for making thickness measurements of articular cartilage from MR images. Magn Reson Med 1997; 37 (06) 943-952
• 57 Fripp J, Crozier S, Warfield SK, Ourselin S. Automatic segmentation and quantitative analysis of the articular cartilages from magnetic resonance images of the knee. IEEE Trans Med Imaging 2010; 29 (01) 55-64
• 58 Fripp J, Crozier S, Warfield SK, Ourselin S. Automatic segmentation of the bone and extraction of the bone-cartilage interface from magnetic resonance images of the knee. Phys Med Biol 2007; 52 (06) 1617-1631
• 59 Schmid J, Kim J, Magnenat-Thalmann N. Robust statistical shape models for MRI bone segmentation in presence of small field of view. Med Image Anal 2011; 15 (01) 155-168
• 60 Bach Cuadra M, Duay V, Thiran J-Ph. Atlas-based segmentation. In: Paragios N, Ayache N, Duncan J. eds. Handbook of Biomedical Imaging: Methodologies and Clinical Research. New York, NY: Springer; 2015: 221-244
• 61 Iglesias JE, Sabuncu MR. Multi-atlas segmentation of biomedical images: A survey. Med Image Anal 2015; 24 (01) 205-219
• 62 Kapur T, Beardsley P, Frisken S, Eric W, Grimson L, Wells W. Model-based segmentation of clinical knee MRI. Paper presented at: IEEE International Workshop on Model-Based 3D Image Analysis; 1998 Bombay, India:
• 63 Lee JG, Gumus S, Moon CH, Kwoh CK, Bae KT. Fully automated segmentation of cartilage from the MR images of knee using a multi-atlas and local structural analysis method. Med Phys 2014; 41 (09) 092303
• 64 Chen X, Pan L. A Survey of graph cuts/graph search based medical image segmentation. IEEE Rev Biomed Eng 2018; 11: 112-124
• 65 Boykov Y, Veksler O, Zabih R. Fast approximate energy minimization via graph cuts. Accessed October 18, 2019 at: http://www.cs.cornell.edu/rdz/Papers/BVZ-iccv99.pdf
• 66 Shim H, Chang S, Tao C, Wang JH, Kwoh CK, Bae KT. Knee cartilage: efficient and reproducible segmentation on high-spatial-resolution MR images with the semiautomated graph-cut algorithm method. Radiology 2009; 251 (02) 548-556
• 67 Bishop CM. Pattern Recognition and Machine Learning. New York, NY: Springer; 2006
• 68 Zhang DQ, Chen SC. A novel kernelized fuzzy C-means algorithm with application in medical image segmentation. Artif Intell Med 2004; 32 (01) 37-50
• 69 Bezdek JC. Pattern recognition with fuzzy objective function algorithms. Springer US; 1981. . Accessed August 26, 2019 at: https://www.springer.com/gp/book/9781475704525
• 70 Carballido-Gamio J, Phan C, Link TM, Majumdar S. Characterization of trabecular bone structure from high-resolution magnetic resonance images using fuzzy logic. Magn Reson Imaging 2006; 24 (08) 1023-1029
• 71 Folkesson J, Carballido-Gamio J, Eckstein F, Link TM, Majumdar S. Local bone enhancement fuzzy clustering for segmentation of MR trabecular bone images. Med Phys 2010; 37 (01) 295-302
• 72 Folkesson J, Dam EB, Olsen OF, Pettersen PC, Christiansen C. Segmenting articular cartilage automatically using a voxel classification approach. IEEE Trans Med Imaging 2007; 26 (01) 106-115
• 73 Zhang K, Lu W, Marziliano P. Automatic knee cartilage segmentation from multi-contrast MR images using support vector machine classification with spatial dependencies. Magn Reson Imaging 2013; 31 (10) 1731-1743
• 74 Goodfellow I, Bengio Y, Courville A. Deep Learning. Cambridge, MA: MIT Press; 2016
• 75 Hirschmann A, Cyriac J, Stieltjes B, Kober T, Richiardi J, Omoumi P. Artificial intelligence in musculoskeletal imaging: review of current literature, challenges, and trends. Semin Musculoskelet Radiol 2019; 23 (03) 304-311
• 76 Prasoon A, Petersen K, Igel C, Lauze F, Dam E, Nielsen M. Deep feature learning for knee cartilage segmentation using a triplanar convolutional neural network. In: Salinesi C, Norrie MC, Pastor Ó. eds. Advanced Information Systems Engineering. Vol 7908. Berlin, Germany: Springer; 2013: 246-253
• 77 Liu F, Zhou Z, Jang H, Samsonov A, Zhao G, Kijowski R. Deep convolutional neural network and 3D deformable approach for tissue segmentation in musculoskeletal magnetic resonance imaging. Magn Reson Med 2018; 79 (04) 2379-2391
• 78 Heimann T, Morrison BJ, Styner MA, Niethammer M, Warfield SK. Segmentation of knee images: a grand challenge. Accessed October 18, 2019 at: http://www.ski10.org/ski10.pdf
• 79 Omoumi P, Becce F, Ott JG, Racine D, Verdun FR. Optimization of radiation dose and image quality in musculoskeletal CT: emphasis on iterative reconstruction techniques (Part 1). Semin Musculoskelet Radiol 2015; 19 (05) 415-421
• 80 Omoumi P, Verdun FR, Becce F. Optimization of radiation dose and image quality in musculoskeletal CT: emphasis on iterative reconstruction techniques (Part 2). Semin Musculoskelet Radiol 2015; 19 (05) 422-430
• 81 Rizzo S, Botta F, Raimondi S. et al. Radiomics: the facts and the challenges of image analysis. Eur Radiol Exp 2018; 2 (01) 36
• 82 Depeursinge A, Foncubierta-Rodriguez A, Van De Ville D, Müller H. Three-dimensional solid texture analysis in biomedical imaging: review and opportunities. Med Image Anal 2014; 18 (01) 176-196
• 83 Miles KA, Ganeshan B, Hayball MP. CT texture analysis using the filtration-histogram method: what do the measurements mean?. Cancer Imaging 2013; 13 (03) 400-406
• 84 Erickson BJ, Korfiatis P, Akkus Z, Kline TL. Machine learning for medical imaging. Radiographics 2017; 37 (02) 505-515
• 85 Yin P, Mao N, Zhao C, Wu J, Chen L, Hong N. A triple-classification radiomics model for the differentiation of primary chordoma, giant cell tumor, and metastatic tumor of sacrum based on T2-weighted and contrast-enhanced T1-weighted MRI. J Magn Reson Imaging 2019; 49 (03) 752-759
• 86 Park JE, Kim D, Kim HS. et al. Quality of science and reporting of radiomics in oncologic studies: room for improvement according to radiomics quality score and TRIPOD statement. Eur Radiol 2019; July 26 (Epub ahead of print)
• 87 Collins GS, Reitsma JB, Altman DG, Moons KGM. Transparent Reporting of a multivariable prediction model for Individual Prognosis or Diagnosis (TRIPOD): the TRIPOD statement. Ann Intern Med 2015; 162 (01) 55-63
• 88 Zwanenburg A, Leger S, Vallières M, Löck S. Image biomarker standardisation initiative. 2016 . Abstract Accessed September 1, 2019 at: http://arxiv.org/abs/1612.07003
• 89 Zhang Y, Zhu Y, Shi X. et al. Soft tissue sarcomas: preoperative predictive histopathological grading based on radiomics of MRI. Acad Radiol 2019; 26 (09) 1262-1268
• 90 Hayano K, Tian F, Kambadakone AR. et al. Texture analysis of non-contrast enhanced computed tomography for assessing angiogenesis and survival of soft tissue sarcoma. J Comp Assist Tomogr 2015; 39 (04) 607-612
• 91 Yi J, Lee YH, Kim SK. et al. Response evaluation of giant-cell tumor of bone treated by denosumab: histogram and texture analysis of CT images. J Orthop Sci 2018; 23 (03) 570-577
• 92 Ferizi U, Honig S, Chang G. Artificial intelligence, osteoporosis and fragility fractures. Curr Opin Rheumatol 2019; 31 (04) 368-375
• 93 Ho-Le TP, Center JR, Eisman JA, Nguyen TV, Nguyen HT. Prediction of hip fracture in post-menopausal women using artificial neural network approach. Conf Proc IEEE Eng Med Biol Soc 2017; 2017: 4207-4210
• 94 Kruse C, Eiken P, Vestergaard P. Clinical fracture risk evaluated by hierarchical agglomerative clustering. Osteoporos Int 2017; 28 (03) 819-832
• 95 Kruse C, Eiken P, Vestergaard P. Machine learning principles can improve hip fracture prediction. Calcif Tissue Int 2017; 100 (04) 348-360
• 96 Areeckal AS, Jayasheelan N, Kamath J, Zawadynski S, Kocher M, David S S. Early diagnosis of osteoporosis using radiogrammetry and texture analysis from hand and wrist radiographs in Indian population. Osteoporos Int 2018; 29 (03) 665-673
• 97 Singh A, Dutta MK, Jennane R, Lespessailles E. Classification of the trabecular bone structure of osteoporotic patients using machine vision. Comput Biol Med 2017; 91: 148-158
• 98 Valentinitsch A, Trebeschi S, Kaesmacher J. et al. Opportunistic osteoporosis screening in multi-detector CT images via local classification of textures. Osteoporos Int 2019; 30 (06) 1275-1285
• 99 Muehlematter UJ, Mannil M, Becker AS. et al. Vertebral body insufficiency fractures: detection of vertebrae at risk on standard CT images using texture analysis and machine learning. Eur Radiol 2019; 29 (05) 2207-2217
• 100 Dimai HP, Ljuhar R, Ljuhar D. et al. Assessing the effects of long-term osteoporosis treatment by using conventional spine radiographs: results from a pilot study in a sub-cohort of a large randomized controlled trial. Skeletal Radiol 2019; 48 (07) 1023-1032
• 101 Carballido-Gamio J, Majumdar S. Atlas-based knee cartilage assessment. Magn Reson Med 2011; 66 (02) 574-583
• 102 Ashinsky BG, Bouhrara M, Coletta CE. et al. Predicting early symptomatic osteoarthritis in the human knee using machine learning classification of magnetic resonance images from the osteoarthritis initiative. J Orthop Res 2017; 35 (10) 2243-2250
• 103 Pedoia V, Li X, Su F, Calixto N, Majumdar S. Fully automatic analysis of the knee articular cartilage T1ρ relaxation time using voxel-based relaxometry. J Magn Reson Imaging 2016; 43 (04) 970-980
• 104 Favre J, Erhart-Hledik JC, Blazek K, Fasel B, Gold GE, Andriacchi TP. Anatomically standardized maps reveal distinct patterns of cartilage thickness with increasing severity of medial compartment knee osteoarthritis. J Orthop Res 2017; 35 (11) 2442-2451
• 105 Edd SN, Omoumi P, Andriacchi TP, Jolles BM, Favre J. Modeling knee osteoarthritis pathophysiology using an integrated joint system (IJS): a systematic review of relationships among cartilage thickness, gait mechanics, and subchondral bone mineral density. Osteoarthritis Cartilage 2018; 26 (11) 1425-1437
• 106 Andriacchi TP, Favre J, Erhart-Hledik JC, Chu CR. A systems view of risk factors for knee osteoarthritis reveals insights into the pathogenesis of the disease. Ann Biomed Eng 2015; 43 (02) 376-387
• 107 Yushkevich PA, Piven J, Hazlett HC. et al. User-guided 3D active contour segmentation of anatomical structures: significantly improved efficiency and reliability. Neuroimage 2006; 31 (03) 1116-1128