Download PDF
Semin Musculoskelet Radiol 2020; 24(01): 38-49
DOI: 10.1055/s-0039-3400266
Review Article
Thieme Medical Publishers 333 Seventh Avenue, New York, NY 10001, USA.

Pattern Recognition in Musculoskeletal Imaging Using Artificial Intelligence

Natalia Gorelik
1   Department of Diagnostic Radiology, McGill University Health Center, Montreal, Quebec, Canada
,
Jaron Chong
1   Department of Diagnostic Radiology, McGill University Health Center, Montreal, Quebec, Canada
,
Dana J. Lin
2   Division of Musculoskeletal Radiology, Department of Radiology, NYU Langone Health, New York, New York
› Author Affiliations
Further Information

Publication History

Publication Date:
28 January 2020 (online)

Also available at
    Permissions and Reprints

Abstract

Artificial intelligence (AI) has the potential to affect every step of the radiology workflow, but the AI application that has received the most press in recent years is image interpretation, with numerous articles describing how AI can help detect and characterize abnormalities as well as monitor disease response. Many AI-based image interpretation tasks for musculoskeletal (MSK) pathologies have been studied, including the diagnosis of bone tumors, detection of osseous metastases, assessment of bone age, identification of fractures, and detection and grading of osteoarthritis. This article explores the applications of AI for image interpretation of MSK pathologies.

Keywords

musculoskeletal - deep learning - neural network
 

  • References

  • 1 Choy G, Khalilzadeh O, Michalski M. , et al. Current applications and future impact of machine learning in radiology. Radiology 2018; 288 (02) 318-328
    CrossrefPubMedSearch in Google Scholar
  • 2 European Society of Radiology (ESR). What the radiologist should know about artificial intelligence—an ESR white paper. Insights Imaging 2019; 10 (01) 44
    CrossrefPubMedSearch in Google Scholar
  • 3 Gyftopoulos S, Lin D, Knoll F, Doshi AM, Rodrigues TC, Recht MP. Artificial intelligence in musculoskeletal imaging: current status and future directions. AJR Am J Roentgenol 2019; 213 (03) 506-513
    CrossrefPubMedSearch in Google Scholar
  • 4 Hosny A, Parmar C, Quackenbush J, Schwartz LH, Aerts HJWL. Artificial intelligence in radiology. Nat Rev Cancer 2018; 18 (08) 500-510
    CrossrefPubMedSearch in Google Scholar
  • 5 Tang A, Tam R, Cadrin-Chênevert A. , et al; Canadian Association of Radiologists (CAR) Artificial Intelligence Working Group. Canadian Association of Radiologists white paper on artificial intelligence in radiology. Can Assoc Radiol J 2018; 69 (02) 120-135
    CrossrefPubMedSearch in Google Scholar
  • 6 Mutasa S, Chang PD, Ruzal-Shapiro C, Ayyala R. MABAL: a novel deep-learning architecture for machine-assisted bone age labeling. J Digit Imaging 2018; 31 (04) 513-519
    CrossrefPubMedSearch in Google Scholar
  • 7 Ping YY, Yin CW, Kok LP. Computer Aided Bone Tumor Detection and Classification Using X-ray Images. Berlin, Germany: Springer; 2008
    Search in Google Scholar
  • 8 Hammon M, Dankerl P, Tsymbal A. , et al. Automatic detection of lytic and blastic thoracolumbar spine metastases on computed tomography. Eur Radiol 2013; 23 (07) 1862-1870
    CrossrefPubMedSearch in Google Scholar
  • 9 Larson DB, Chen MC, Lungren MP, Halabi SS, Stence NV, Langlotz CP. Performance of a deep-learning neural network model in assessing skeletal maturity on pediatric hand radiographs. Radiology 2018; 287 (01) 313-322
    CrossrefPubMedSearch in Google Scholar
  • 10 Chung SW, Han SS, Lee JW. , et al. Automated detection and classification of the proximal humerus fracture by using deep learning algorithm. Acta Orthop 2018; 89 (04) 468-473
    CrossrefPubMedSearch in Google Scholar
  • 11 Antony J, McGuinness K, O'Connor NE, Moran K. Quantifying radiographic knee osteoarthritis severity using deep convolutional neural networks. Paper presented at: 23rd International Conference on Pattern Recognition (ICPR); December 4–8, 2016; Cancún, Mexico
    PubMed
  • 12 Syed AB, Zoga AC. Artificial intelligence in radiology: current technology and future directions. Semin Musculoskelet Radiol 2018; 22 (05) 540-545
    Thieme ConnectPubMedSearch in Google Scholar
  • 13 The Lodwick Award. Massachusetts General Hospital. www.massgeneral.org/imaging/about/lodwick-award.aspx . Accessed June 1, 2019
    PubMed
  • 14 Lodwick GS, Haun CL, Smith WE, Keller RF, Robertson ED. Computer diagnosis of primary bone tumors: a preliminary report. Radiology 1963; 80: 273-275
    CrossrefPubMedSearch in Google Scholar
  • 15 Kahn Jr CE, Laur JJ, Carrera GF. A Bayesian network for diagnosis of primary bone tumors. J Digit Imaging 2001; 14 (02) (Suppl. 01) 56-57
    CrossrefPubMedSearch in Google Scholar
  • 16 Lejbkowicz I, Wiener F, Nachtigal A, Militiannu D, Kleinhaus U, Applbaum YH. Bone Browser a decision-aid for the radiological diagnosis of bone tumors. Comput Methods Programs Biomed 2002; 67 (02) 137-154
    CrossrefPubMedSearch in Google Scholar
  • 17 Do BH, Langlotz C, Beaulieu CF. Bone tumor diagnosis using a naïve Bayesian model of demographic and radiographic features. J Digit Imaging 2017; 30 (05) 640-647
    CrossrefPubMedSearch in Google Scholar
  • 18 Reinus WR, Wilson AJ, Kalman B, Kwasny S. Diagnosis of focal bone lesions using neural networks. Invest Radiol 1994; 29 (06) 606-611
    CrossrefPubMedSearch in Google Scholar
  • 19 Bandyopadhyay O, Biswas A, Bhattacharya BB. Bone-cancer assessment and destruction pattern analysis in long-bone X-ray image. J Digit Imaging 2019; 32 (02) 300-313
    CrossrefPubMedSearch in Google Scholar
  • 20 Kumar R, Suhas MV. Classification of benign and malignant bone lesions on CT images using support vector machine: A comparison of kernel functions. Paper presented at: IEEE International Conference on Recent Trends in Electronics, Information & Communication Technology; May 20–21, 2016; Bangalore, India
    PubMed
  • 21 Mishra A, Suhas MV. Classification of benign and malignant bone lesions on CT images using Random Forest. Paper presented at: IEEE International Conference on Recent Trends in Electronics Information Communication Technology; May 20–21, 2016; Bangalore, India
    PubMed
  • 22 Banerjee I, Kurtz C, Devorah AE, Do B, Rubin DL, Beaulieu CF. Relevance feedback for enhancing content based image retrieval and automatic prediction of semantic image features: application to bone tumor radiographs. J Biomed Inform 2018; 84: 123-135
    CrossrefPubMedSearch in Google Scholar
  • 23 Burns JE, Yao J, Wiese TS, Muñoz HE, Jones EC, Summers RM. Automated detection of sclerotic metastases in the thoracolumbar spine at CT. Radiology 2013; 268 (01) 69-78
    CrossrefPubMedSearch in Google Scholar
  • 24 Wiese T, Yao J, Burns JE, Summers RM. Detection of sclerotic bone metastases in the spine using watershed algorithm and graph cut. Proc. SPIE 8315, Medical Imaging 2012: Computer-Aided Diagnosis, 831512, February 23, 2012; https://doi.org/10.1117/12.911700
    CrossrefPubMed
  • 25 Roth HR, Lu L, Liu J. , et al. Improving computer-aided detection using convolutional neural networks and random view aggregation. IEEE Trans Med Imaging 2016; 35 (05) 1170-1181
    CrossrefPubMedSearch in Google Scholar
  • 26 Yao J, O'Connor SD, Summers R. Computer aided lytic bone metastasis detection using regular CT images. Proc. SPIE 6144, Medical Imaging 2006: Image Processing, 614459, March 15, 2006; https://doi.org/10.1117/12.652288
    CrossrefPubMed
  • 27 O'Connor SD, Yao J, Summers RM. Lytic metastases in thoracolumbar spine: computer-aided detection at CT--preliminary study. Radiology 2007; 242 (03) 811-816
    CrossrefPubMedSearch in Google Scholar
  • 28 Wels M, Kelm BM, Tsymbal A. , et al. Multi-stage osteolytic spinal bone lesion detection from CT data with internal sensitivity control. Proc. SPIE 8315, Medical Imaging 2012: Computer-Aided Diagnosis, 831513, February 23, 2012; https://doi.org/10.1117/12.911169
    CrossrefPubMed
  • 29 Huang SF, Chiang KH. Automatic detection of bone metastasis in vertebrae by using CT images. Paper presented at: Proceedings of the World Congress on Engineering; July 4–6, 2012; London, UK
    PubMed
  • 30 Yao J, Burns JE, Sanoria V, Summers RM. Mixed spine metastasis detection through positron emission tomography/computed tomography synthesis and multiclassifier. J Med Imaging (Bellingham) 2017; 4 (02) 024504
    PubMedSearch in Google Scholar
  • 31 Yao J, Burns JE, Summers RM. Sclerotic rib metastases detection on routine CT images. Paper presented at: 2012 9th IEEE International Symposium on Biomedical Imaging (ISBI); May 2, 2012; Barcelona, Spain
    PubMed
  • 32 Oh J, Kim G, Lee J. , et al. Automated detection of bone metastatic changes using serial CT scans. Comput Med Imaging Graph 2017; 58: 62-74
    CrossrefPubMedSearch in Google Scholar
  • 33 Sakamoto R, Yakami M, Fujimoto K. , et al. Temporal subtraction of serial CT images with large deformation diffeomorphic metric mapping in the identification of bone metastases. Radiology 2017; 285 (02) 629-639
    CrossrefPubMedSearch in Google Scholar
  • 34 Ueno M, Aoki T, Murakami S. , et al. CT temporal subtraction method for detection of sclerotic bone metastasis in the thoracolumbar spine. Eur J Radiol 2018; 107: 54-59
    CrossrefPubMedSearch in Google Scholar
  • 35 Martínez-Martínez F, Kybic J, Lambert L, Mecková Z. Fully automated classification of bone marrow infiltration in low-dose CT of patients with multiple myeloma based on probabilistic density model and supervised learning. Comput Biol Med 2016; 71: 57-66
    CrossrefPubMedSearch in Google Scholar
  • 36 Jerebko AK, Schmidt GP, Zhou X. , et al. Robust parametric modeling approach based on domain knowledge for computer aided detection of vertebrae column metastases in MRI. Inf Process Med Imaging 2007; 20: 713-724
    PubMedSearch in Google Scholar
  • 37 Perk T, Bradshaw T, Chen S. , et al. Automated classification of benign and malignant lesions in 18F-NaF PET/CT images using machine learning. Phys Med Biol 2018; 63 (22) 225019
    CrossrefPubMedSearch in Google Scholar
  • 38 Bradshaw T, Perk T, Chen S. , et al. Deep learning for classification of benign and malignant bone lesions in [F-18] NaF PET/CT images. J Nucl Med 2018; 59 (Suppl. 01) 327-327
    CrossrefPubMedSearch in Google Scholar
  • 39 Xu L, Tetteh G, Lipkova J. , et al. Automated whole-body bone lesion detection for multiple myeloma on 68Ga-pentixafor PET/CT imaging using deep learning methods. Contrast Media Mol Imaging 2018; 2018: 2391925
    PubMedSearch in Google Scholar
  • 40 Greulich WW, Pyle SI. Radiographic Atlas of Skeletal Development of the Hand and Wrist. 2nd ed. Stanford, CA:: Stanford University Press;; 1959
    Search in Google Scholar
  • 41 Tanner JM, Whitehouse R, Cameron N, Marshall W, Healy M, Goldstein H. Assessment of Skeletal Maturity and Prediction of Adult Height (TW2 Method). Vol 16. London, UK:: Academic Press;; 1975
    Search in Google Scholar
  • 42 Tajmir SH, Lee H, Shailam R. , et al. Artificial intelligence-assisted interpretation of bone age radiographs improves accuracy and decreases variability. Skeletal Radiol 2019; 48 (02) 275-283
    CrossrefPubMedSearch in Google Scholar
  • 43 Tanner JM, Gibbons RD. A computerized image analysis system for estimating Tanner-Whitehouse 2 bone age. Horm Res 1994; 42 (06) 282-287
    CrossrefPubMedSearch in Google Scholar
  • 44 Michael DJ, Nelson AC. HANDX: a model-based system for automatic segmentation of bones from digital hand radiographs. IEEE Trans Med Imaging 1989; 8 (01) 64-69
    CrossrefPubMedSearch in Google Scholar
  • 45 Lee H, Tajmir S, Lee J. , et al. Fully automated deep learning system for bone age assessment. J Digit Imaging 2017; 30 (04) 427-441
    CrossrefPubMedSearch in Google Scholar
  • 46 Pietka E, McNitt-Gray MF, Kuo ML, Huang HK. Computer-assisted phalangeal analysis in skeletal age assessment. IEEE Trans Med Imaging 1991; 10 (04) 616-620
    CrossrefPubMedSearch in Google Scholar
  • 47 Tanner JM, Gibbons RD. Automatic bone age measurement using computerized image analysis. J Pediatr Endocrinol Metab 1994; 7 (02) 141-145
    CrossrefPubMedSearch in Google Scholar
  • 48 Thodberg HH, Kreiborg S, Juul A, Pedersen KD. The BoneXpert method for automated determination of skeletal maturity. IEEE Trans Med Imaging 2009; 28 (01) 52-66
    CrossrefPubMedSearch in Google Scholar
  • 49 van Rijn RR, Thodberg HH. Bone age assessment: automated techniques coming of age?. Acta Radiol 2013; 54 (09) 1024-1029
    CrossrefPubMedSearch in Google Scholar
  • 50 Zhang J, Lin F, Ding X. Maturation disparity between hand-wrist bones in a chinese sample of normal children: an analysis based on automatic BoneXpert and manual Greulich and Pyle atlas assessment. Korean J Radiol 2016; 17 (03) 435-442
    CrossrefPubMedSearch in Google Scholar
  • 51 Halabi SS, Prevedello LM, Kalpathy-Cramer J. , et al. The RSNA pediatric bone age machine learning challenge. Radiology 2019; 290 (02) 498-503
    CrossrefPubMedSearch in Google Scholar
  • 52 Kim JR, Shim WH, Yoon HM. , et al. Computerized bone age estimation using deep learning based program: evaluation of the accuracy and efficiency. AJR Am J Roentgenol 2017; 209 (06) 1374-1380
    CrossrefPubMedSearch in Google Scholar
  • 53 Zhang A, Sayre JW, Vachon L, Liu BJ, Huang HK. Racial differences in growth patterns of children assessed on the basis of bone age. Radiology 2009; 250 (01) 228-235
    CrossrefPubMedSearch in Google Scholar
  • 54 Kim DH, MacKinnon T. Artificial intelligence in fracture detection: transfer learning from deep convolutional neural networks. Clin Radiol 2018; 73 (05) 439-445
    CrossrefPubMedSearch in Google Scholar
  • 55 Lindsey R, Daluiski A, Chopra S. , et al. Deep neural network improves fracture detection by clinicians. Proc Natl Acad Sci U S A 2018; 115 (45) 11591-11596
    CrossrefPubMedSearch in Google Scholar
  • 56 Cheng CT, Ho TY, Lee TY. , et al. Application of a deep learning algorithm for detection and visualization of hip fractures on plain pelvic radiographs. Eur Radiol 2019; 29 (10) 5469-5477
    CrossrefPubMedSearch in Google Scholar
  • 57 Urakawa T, Tanaka Y, Goto S, Matsuzawa H, Watanabe K, Endo N. Detecting intertrochanteric hip fractures with orthopedist-level accuracy using a deep convolutional neural network. Skeletal Radiol 2019; 48 (02) 239-244
    CrossrefPubMedSearch in Google Scholar
  • 58 Gale W, Oakden-Rayner L, Carneiro G, Bradley A, Palmer L. Detecting hip fractures with radiologist-level performance using deep neural networks. 2017 . Abstract available at: arxiv.org/abs/1711.06504
    PubMedSearch in Google Scholar
  • 59 Kitamura G, Chung CY, Moore II BE. Ankle fracture detection utilizing a convolutional neural network ensemble implemented with a small sample, de novo training, and multiview incorporation. J Digit Imaging 2019; 32 (04) 672-677
    CrossrefPubMedSearch in Google Scholar
  • 60 Olczak J, Fahlberg N, Maki A. , et al. Artificial intelligence for analyzing orthopedic trauma radiographs. Acta Orthop 2017; 88 (06) 581-586
    CrossrefPubMedSearch in Google Scholar
  • 61 Pranata YD, Wang KC, Wang JC. , et al. Deep learning and SURF for automated classification and detection of calcaneus fractures in CT images. Comput Methods Programs Biomed 2019; 171: 27-37
    CrossrefPubMedSearch in Google Scholar
  • 62 Kasai S, Li F, Shiraishi J, Doi K. Usefulness of computer-aided diagnosis schemes for vertebral fractures and lung nodules on chest radiographs. AJR Am J Roentgenol 2008; 191 (01) 260-265
    CrossrefPubMedSearch in Google Scholar
  • 63 Burns JE, Yao J, Summers RM. Vertebral body compression fractures and bone density: automated detection and classification on CT images. Radiology 2017; 284 (03) 788-797
    CrossrefPubMedSearch in Google Scholar
  • 64 Tomita N, Cheung YY, Hassanpour S. Deep neural networks for automatic detection of osteoporotic vertebral fractures on CT scans. Comput Biol Med 2018; 98: 8-15
    CrossrefPubMedSearch in Google Scholar
  • 65 Mehta SD, Sebro R. Computer-aided detection of incidental lumbar spine fractures from routine dual-energy X-ray absorptiometry (DEXA) studies using a support vector machine (SVM) classifier. J Digit Imaging 2019 ; May 6 (Epub ahead of print)
    PubMedSearch in Google Scholar
  • 66 Badgeley MA, Zech JR, Oakden-Rayner L. , et al. Deep learning predicts hip fracture using confounding patient and healthcare variables. NPJ Digit Med 2019; 2: 31
    CrossrefPubMedSearch in Google Scholar
  • 67 Zech JR, Badgeley MA, Liu M, Costa AB, Titano JJ, Oermann EK. Variable generalization performance of a deep learning model to detect pneumonia in chest radiographs: A cross-sectional study. PLoS Med 2018; 15 (11) e1002683
    CrossrefPubMedSearch in Google Scholar
  • 68 Yi PH, Kim TK, Wei J. , et al. Automated semantic labeling of pediatric musculoskeletal radiographs using deep learning. Pediatr Radiol 2019; 49 (08) 1066-1070
    CrossrefPubMedSearch in Google Scholar
  • 69 Jamaludin A, Lootus M, Kadir T. , et al; Genodisc Consortium. ISSLS PRIZE IN BIOENGINEERING SCIENCE 2017: Automation of reading of radiological features from magnetic resonance images (MRIs) of the lumbar spine without human intervention is comparable with an expert radiologist. Eur Spine J 2017; 26 (05) 1374-1383
    CrossrefPubMedSearch in Google Scholar
  • 70 Xue Y, Zhang R, Deng Y, Chen K, Jiang T. A preliminary examination of the diagnostic value of deep learning in hip osteoarthritis. PLoS One 2017; 12 (06) e0178992
    CrossrefPubMedSearch in Google Scholar
  • 71 Tiulpin A, Thevenot J, Rahtu E, Lehenkari P, Saarakkala S. Automatic knee osteoarthritis diagnosis from plain radiographs: a deep learning-based approach. Sci Rep 2018; 8 (01) 1727
    CrossrefPubMedSearch in Google Scholar
  • 72 Norman B, Pedoia V, Noworolski A, Link TM, Majumdar S. Applying densely connected convolutional neural networks for staging osteoarthritis severity from plain radiographs. J Digit Imaging 2019; 32 (03) 471-477
    CrossrefPubMedSearch in Google Scholar