Download PDF
CC BY-NC-ND 4.0 · Methods Inf Med
DOI: 10.1055/a-2305-2115
Original Article

Deep Learning for Predicting Progression of Patellofemoral Osteoarthritis Based on Lateral Knee Radiographs, Demographic Data, and Symptomatic Assessments

Neslihan Bayramoglu
1   Research Unit of Health Sciences and Technology, University of Oulu, Oulu, Finland
,
Martin Englund
2   Orthopaedics, Department of Clinical Sciences Lund Faculty of Medicine, Lund University, Lund, Sweden
,
Ida K. Haugen
3   Center for Treatment of Rheumatic and Musculoskeletal Diseases (REMEDY), Diakonhjemmet Hospital, Oslo, Norway
,
Muneaki Ishijima
4   Department of Orthopaedics, Faculty of Medicine, Juntendo University, Tokyo, Japan
,
Simo Saarakkala
1   Research Unit of Health Sciences and Technology, University of Oulu, Oulu, Finland
5   Department of Diagnostic Radiology, Oulu University Hospital, Oulu, Finland
› Author Affiliations Funding Multicenter Osteoarthritis Study (MOST) Funding Acknowledgment. MOST comprised four cooperative grants (Felson—AG18820; Torner—AG18832, Lewis—AG18947, and Nevitt—AG19069) funded by the National Institutes of Health, a branch of the Department of Health and Human Services, and conducted by MOST study investigators. This manuscript was prepared using MOST data and does not necessarily reflect the opinions or views of MOST investigators. We would like to acknowledge the NORDFORSK grant from the project “Molecular and structural biomarkers for personalised care in osteoarthritis” (Project No.: 116406).
› Further Information
   Permissions and Reprints

Abstract

Objective In this study, we propose a novel framework that utilizes deep learning and attention mechanisms to predict the radiographic progression of patellofemoral osteoarthritis (PFOA) over a period of 7 years.

Material and Methods This study included subjects (1,832 subjects, 3,276 knees) from the baseline of the Multicenter Osteoarthritis Study (MOST). Patellofemoral joint regions of interest were identified using an automated landmark detection tool (BoneFinder) on lateral knee X-rays. An end-to-end deep learning method was developed for predicting PFOA progression based on imaging data in a five-fold cross-validation setting. To evaluate the performance of the models, a set of baselines based on known risk factors were developed and analyzed using gradient boosting machine (GBM). Risk factors included age, sex, body mass index, and Western Ontario and McMaster Universities Arthritis Index score, and the radiographic osteoarthritis stage of the tibiofemoral joint (Kellgren and Lawrence [KL] score). Finally, to increase predictive power, we trained an ensemble model using both imaging and clinical data.

Results Among the individual models, the performance of our deep convolutional neural network attention model achieved the best performance with an area under the receiver operating characteristic curve (AUC) of 0.856 and average precision (AP) of 0.431, slightly outperforming the deep learning approach without attention (AUC = 0.832, AP = 0.4) and the best performing reference GBM model (AUC = 0.767, AP = 0.334). The inclusion of imaging data and clinical variables in an ensemble model allowed statistically more powerful prediction of PFOA progression (AUC = 0.865, AP = 0.447), although the clinical significance of this minor performance gain remains unknown. The spatial attention module improved the predictive performance of the backbone model, and the visual interpretation of attention maps focused on the joint space and the regions where osteophytes typically occur.

Conclusion This study demonstrated the potential of machine learning models to predict the progression of PFOA using imaging and clinical variables. These models could be used to identify patients who are at high risk of progression and prioritize them for new treatments. However, even though the accuracy of the models were excellent in this study using the MOST dataset, they should be still validated using external patient cohorts in the future.

Keywords

patellofemoral osteoarthritis - deep learning - prediction of osteoarthritis progression - knee

Authors' Contributions

N.B. originated the idea of the study, and performed the experiments and took major part in writing of the manuscript. S.S. supervised the project. All authors participated in producing the final manuscript draft and approved the final submitted version.


Supplementary Material



Publication History

Received: 12 August 2023

Accepted: 29 March 2024

Accepted Manuscript online:
11 April 2024

Article published online:
14 May 2024

© 2024. The Author(s). This is an open access article published by Thieme under the terms of the Creative Commons Attribution-NonDerivative-NonCommercial License, permitting copying and reproduction so long as the original work is given appropriate credit. Contents may not be used for commercial purposes, or adapted, remixed, transformed or built upon. (https://creativecommons.org/licenses/by-nc-nd/4.0/)

Georg Thieme Verlag KG
Rüdigerstraße 14, 70469 Stuttgart, Germany

 

  • References

  • 1 Lankhorst NE, Damen J, Oei EH. et al. Incidence, prevalence, natural course and prognosis of patellofemoral osteoarthritis: the Cohort Hip and Cohort Knee study. Osteoarthritis Cartilage 2017; 25 (05) 647-653
    CrossrefPubMedGoogle Scholar
  • 2 Duncan R, Peat G, Thomas E, Wood L, Hay E, Croft P. How do pain and function vary with compartmental distribution and severity of radiographic knee osteoarthritis?. Rheumatology (Oxford) 2008; 47 (11) 1704-1707
    CrossrefPubMedGoogle Scholar
  • 3 Crossley KM, Hinman RS. The patellofemoral joint: the forgotten joint in knee osteoarthritis. Osteoarthritis Cartilage 2011; 19 (07) 765-767
    CrossrefPubMedGoogle Scholar
  • 4 Duncan R, Peat G, Thomas E, Hay EM, Croft P. Incidence, progression and sequence of development of radiographic knee osteoarthritis in a symptomatic population. Ann Rheum Dis 2011; 70 (11) 1944-1948
    CrossrefPubMedGoogle Scholar
  • 5 Duncan R, Peat G, Thomas E, Wood L, Hay E, Croft P. Does isolated patellofemoral osteoarthritis matter?. Osteoarthritis Cartilage 2009; 17 (09) 1151-1155
    CrossrefPubMedGoogle Scholar
  • 6 Kim YM, Joo YB. Patellofemoral osteoarthritis. Knee Surg Relat Res 2012; 24 (04) 193-200
    CrossrefPubMedGoogle Scholar
  • 7 van Middelkoop M, Bennell KL, Callaghan MJ. et al. International patellofemoral osteoarthritis consortium: consensus statement on the diagnosis, burden, outcome measures, prognosis, risk factors and treatment. Semin Arthritis Rheum 2018; 47: 666-675
    CrossrefPubMedGoogle Scholar
  • 8 Kobayashi S, Pappas E, Fransen M, Refshauge K, Simic M. The prevalence of patellofemoral osteoarthritis: a systematic review and meta-analysis. Osteoarthritis Cartilage 2016; 24 (10) 1697-1707
    CrossrefPubMedGoogle Scholar
  • 9 Macri EM. Patellofemoral osteoarthritis: characterizing knee alignment and morphology [PhD thesis]. British Columbia: University of British Columbia; 2017
    Google Scholar
  • 10 Peat G, Duncan RC, Wood LRJ, Thomas E, Muller S. Clinical features of symptomatic patellofemoral joint osteoarthritis. Arthritis Res Ther 2012; 14 (02) R63
    CrossrefPubMedGoogle Scholar
  • 11 de Lange-Brokaar BJE, Bijsterbosch J, Kornaat PR. et al. Radiographic progression of knee osteoarthritis is associated with MRI abnormalities in both the patellofemoral and tibiofemoral joint. Osteoarthritis Cartilage 2016; 24 (03) 473-479
    CrossrefPubMedGoogle Scholar
  • 12 Bayramoglu N, Nieminen MT, Saarakkala S. Automated detection of patellofemoral osteoarthritis from knee lateral view radiographs using deep learning: data from the Multicenter Osteoarthritis Study (MOST). Osteoarthritis Cartilage 2021; 29 (10) 1432-1447
    CrossrefPubMedGoogle Scholar
  • 13 Bayramoglu N, Nieminen MT, Saarakkala S. Machine learning based texture analysis of patella from X-rays for detecting patellofemoral osteoarthritis. Int J Med Inform 2022; 157: 104627
    CrossrefPubMedGoogle Scholar
  • 14 Lindner C, Thiagarajah S, Wilkinson JM, Wallis GA, Cootes TF. arcOGEN Consortium. Fully automatic segmentation of the proximal femur using random forest regression voting. IEEE Trans Med Imaging 2013; 32 (08) 1462-1472
    CrossrefPubMedGoogle Scholar
  • 15 Friedman JH. Greedy function approximation: a gradient boosting machine. Ann Statist 2001; 29: 1189-1232
    CrossrefPubMedGoogle Scholar
  • 16 Roemer FW, Guermazi A, Hunter DJ. et al. The association of meniscal damage with joint effusion in persons without radiographic osteoarthritis: the Framingham and MOST osteoarthritis studies. Osteoarthritis Cartilage 2009; 17 (06) 748-753
    CrossrefPubMedGoogle Scholar
  • 17 Yan Y, Kawahara J, Hamarneh G. Melanoma recognition via visual attention. In: Information Processing in Medical Imaging. Paper presented at: 26th International Conference, IPMI 2019, June 2–7, 2019, Hong Kong, China. Proceedings 26. Springer; 2019:793–804
    PubMedGoogle Scholar
  • 18 Simonyan K, Zisserman A. Very deep convolutional networks for large-scale image recognition arXiv preprint. arXiv 1409.1556 2014
    PubMedGoogle Scholar
  • 19 He K, Zhang X, Ren S, Sun J. Delving deep into rectifiers: Surpassing human-level performance on imageNet classification. Paper presented at: Proceedings of the IEEE International Conference on Computer Vision; 2015:1026–1034
    PubMedGoogle Scholar
  • 20 Lin TY, Goyal P, Girshick R, He K, Doll'ar P. Focal loss for dense object detection. Paper presented at: Proceedings of the IEEE International Conference on Computer Vision; 2017:2980–2988
    PubMedGoogle Scholar
  • 21 Selvaraju RR, Cogswell M, Das A, Vedantam R, Parikh D, Batra D. Grad-cam: Visual explanations from deep networks via gradient-based localization. Paper presented at: Proceedings of the IEEE International Conference on Computer Vision 2017:618–626
    PubMedGoogle Scholar
  • 22 Vaswani A, Shazeer N, Parmar N. et al. Attention is all you need. Adv Neural Inf Process Syst 2017; 30
    PubMedGoogle Scholar
  • 23 Ke G, Meng Q, Finley T. et al. LightGBM: A highly efficient gradient boosting decision tree. In: Advances in Neural Information Processing Systems. 2017: 3146-3154
    Google Scholar
  • 24 Brier GW. Verification of forecasts expressed in terms of probability. Mon Weather Rev 1950; 78: 1-3
    CrossrefPubMedGoogle Scholar
  • 25 DeLong ER, DeLong DM, Clarke-Pearson DL. Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach. Biometrics 1988; 44 (03) 837-845
    CrossrefPubMedGoogle Scholar
  • 26 Lundberg SM, Lee S-I. A unified approach to interpreting model predictions. In: Advances in Neural Information Processing Systems 30. 2017: 4765-4774
    Google Scholar
  • 27 Tiulpin A, Klein S, Bierma-Zeinstra SMA. et al. Multimodal machine learning-based knee osteoarthritis progression prediction from plain radiographs and clinical data. Sci Rep 2019; 9 (01) 20038
    CrossrefPubMedGoogle Scholar
  • 28 Niu Z, Zhong G, Yu H. A review on the attention mechanism of deep learning. Neurocomputing 2021; 452: 48-62
    CrossrefPubMedGoogle Scholar