Download PDF
Methods Inf Med 2006; 45(05): 536-540
DOI: 10.1055/s-0038-1634115
Original Article
Schattauer GmbH

Two Models for Outcome Prediction

A Comparison of Logistic Regression and Neural Networks
R. Linder   *
1   Institut für Medizinische Informatik, Universität zu Lübeck, Lübeck, Germany
,
I. R. König   *
2   Institut für Medizinische Biometrie und Statistik, Universität zu Lübeck, Lübeck, Germany
,
C. Weimar
3   Klinik für Neurologie, Universitätsklinikum Essen, Universität Duisberg-Essen, Germany
,
H. C. Diener
3   Klinik für Neurologie, Universitätsklinikum Essen, Universität Duisberg-Essen, Germany
,
S. J. Pöppl
1   Institut für Medizinische Informatik, Universität zu Lübeck, Lübeck, Germany
,
A. Ziegler
2   Institut für Medizinische Biometrie und Statistik, Universität zu Lübeck, Lübeck, Germany
› Author Affiliations
Further Information

Publication History

Received: 07 April 2005

accepted: 05 January 2006

Publication Date:
07 February 2018 (online)

    Permissions and Reprints

Summary

Objectives: Accurately predicting disease progress from a set of predictive variables is an important aspect of clinical work. For binary outcomes, the classical approach is to develop prognostic logistic regression (LR) models. Alternatively, machine learning algorithms were proposed with artificial neural networks (ANN) having become popular over the last decades. Although some studies have compared predictive accuracies of LR and ANN models, some concerns regarding their methodological quality have been voiced. Our comparison has the advantage of being based on two large independent data sets allowing for elaborate model development and independent validation.

Methods: From the German Stroke Database, a learning data set including 1754 prospectively recruited patients with acute ischemic stroke was used. Utilizing LR and ANN, two prognostic models were developed predicting restitution of functional independence and survival after 100 days. The resulting models were applied to classify 1470 patients with acute ischemic stroke; this test data set was collected independently from the learning data. Error fractions in the test data were determined, and differences in error fractions between the algorithms were calculated with 95% confidence intervals.

Results: For most prognostic models, error fractions in the test data were below 40%. There was no difference between the algorithms except for the model predicting completely versus incompletely restituted or deceased patients (difference in error fractions = 4.01% [2.10-5.96%], p = 0.0001).

Conclusions: The conscientiously applied LR remains the gold standard for prognostic modelling; however, ANN can be an alternative automated “quick and easy” multivariate analysis.

Keywords

Artificial neural network - regression - benchmarking - prognosis - stroke

* Both authors contributed equally to the work.


 

  • References

  • 1 Royston P. A strategy for modelling the effect of a continuous covariate in medicine and epidemiology. Stat Med 2000; 19: 1831-47.
    CrossrefPubMedSearch in Google Scholar
  • 2 Harrell FE, Lee KL, Mark DB. Multivariable prognostic models: issues in developing models, evaluating assumptions and adequacy, and measuring and reducing errors. Stat Med 1996; 15: 361-87.
    CrossrefPubMedSearch in Google Scholar
  • 3 Tu J. Advantages and disadvantages of using artificial neural networks versus logistic regression for predicting medical outcomes. J Clin Epidemiol 1996; 49: 1225-31.
    CrossrefPubMedSearch in Google Scholar
  • 4 Heckerling PS, Gerber BS, Tape TG, Wigton RS. Selection of predictor variables for pneumonia using neural networks and genetic algorithms. Methods Inf Med 2005; 44: 89-97.
    Thieme ConnectPubMedSearch in Google Scholar
  • 5 Seuchter SA, Eisenacher M, Riesbeck M, Gaebel W, Köpcke W. Methods for predictor analysis of repeated measurements: application of psychiatric data. Methods Inf Med 2004; 43: 184-91.
    Thieme ConnectPubMedSearch in Google Scholar
  • 6 Wagenknecht G, Kaiser H-J. Buell U, Sabri O. MRI-based individual 3D region-of-interest atlases of the human brain: a new method for analyzing functional data. Methods Inf Med 2004; 43: 383-90.
    Thieme ConnectPubMedSearch in Google Scholar
  • 7 Zaknich A. Neural Networks for Intelligent Signal Processing. Singapore: World Scientific Publishing; 2003
    Search in Google Scholar
  • 8 Dreiseitl S, Ohno-Machado L. Logistic regression and artificial neural network classification models: a methodology review. J Biomed Inform 2002; 35: 352-9.
    CrossrefPubMedSearch in Google Scholar
  • 9 Schwarzer G, Vach W, Schumacher M. On the misuses of artificial neural networks for prognostic and diagnostic classification in oncology. Stat Med 2000; 19: 541-61.
    CrossrefPubMedSearch in Google Scholar
  • 10 Schumacher M, Olschewski M, Schwarzer G. Contributions of artificial neural networks to knowledge in clinical medicine - is there evidence for improvement? Biom J. 2004; 46: 39.
    PubMedSearch in Google Scholar
  • 11 Schumacher M, Roßner R, Vach W.. Neural networks and logistic regression: part I. Computa- tional Statistics & Data Analysis 1996; 21: 661-82.
    PubMedSearch in Google Scholar
  • 12 Vach W. Roßner R, Schumacher M. Neural networks and logistic regression: part II. Computational Statistics & Data Analysis 1996; 21: 683-701.
    CrossrefPubMedSearch in Google Scholar
  • 13 Benítez J, Castro J, Requena I. Are artificial neural networks black boxes?. IEEE Transactions on Neural Networks 1997; 8: 1156-64.
    CrossrefPubMedSearch in Google Scholar
  • 14 König IR, Weimar C, Diener HC, Ziegler A. Vorhersage des Funktionsstatus 100 Tage nach einem ischämischen Schlaganfall: Design einer prospektiven Studie zur externen Validierung eines prognostischen Modells. Z Arztl Fortbild Qualitatssich 2003; 97: 717-22.
    PubMedSearch in Google Scholar
  • 15 Weimar C, König IR, Kraywinkel K, Ziegler A, Diener HC. Age and the National Institutes of Health-Stroke Scale within 6 h after onset are accurate predictors of outcome after cerebral ischemia: Development and external validation of prognostic models. Stroke 2004; 35: 158-62.
    CrossrefPubMedSearch in Google Scholar
  • 16 Weimar C, Ziegler A, König IR, Diener HC. Predicting functional and vital outcome after acute ischemic stroke. J Neurol 2002; 249: 888-95.
    CrossrefPubMedSearch in Google Scholar
  • 17 Walker AE, Robins M, Weinfeld FD. The National Survey of Stroke. Clinical findings. Stroke 1981; 12: 113-44.
    PubMedSearch in Google Scholar
  • 18 Committee for Proprietary Medicinal Products (CPMP). Points to consider on clinical investigation of medicinal products for the treatment of acute stroke. The European Agency for the Evaluation of Medicinal Products. 2000 CPMP/EWP/560/98.
    PubMedSearch in Google Scholar
  • 19 Mahoney FI, Barthel DW. Functional evaluation: The Barthel Index. Md State Med J 1965; 14: 61-5.
    PubMedSearch in Google Scholar
  • 20 Roberts L, Counsell C. Assessment of clinical outcomes in acute stroke trials. Stroke 1998; 29: 986-91.
    CrossrefPubMedSearch in Google Scholar
  • 21 Schuntermann MF. The international classification of impairments, disabilities and handicaps (ICIDH) - results and problems. Int J Rehabil Res 1996; 19: 1-11.
    CrossrefPubMedSearch in Google Scholar
  • 22 The German Stroke Collaboration. Predicting outcome after acute ischemic stroke: An external validation of prognostic models. Neurology 2004; 62: 581-5.
    CrossrefPubMedSearch in Google Scholar
  • 23 Tango T. Re: Improved confidence intervals for the difference between binomial proportions based on paired data by Robert G. Newcombe, Statistics in Medicine 1998 17 2635-50. Stat Med 1999; 18 3511 3.
    PubMedSearch in Google Scholar
  • 24 Linder R, Pöppl S. ACMD: A practical tool for automatic neural net based learning. Lect Notes Comp Sciences 2001; 2199: 168-73.
    PubMedSearch in Google Scholar
  • 25 Dybowski R.. Clinical Applications of Artificial Neural Networks: Cambridge University Press; 2001
  • 26 Malmgren H, Borga M, Niklasson L. Artificial Neural Networks in Medicine and Biology. Proceedings of the Animab-1 Conference 2000.. Telos: Springer; 2000
    Search in Google Scholar
  • 27 Geddes C, Fox J, Allison M, Boulton-Jones J, Simpson K. An artificial neural network can select patients at high risk of developing progressive IgA nephropathy more accurately than experienced nephrologists. Nephrol Dial Transplant 1998; 13: 67-71.
    CrossrefPubMedSearch in Google Scholar
  • 28 Geman S, Bienenstock E, Doursat R. Neural networks and the bias/variance dilemma. Neural Comp 1992; 4: 1-58.
    CrossrefPubMedSearch in Google Scholar
  • 29 Bishop C. Neural networks for pattern recognition. Oxford: Clarendon Press; 1995
    Search in Google Scholar
  • 30 Linder R, Wirtz S, Pöppl S. Speeding up backpropagation learning by the APROP algorithm. In: ICSC Symposia on Neural Computation; May 23-26, 2000. Berlin, Germany 2000
    PubMedSearch in Google Scholar
  • 31 Rumelhart D, Hinton G, Williams R. Learning representations by back-propagating errors. Nature 1986; 323: 533-6.
    CrossrefPubMedSearch in Google Scholar
  • 32 Royston P, Altman DG. Regression using fractional polynomials of continuous covariates: Parsimonious parametric modelling. Appl Stat 1994; 43: 429-67.
    CrossrefPubMedSearch in Google Scholar
  • 33 Agresti A. Categorical Data Analysis. New York: Wiley; 1990
    Search in Google Scholar
  • 34 Brott T, Adams Jr. HP, Olinger CP, Marler JR, Barsan WG, Biller J. et al Measurements of acute cerebral infarction: a clinical examination scale. Stroke 1989; 20: 864-70.
    CrossrefPubMedSearch in Google Scholar
  • 35 Arminger G, Enache D, Bonne T. Analyzing credit risk data: a comparison of logistic discrimination, classification tree analysis, and feedforward networks. Computational Statistics & Data Analysis 1997; 12: 293-310.
    PubMedSearch in Google Scholar
  • 36 Royston P, Sauerbrei W. Stability of multivariable fractional polynomial models with selection of variables and transformations: a bootstrap investigation. Stat Med 2003; 22: 639-59.
    CrossrefPubMedSearch in Google Scholar
  • 37 Royston P, Sauerbrei W. A new approach to modelling interactions between treatment and continuous covariates in clinical trials by using fractional polynomials. Stat Med 2004; 23: 2509-25.
    CrossrefPubMedSearch in Google Scholar
  • 38 Hand D. Academic obsessions and classification realities: ignoring practicalities in supervised classification. In Banks D, House L, McMorris F, Arabie P, Gaul W. editors Classification, Clustering and Data Mining Applications. Springer 2004
    PubMedSearch in Google Scholar