Modeling Length of Stay as an Optimized Two-class Prediction Problem

M. Verduijn; N. Peek; F. Voorbraak; E. de Jonge; B. A. J. M. de Mol

doi:10.1160/ME0368

Methods of Information in Medicine, Table of Contents

Methods Inf Med 2007; 46(03): 352-359
DOI: 10.1160/ME0368

paper

Schattauer GmbH

Modeling Length of Stay as an Optimized Two-class Prediction Problem

Authors

M. Verduijn

¹Department of Medical Informatics, Academic Medical Center (AMC), Amsterdam, The Netherlands

⁴Department of Biomedical Engineering, University of Technology, Eindhoven, The Netherlands
N. Peek

¹Department of Medical Informatics, Academic Medical Center (AMC), Amsterdam, The Netherlands
F. Voorbraak

¹Department of Medical Informatics, Academic Medical Center (AMC), Amsterdam, The Netherlands
E. de Jonge

²Department of Intensive Care Medicine, AMC, Amsterdam, The Netherlands
B. A. J. M. de Mol

³Department of Cardiothoracic Surgery, AMC, Amsterdam, The Netherlands

⁴Department of Biomedical Engineering, University of Technology, Eindhoven, The Netherlands

Abstract

Summary

Objectives: To develop a predictive model for the outcome length of stay at the Intensive Care Unit (ICU LOS), including the choice of an optimal dichotomization threshold for this outcome. Reduction of prediction problems of this type of outcome to a two-class problem is a common strategy to identify high-risk patients.

Methods: Threshold selection and model development are performed simultaneously. From the range of possible threshold values, the value is chosen for which the corresponding predictive model has maximal precision based on the data. To compare the precision of models for different dichotomizations of the outcome, the MALOR performance statistic is introduced. This statistic is insensitive to the prevalence of positive cases in a two-class prediction problem.

Results: The procedure is applied to data from cardiac surgery patients to dichotomize the outcome ICU LOS. The class probabilitytree method is used to develop predictive models. Within our data, the best model precision is found at the threshold of seven days.

Conclusions: The presented method extends existing procedures for predictive modeling with optimization of the outcome definition for predictive purposes. The method can be applied to all prediction problems where the outcome variable needs to be dichotomized, and is insensitive to changes in the prevalence of positive cases with different dichotomization thresholds.

Keywords

Predictive modeling - dichotomization - model precision - ICU length of stay

Full Text

References

References
1 Marcin JP, Slonim AD, Pollack MM, Ruttimann UE. Long-stay patients in the pediatric intensive care unit. Critical Care Medicine 2001; 29: 652-57.
2 Lawrence DR, Valencia O, Smith EEJ, Murday A, Treasure T. Parsonnet score is a good predictor of the duration of intensive care unit stay following cardiac surgery. Heart 2000; 83: 429-32.
3 Tu JV, Jaglal SB, Naylor CD. the Steering Committee of the Provincial Adult Cardiac Care Network of Ontario. Multicenter validation of a risk index for mortality intensive care unit stay, and overall hospital length of stay after cardiac surgery. Circulation 1995; 91: 677-84.
4 Hugot P, Sicsic J, Schaffuser A, Sellin M, Corbineau H, Chaperon J, Ecoffey C. Base deficit in immediate postoperative period of coronary surgery with cardiopulmonary bypass and length of stay in intensive care unit. Intensive Care Medicine 2003; 29: 257-61.
5 Janssen DPB, Noyez L, Wouters C, Brouwer RMHJ. Preoperative prediction of prolonged stay in the intensive care unit for coronary bypass surgery. European Journal of Cardio-Thoracic Surgery 2004; 25: 203-7.
6 Vargas Hein O, Birnbaum J, Wernecke K, England M, Konertz W, Spies C. Prolonged intensive care unit stay in cardiac surgery: risk factors and long-term-survival. Annals of Thoracic Surgery 2006; 81: 880-5.
7 Stein PK, Schmieg RE, El-Fouly A, Domitrovich PP, Buchman TG. Association between heart rate variability recorded on postoperative day 1 and length of stay in abdominal aortic surgery patients. Critical Care Medicine 2001; 29: 1738-43.
8 Brier GW. Verification of forecasts expressed in terms of probabilities. Monthly Weather Review 1950; 78: 1-3.
9 Christakis GT, Fremes SE, Naylor CD, Chen E, Rao V, Goldman BS. Impact of preoperative risk and perioperative morbidity on ICU stay following coronary bypass surgery. Cardiovascular Surgery 1996; 04: 29-35.
10 Bashour CA, Yared J, Ryan TA, Rady MY, Mascha E, Leventhal MJ, Starr NJ. Long-term survival and functional capacity in cardiac surgery patients after prolonged intensive care. Critical Care Medicine 2000; 28: 3847-53.
11 Hand DJ. Construction and Assessment of Classification Rules.. New York: John Wiley & Sons; 1997
12 Breiman L, Friedman JH, Olshen RA, Stone CJ. Classification and Regression Trees.. Monterey: Wadsworth & Brooks; 1984
13 Breiman L. Bagging predictors. Machine Learning 1996; 26: 123-40.
14 Efron B, Tibshirani R. An Introduction to the Bootstrap.. London: Chapman and Hall; 1993
15 Hastie T, Tibshirani R, Friedman J. The Elements of Statistical Learning.. Berlin: Springer; 2001
16 Therneau TM, Atkinson EJ. An introduction to recursive partitioning using the Rpart routines. Technical report. Mayo Foundation; 1997
17 Fleming TR, Harrington DP. Counting Processes and Survival Analysis.. New Work: John Wiley & Sons; 1991
18 Cox DR. Regression models and life-tables. Journal of the Royal Statistical Society 1972; B 34: 187-220.
19 Keleș S, Segal MR. Residual-based tree-structured survival analysis. Statistics in Medicine 2002; 21: 313-26.
20 Ohno-Machado L. Modeling medical prognosis: survival analysis techniques. Journal of Biomedical Informatics 2001; 34: 428-39.
21 Zupan B, Demšar J, Kattan MW, Beck JR, Bratko I. Machine learning for survival analysis: a case study on recurrence of prostate cancer. Artificial Intelligence in Medicine 2000; 20: 59-75.
22 Anand SS, Hamilton PW, Hughes JG, Bell DA. On prognostic models, artificial intelligence and censored observations. Methods of Information in Medicine 2001; 40: 18-24.
23 Metz CE. Basic principles of ROC analysis. Seminars in Nuclear Medicine 1978; 08: 283-98.
24 Ash A, Shwartz M. R2: auseful measure of model performance when predicting a dichotomous outcome. Statistics in Medicine 1999; 18: 375-384.
25 Holländer N, Augustin NH, Sauerbrei W. Investigation on the improvement of prediction by bootstrap model averaging. Methods Inf Med 2006; 45: 44-50.
26 Wyatt J, Altman DG. Prognostic models: clinically useful or quickly forgotten?. British Medical Journal 1995; 311: 1539-41.
27 Hosmer DW, Lemeshow S. Goodness-of-fit tests for the multiple logistic regression model. Communications in Statistics-Theory and Methods 1980; 09: 1043-69.
28 Hosmer DW, Hosmer T, Cessie SL, Lemeshow S. A comparison of goodness-of-fit tests for the logistic regression model. Statistics in Medicine 1997; 16: 965-80.