On Prognostic Models, Artificial Intelligence and Censored Observations

Abstract

The development of prognostic models for assisting medical practitioners with decision making is not a trivial task. Models need to possess a number of desirable characteristics and few, if any, current modelling approaches based on statistical or artificial intelligence can produce models that display all these characteristics. The inability of modelling techniques to provide truly useful models has led to interest in these models being purely academic in nature. This in turn has resulted in only a very small percentage of models that have been developed being deployed in practice. On the other hand, new modelling paradigms are being proposed continuously within the machine learning and statistical community and claims, often based on inadequate evaluation, being made on their superiority over traditional modelling methods. We believe that for new modelling approaches to deliver true net benefits over traditional techniques, an evaluation centric approach to their development is essential. In this paper we present such an evaluation centric approach to developing extensions to the basic k-nearest neighbour (k-NN) paradigm. We use standard statistical techniques to enhance the distance metric used and a framework based on evidence theory to obtain a prediction for the target example from the outcome of the retrieved exemplars. We refer to this new k-NN algorithm as Censored k-NN (Ck-NN). This reflects the enhancements made to k-NN that are aimed at providing a means for handling censored observations within k-NN.

Keywords

Colorectal Cancer - Survival Analysis - Nearest Neighbour - Censored Observations

• References

• 1 Anand SS, Bell DA, Hughes JG. A General Framework for Database Mining based on Evidential Theory. Data and Knowledge Eng J 1996; 18: 189-223.
• 2 Anand SS, Hughes JG. Hybrid Data Mining Systems, Proceedings of the 2nd Pacific Asia Conference on Knowledge Discovery and Data Mining. 1998: 13-24.
• 3 Anand SS, Smith AE, Hamilton P, Anand JS, Hughes JG, Bartel P. An Evaluation of Intelligent Prognostic Systems for Colorectal Cancer. Artif Intell Med 1999; 15 (Suppl. 02) 193-214.
• 4 Collett D. Modelling Survival Data in Medical Research. Chapman and Hall; 1994
• 5 Faraggi D, Simon R. A Neural Network Model for Survival Data. Stat Med 1995; 14: 73-82.
• 6 Kasif S, Salzberg S, Waltz D, Rachlin J, Aha D. A Probabilistic Framework for Memory-Based Reasoning. Artif Intell 1998; 104 1-2 297-312.
• 7 LavraȈc N. Data Mining in Medicine: Selected Techniques and Applications, Proceedings of the Second International Conference on the Practical Applications of Knowledge Discovery and Data Mining. London: 1998: 11-31.
• 8 Wyatt J. Nervous about Artificial Neural Networks. Lancet 1995; 346: 1175-7.
• 9 Shafer G. A Mathematical Theory of Evidence. Prinston New Jersey: Prinston University Press; 1976
• 10 Guan JW, Bell DA. Evidence Theory and its Applications. Amsterdam: North-Holland; 1991. vol.1
• 11 Breiman L, Friedman JH, Olshen RA, Stone CJ. Classification and Regression Trees. Chapman and Hall; 1990
• 12 Ahn H, Loh W-Y. Tree-Structured Proportional Hazards Regression Modelling. Biometrics 1994; 50: 471-85.
• 13 LeBlanc M, Crowley J. Survival Trees by Goodness of Split. J Am Stat Assoc; 1993: 88.
• 14 Segal M. Regression Trees for Censored Data. Biometrics 1988; 44: 35-48.