Some Reflections on the Evaluation of Expert Systems in Medicine

Summary

Several examples are given to underline the difficulties of evaluating expert systems in order to demonstrate that in the medical domain in particular their routine applications have not yet been developed to a great extent.

Anhand mehrerer Beispiele werden die Schwierigkeiten unterstrichen, die bei der Bewertung von Expertensystemen auftreten; es wird gezeigt, daß insbesondere im Bereich der Medizin deren Routineanwendung noch nicht sehr weit entwickelt ist.

Key-Words:

Expert Systems - Artificial Intelligence - Evaluation

Schlüssel-Wörter:

Expertensysteme - künstliche Intelligenz - Auswertung

• REFERENCES

• 1 Aikins J. S. Prototypes and Production Rules: A Knowledge Representation for Computer Consultations. Technical Report Memo HPP 80-17. Stanford Heuristic Programming Project, August. 1980
• 2 Buchanan B. G. Research on Expert Systems. Technical Report HPP 81-1. Stanford Heuristic Programming Project, February. 1981
• 3 Clancey W. J. The Epistemology of a Rule Based Expert System: A Framework for Explanation. Art. Intell 20 1983; 215-251.
• 4 Davis R, Lenat D. Knowledge Based Systems in Artificial Intelligence. New York: McGraw-Hill; 1982
• 5 Fagan L. M. VM: Representing Time Dependent Relations in a Medical Setting. PhD Thesis. Stanford University. 1980
• 6 Feigenbaum E, McCorduck P. The Fifth Generation: Artificial Intelligence and Japan's Challenge to the World. Reading, MA: Addison Wesley; 1983
• 7 Goldberger H. A. DAISY: A Model of Temporal Reasoning in Medicine. Proposal for PhD Thesis. Cambridge, MA: M. I. T, Artificial Intelligence Laboratory; 1982
• 8 Kulikowski C. A, Ostroff J. H. Constructing an Expert Knowledge Base for Thyroid Consultation Using Generalized A. I. Techniques. In Proc. 4th Ann. Symp. Comput. Appl. in Med. Care. New York: IEEE; 1980
• 9 Kulikowski C. A, Weiss S. M. Representation of Expert Knowledge for Consultation: The CASNET and EXPERT Projects. In Szolovits P. (Edit.) Artificial Intelligence in Medicine. (AAAS Selected Symposium, Vol.51). Boulder, CO: Westview Press; 1982
• 10 Lindberg D. A. B, Sharp G. C, Kings-land G, Weiss S. M, Hayes P, Ueno H, Hazelwood S. E. Computer Based Rheumatology Consultant. In Lindberg D. A. B, Kaihara S. (Eds) MEDIN-FO 80. Amsterdam: North-Holland; 1980: 1311-1315.
• 11 Miller R. A. INTERNIST-1/ CADUCEUS: Problems Facing Expert Consultant Programs. Meth. Inform. Med 23 1984; 9-14.
• 12 Patil R. S. Causal Representation of Patient Illness for Electrolyte and Acid Base Diagnosis. Technical Report TR 267. Cambridge, MA: M.I.T., Laboratory for Computer Science; 1981
• 13 Pauker S. G, Szolovits P. Analyzing and Simulating Taking the History of the Present Illness: Context Formation. In Schneider W, Sägwall Hein A.-L. (Eds) Computational Linguistics in Medicine. Amsterdam: North-Holland; 1977: 109-118.
• 14 Pople Jr H. E. DIALOG: A Model of Diagnosis Logic for Internal Medicine. In Proc. 4th I.J.C.A.I. Tbilissi: 1975
• 15 Sherman H. B. A Comparative Study of Computer Aided Clinical Diagnosis of Birth Defects. Technical Report M.I.T./. LCS/TR283. Cambridge, MA: M.I.T., Laboratory for Computer Science; 1981
• 16 Shortliffe E. H. Computer Based Medical Consultations: MYCIN. New York: Elsevier; 1976
• 17 Shortliffe E. H. Evaluating Expert Systems. Technical Report Memo HPP 81-9. Stanford Heuristic Programming Project. 1981
• 18 Shortliffe E. H, Scott A. C, Bischoff M. B, Campbell A. B, van Melle W, Jacobs C. D. An Expert System for Oncology Protocol Management. In Shortliffe E. H, Buchanan B. G. (Eds) Rule-Based Expert Systems. Reading, MA: Addison Wesley; 1984: 653-665.
• 19 Szolovits P. Remarks on Scoring. (Unpublished Course Notes).
• 20 van Melle W. A Domain Independent System that Aids in Constructing Knowl- edge Based Consultation Programs. Technical Report Memo HPP 80-22. Stanford Heuristic Programming Project. 1980
• 21 Weinstein M. C, Fineberg H. V, El-stein A. S, Frazier H. S, Neuhauser D, Neutra R, McNeil B. J. Clinical Decision Analysis. Philadelphia, PA: W. B. Saunders; 1980
• 22 Yu V. L, Buchanan B. G, Shortliffe E. H, Wraith S. M, Davis R, Scott A. C, Cohen S. N. Evaluating the Performance of a Computer-based Consultant. Comput. Progr. Biomed 9 1979; 95-102.
• 23 Zadeh L. A. A Theory of Approximate Reasoning (AR). In Hayes J. E. et al (Eds) Machine Intelligence: Machine Expertise and the Human Interface. (Machine Intelligence, Vol. 9). New York: Halsted Press; 1979