The Impact of an Explanation Capability for a Computer Consultation System

H. P. Erdman

doi:10.1055/s-0038-1635377

Methods of Information in Medicine, Table of Contents

Methods Inf Med 1985; 24(04): 181-191
DOI: 10.1055/s-0038-1635377

Original Article

Schattauer GmbH

The Impact of an Explanation Capability for a Computer Consultation System

Die Bedeutung der Fähigkeit zur Kommentierung für ein Computerberatungssystem

Authors

H. P. Erdman

¹(From the Psychiatric Computing Laboratory, Department of Psychiatry, University of Wisconsin, Madison, USA)

Abstract

The impact of a computer consultation program including explanations for its decisions was examined. The consultation program was designed as a decision aid for non-psychiatrists regarding the treatment of depression. Both psychiatrists and non-psychiatrists were used to evaluate the system, with different results for the two groups. Explanations improved the non-psychiatrists’ attitudes towards the computer recommendations, while for psychiatrists the impact of explanation depended on the specific content of the recommendations and explanations. The results for psychiatrists were consistent with the view that explanations are important for computer consultation systems because they may help physicians decide whether or not the computer’s advice is reasonable.

Die Bedeutung eines Computerberatungsprogramms mit Erklärungen für seine Entscheidungen wurde untersucht. Das Konsultationsprogramm wurde als Entscheidurigshilfe für Nicht-psychiater bei der Behandlung von Depressionen erarbeitet. Sowohl Psychiater als auch Nichtpsychiater werteten das System aus – mit unterschiedlichen Ergebnissen in beiden Gruppen.

Erklärungen verbesserten die Einstellung der Nichtpsychiater gegenüber den Computerempfehlungen, während bei den Psychiatern das Gewicht des Kommentares von dem spezifischen Inhalt der Empfehlungen und Erklärungen abhängig war. Die Ergebnisse bei den Psychiatern bestätigten die Ansicht, daß Erklärungen von Bedeutung für Computerberatungssysteme sind, weil sie den Ärzten die Entscheidung darüber, ob der Rat des Computers vernünftig ist oder nicht, erleichtern können.

Key-Words:

Computer Consultation Systems - Expert Systems - Explanation - Medical Decision Making

Schlüssel-Wörter:

Computerberatungssysteme - Expertensysteme - Erklärung - ärztliche Entscheidungsfindung

Full Text

References

REFERENCES
1 Bar-Hillel Y. Is information retrieval approaching a crisis? Amer. Doc 1963; 14: 95-97.
2 Bleich H. L. Computer evaluation of acid-base disorders. J clin. Invest 1969; 48: 1689-1696.
3 Cohen J. Weighted kappa: nominal scale agreement with provision for scaled disagreement or partial credit. Psychol. Bull 1968; 70: 213-220.
4 Cronbach L. J. Coefficient alpha and the internal structure of tests. Psychometrika 1951; 16: 297-334.
5 D’Arcy C. Patterns in the delivery of psychiatric care in Saskatchewan 1971-1972 An overview of service sectors and patient volumes. Can. Psychiatr. Ass. J 1976; 21: 91
6 Dannenberg A. L, Shapiro A. R, Fries J. F. Enhancement of clinical predictive ability by computer consultation. Meth. Inform. Med 1979; 18: 10-14.
7 De Dombal F. T, Leaper D. J, Horrocks J. C, Staniland J. R, McCann A. P. Human and computer-aided diagnosis of abdominal pain: further report with emphasis on performance of clinicians. Brit. med. J 1974; I: 376-380.
8 Einhorn H. J, Hogarth R. M. Confidence in judgment: persistence of the illusion of validity. Psychol. Rev 1978; 28: 395-416.
9 Erdman H. P, Greist J. H, Klein M. H, Jefferson J. W, Olson W, Salinger R. Computer Consultation for Psychiatric Diagnosis. In O’Neill J. F. (Edit.) The Fourth Annual Symposium on Computer Applications in Medical Care. New York: IEEE; 1980
10 Fischhoff B. Hindsight Φ foresight: the effect of outcome knowledge on judgment under uncertainty. J. exper. Psychol 1975; 3: 288-299.
11 Fox J, Bardhan K. D, Barber D. C. Alternatives to Bayes? A quantitative comparison with rule-based diagnostic inference. Meth. Inform. Med 1980; 19: 210-215.
12 Fryback D. G, Erdman H. Prospects for calibration of physicians’ probabilistic judgments: Design of a feedback system Proceed. IEEE Internat. Conf. on Cybernetics and Society. New York: IEEE; 1979: 340-345 .
13 Fryback D. G, Thornbury J. R. Evaluation of a computerized Bayesian model for diagnosis of renal cyst vs tumor vs. normal variant from excretory urogram information. Invest. Radiol 1976; 11: 102-111.
14 Gillis J. S, Lipkin J. O, Moran T. J. Drug therapy decisions: a social judgment analysis. J. nerv. ment. Dis 1981; 169: 439-447.
15 Goldberg L. R. Man versus model of man: A rationale, plus some evidence, for a method of improving on clinical inferences. Psychol. Bull 1970; 73: 422-432.
16 Goldman L, Weinberg M, Weisberg M, Olshen R, Cook E. E, Sargent R. K, Lamas G. A, Dennis C, Wilson C, Deckelbaum L, Fineberg H, Stiratelli R. A computer derived protocol to aid in the diagnosis of emergency room patients with acute chest pain. New Engl. J. Med 1982; 307: 588-596.
17 Gorry G. A. Computer-assisted clinical decision making. Meth. Inform. Med 1973; 12: 45-51.
18 Gustafson D. H, Greist J. H, Stauss F. F, Erdman H. P, Laughren T. P. A probabilistic system for identifying suicide attempters. Comput. biomed. Res 1977; 10: 83-89.
19 Gustafson D. H, Kestly J. J, Greist J. H, Jensen N. M. Initial evaluation of a subjective Bayesian Diagnostic System. Health Serv. Res 1971; 6: 204-213.
20 Hoeper E. W, Nycz G. R, Cleary P. D, Regier D. A, Goldberg I. G. Estimated prevalence of RDC mental disorder in primary medical care. Int. J. ment. Hlth 1980; 8: 6-15.
21 Landis J. R, Koch G. G. The measurement of observer agreement for categorical data. Biometrics 1977; 33: 159-174.
22 Levin J. R, Marascuilo L. A. Type IV errors and interactions. Psychol. Bull 1972; 78: 368-374.
23 Lichtenstein S, Fischhoff B. Do those who know more also know more about what they know? Organizat. Behav. Human Perform 1977; 20: 159-183.
24 McDonald C. J. Protocol-based computer reminders, the quality of care and the non-perfectibility of man. New Engl. J. Med 1976; 295: 1351-1355.
25 Mesel E, Wirtschafter D. D, Carpenter J. T, Durant J. R, Henke C, Gray E. A. Clinical algorithms for cancer chemotherapy – systems for community-based consultant – extenders and oncology centers. Meth. Inform. Med 1976; 15: 168-173.
26 Miller R. A, Pople H. E, Myers J. D. Internist-I, an experimental computer-based diagnostic consultant for general internal medicine. New Engl. J. Med 1982; 307: 468-476.
27 Mumford E. On being a patient. In Simons R. C, Pardes H. (Eds) Understanding Human Behavior in Health and Illness. Baltimore: Williams and Williams; 1981
28 Nielsen A. C, Williams T. A. Depression in ambulatory medical patients. Arch. gen. Psychiat 1980; 37: 999-1004.
29 Pople H. E, Myers J. D, Miller R. A. DIALOG: A model of diagnostic logic for internal medicine Proceed. Fourth Internat. Joint Conf. Artif. Intell. Cambridge/Mass: MIT Art. Intell. Lab. Publ; 1975: 848-855.
30 Raju N. S. Note on two generalizations of coefficient alpha. Psychometrika 1979; 44: 347-349.
31 Regier D. A, Goldberg I. D, Taube C. A. The de facto US mental health services system. Arch. gen. Psychiat 1978; 35: 685-693.
32 Shortliffe E. H, Axline S. G, Buchanan B. G, Cohen S. N. Design considerations for a program to provide consultation in clinical therapeutics. Proceed. San Diego biomed. Symp 1974; 13: 311-319.
33 Shortliffe E. H, Buchanan B. G, Feigenbaum E. A. Knowledge engineering for medical decision making: A review of computer-based clinical decision aids. Proceed. IEEE 1979; 67: 1207-1224.
34 Shortliffe E. H, Davis R, Axline S. G, Buchanan B. G, Green C. C, Cohen S. N. Computer-based consultations in clinical therapeutics: Explanation and rule acquisition capabilities of the MYCIN system. Comput. biomed. Res 1975; 8: 303-320.
35 Spitzer R. L, Endicott J, Cohen J, Fleiss J. L. Constraints on the validity of computer diagnosis. Arch. gen. Psychiat 1974; 31: 197-203.
36 Swartout W. R. Explaining and justifying expert consulting programs. Proceed. Seventh Internat. Joint Conf. Artif. In-tell. Cambridge/Mass: MIT Art. Intell. Lab. Publ; 1981: S15-S22.
37 Taylor T. R, Aitchison J, McGirr E. M. Doctors as decision makers: A computer-assisted study of diagnosis as a cognitive skill. Brit. med. J 1971; III: 35-40.
38 Teach R. L, Shortliffe E. H. An analysis of physician attitudes regarding computer-based clinical consultation systems. Comput. biomed. Res 1981; 14: 542-558.
39 Tversky A, Kahneman D. Judgment under uncertainty: heuristics and biases. Science 1974; 185: 1124-1131.
40 Wallis J. W, Shortliffe E. H. Explanatory power for medical expert systems: studies in the representation of causal relationships for clinical consultation. Meth. Inform. Med 1982; 21: 127-136.
41 Wardle A, Wardle L. Computer-aided diagnosis. Meth. Inform. Med 1978; 17: 15-28.
42 Warner H. R, Olmsted C. M, Rutherford B. D. HELP – A program for medical decision-making. Comput. biomed. Res 1972; 5: 65-74.
43 Weiner J. L. BLAH, a system which explains its reasoning. Artif. Intell 1980; 15: 19-48.
44 Weiss S. M, Kulikowski C. A, Amarel S, Safir A. A model-based method for computer-aided medical decision making. Artif. Intell 1978; 11: 145-172.
45 Weiss S. M, Kulikowski C. A, Safir A. Glaucoma consultation by computer. Comput. biol. Med 1978; 5: 25-40.
46 Weissman M. M, Myers J. K, Thompson D. Depression and its treatment in a US urban community -1975-1976 Arch. gen. Psychiat 1981; 38: 417-421.
47 Whitbeck C, Brooks R. Criteria for evaluating a computer aid to clinical reasoning. J. Med. Philos 1983; 8: 51-65.
48 Winer B. J. Statistical principles in experimental design. New York: McGraw-Hill; 1971
49 Yu V. L, Fagan L. M, Wraith S. M, Clancey W. J, Scott A. C, Hannigan J, Blum R. L, Buchanan B. G, Cohen S. N. Antimicrobial selection by computer: A blinded evaluation by infectious disease experts. J. Amer. med. Ass 1979; 242: 1279-1282.