Download PDF
Methods Inf Med 1978; 17(04): 217-226
DOI: 10.1055/s-0038-1636441
Original Article
Schattauer GmbH

The Measurement of Performance in Probabilistic Diagnosis

I. The Problem, Descriptive Tools, and Measures Based on Classification MatricesDIE LEISTUNGSMESSUNG BEI DER WAHRSCHEINLICHKEITSDIAGNOSEI. DAS PROBLEM, DESKRIPTIVE VERFAHREN UND AUF KLASSIFIKATIONSMATRIZEN GEGRÜNDETE MESSGRÖSSEN
J. D. F. Habbbma
1   From the Department of Publio Health and Social Medicine, Erasmus University, Rotterdam, The Netherlands, and the Institute of Human Genetics, University of Copenhagen, Denmark
,
J. Hilden
1   From the Department of Publio Health and Social Medicine, Erasmus University, Rotterdam, The Netherlands, and the Institute of Human Genetics, University of Copenhagen, Denmark
,
B. Bjerregaard
1   From the Department of Publio Health and Social Medicine, Erasmus University, Rotterdam, The Netherlands, and the Institute of Human Genetics, University of Copenhagen, Denmark
› Author Affiliations
Further Information

Publication History

Publication Date:
19 February 2018 (online)

    Permissions and Reprints

Owing to the inherent uncertainty of diagnostic tasks, diagnostic advice should be offered in a probabilistic, rather than deterministic form. Since the late fifties a lot of effort has been invested in constructing probabilistic diagnosis rules. Much less has been done to devise rational tools for evaluating them; conventional error rates reflect but one aspect of performance in a rather crude way. The aim of this paper and its successors is to offer a body of evaluation tools. After defining a general framework and stating its limitations we apply some graphical techniques to the acute abdominal pain data that will serve as illustration in the next papers as well: dot diagrams, the triangular diagram for the three-disease case, and three tabular representations based 011 categorization of the probabilities, viz. the usual (forced) classification matrix, from which various classification rates are read off; the classification matrix with doubt, in which uncertain and confident diagnoses are distinguished; and the exclusion matrix, which spots diseases that are judged improbable. Together these matrices give a good first impression of the behaviour of a probabilistic diagnosis system. The outlined techniques of later papers are needed for a more complete analysis.

Aufgrund der inhärenten Unsicherheiten diagnostischer Aufgaben sollte diagnostische Beratung besser in probabilistischer als in deterministischer Form geliefert werden. Seit den späten fünfziger Jahren hat man sich sehr um die Erstellung probabilistischer Diagnostikregeln bemüht. Viel weniger ist getan worden, um rationelle Hilfsmittel zu ihrer Bewertimg ausfindig zu machen; die üblichen Felllerquoten reflektieren nur einen Aspekt der Leistimg, zudem in recht grober Weise. Das Ziel dieser Artikelreihe ist es, einen Satz von Bewertungsmethoden anzubieten. Nach Festlegung eines Begriffsrahmens und kurzer Diskussion seiner Grenzen wenden Mir einige graphische und tabellarische Techniken auf Daten an, die aus der Kopenhagener Studie über akute Bauchschmerzen stammen und der Artikelreihe als durchgehendes Beispiel dienen: Punktdiagramme, das Dreieck-Diagramm für den Fall von drei Krankheiten und drei tabellarische Darstellungen, che auf Kategorisierung cler Wahrscheinlichkeiten gegründet sind: che übliche (erzwungene) Klassifikationsmatrix, che Klassifikationsmatrix mit Verzichtsmöglichkeit, in cler unsichere und sichere Diagnosen getrennt gezählt werden, und die Ausschlußmatrix, in cler für unwahrscheinlich gehaltene Krankheiten aufgezählt werden. Zusammen geben diese Matrizen einen guten ersten Eindruck vom Verhalten eines probabilistischen Diagnosesystems. Für eine vollständigere Analyse werden allerdings die Techniken cler beiden folgenden Aufsätze benötigt.

Keywords

Computer-aided Diagnosis - Uncertainty in Diagnosis - Probabilistic Diagnosis - Forced Classification Matrix - Classification Matrix with Doubt - Exclusion Matrix - Triangular Representation

Schlüssel-Wörter

Computerunterstützte Diagnose - Unsicherheit bei cler Diagnose - probabilistische Diagnose - erzwungene Klassifikationsmatrix - Klassifikationsmatrix mit Verzichtsmöglichkeit - Ausschlußmatrix - Dreiecksdarstellung
 

  • References

  • 1 ANDERSON J. A., BOYLE J. A.. Computer Diagnosis: Statistical Aspects. Brit. med. Bull 24 ( 1968; ) 230-235.
    CrossrefPubMedSearch in Google Scholar
  • 2 BJERREGAARD B., BBYNITZ S., HOLST-CHRISTENSEN J., KALAJA E., LTJND-KRISTENSEN J., HIDDEN J., DE DOMBAL F. T., HOBBOCKS J. C.. Computer-aided Diagnosis of the Acute Abdomen : a System from Leeds Used on Copenhagen Patients. In [6] pp. 165-171.
    PubMed
  • 3 BJERREGAARD B., BBYNITZ S., HOLST-CHRISTENSEN J., JESS P., KALAJA B., KNTJTSSON B., LTJND-KRISTENSEN J.. Differentialdiagnostik hos Patienter med Akutte Mavesmerter. Ugeskr. Laeg 140 ( 1978; ) 830-834.
    PubMedSearch in Google Scholar
  • 4 CHOW C. K.. On Optimum Recognition Error and Reject Tradeoff. IEEE Trans. Inform. Theory IT-IS ( 1970; ) 41-41.
    PubMedSearch in Google Scholar
  • 5 DOCUMENTA GEIGY. Wissenschaftliche Tabellen. ( Basle: J. R. Geigy, A. G; 1960. .).
    Search in Google Scholar
  • 6 DOMBAI F. T., de GREMY F.. (Eds) Decision Making and Medical Care. Can Information Science Help ?. ( Amsterdam: North Holland Publ. Co; 1976. .).
    Search in Google Scholar
  • 7 FBAGU P., PATOIS E., HUBER C., LELLOUCH J.. Original Approach of the Hyperthyroidism Diagnosis with Computer. In ANDEBSON J., FORSYTBE J. M.. (Eds) MEDINFO 74, Vol. 2. pp. 559-564 & 1068-1069 ( Amsterdam; North-Holland Publ. Co.: 1974. ).
    Search in Google Scholar
  • 8 HABBEMA J. D. F.. A Discriminant Analysis Approach to the Identification of Human Chromosomes. Biometrics 32 ( 1976; ) 919-928.
    CrossrefPubMedSearch in Google Scholar
  • 9 HABBEMA J. D. F.. Models for Diagnosis and Detection of Combinations of Diseases. In [6] pp. 399-410.
    PubMed
  • 10 HABBEMA J. D. F., HERMANS J., VAN DER BUBGT A. T.. Cases of Doubt in Allocation Problems. Biometrika 61 ( 1974; ) 313-324.
    CrossrefPubMedSearch in Google Scholar
  • 11 HABBEMA J. D. F., HERMANS J., REMME J.. Data Analytical Methods in Discriminant Analysis: the Analysis of Posterior Probabilities. In Data Analysis and Informatics. pp. 211-221. ( Rocquencourt; IRIA: 1977. .).
    Search in Google Scholar
  • 12 HABBEMA J. D. F., HILDEN J., BJERREGAARD B.. The Measurement of Performance in Probabilistic Diagnosis. IV. Measures Based on Utility Considerations and General Recommendations. Meth. Inform. Med. (Under preparation.)
    PubMed
  • 13 HERMANS J., EGGEBMONT J. J., HAGEDOOREN J., ODENTHAL D. W.. Probabilistic Differential Diagnosis of Auditory Dysfunction on the Basis of Eleetroeochleography. Meth. Inform. Med 14 ( 1975; ) 87-95.
    PubMedSearch in Google Scholar
  • 14 HILDEN J., BJERBEGAARD B.. Computer-aided Diagnosis and the Atypical Case. In [6] pp. 365-374.
    PubMed
  • 15 HILDEN J., HABBEMA J. D. F., BJERREGAABD B.. The Measurement of Performance in Probabilistic Diagnosis. II. Trustworthiness of the Exact Values of the Diagnostic Probabilities. Meth. Inform. Med 17 ( 1978; ) 227-237.
    PubMedSearch in Google Scholar
  • 16 HILDEN J., HABBEMA J. D. F., BJERREGAARD B.. The Measurement of Performance in Probabilistic Diagnosis. III.Methods Based on Continuous Functions of the Diagnostic Probabilities. Meth. Inform. Med 17 ( 1978; ) 238-246.
    PubMedSearch in Google Scholar
  • 17 JENNETT B., TEASDALE G., BRAAKMAN R., MINDERHOUD J., KNILL-JONES R.. Predicting Outcome in Individual Patients after Severe Head Injury. Lancet 1976; I: 1031-1034.
    PubMedSearch in Google Scholar
  • 18 J. BOY. COLL. PHYSICNS LONDON Special Issue on Statistical Methods for Clinical Decision Making. 9 ( 1975; ) 191-269.
    PubMedSearch in Google Scholar
  • 19 LACHENBRUCH P. A., MIOKEY M. R.. Estimation of Error Rates in Discriminant Analysis. Technometries 10 ( 1968; ) 1-11.
    CrossrefPubMedSearch in Google Scholar
  • 20 NEW ENGL. J. MED Issue on Decision Making in Health Care. 293 ( 1975; ) 211-244.
    CrossrefPubMedSearch in Google Scholar
  • 21 TOBONTO A. F., VEASY G., WARNER II. R.. Evaluation of a Computer Program for Diagnosis of Congenital Heart Disease. Progr. Cardiov. Dis 5 ( 1963; ) 362-377.
    CrossrefPubMedSearch in Google Scholar
  • 22 VECCHIO T. J.. Predictive Value of a Single Diagnostic Test in Unselected Populations. New Engl. J. Med 274 ( 1966; ) 1171-1173.
    CrossrefPubMedSearch in Google Scholar
  • 23 WAGNER G., TATJTU P., WOLBER U.. Problems of Medical Diagnosis — A Bibliography. Meth. Inform. Med 17 ( 1978; ) 55-74.
    PubMedSearch in Google Scholar