Extracting Findings from Narrative Reports: Software Transferability and Sources of Physician Disagreement

G. Hripcsak; G.J. Kuperman; C. Friedman

doi:10.1055/s-0038-1634499

Methods of Information in Medicine, Table of Contents

Methods Inf Med 1998; 37(01): 01-07
DOI: 10.1055/s-0038-1634499

Original Article

Schattauer GmbH

Extracting Findings from Narrative Reports: Software Transferability and Sources of Physician Disagreement

G. Hripcsak

¹Department of Medical Informatics, Columbia University, New York, USA

,

G.J. Kuperman

²Department of Information Systems, Brigham and Women's Hospital, Boston, USA

,

C. Friedman

³Department of Computer Science, Queens College, New York, USA

¹Department of Medical Informatics, Columbia University, New York, USA

› Author Affiliations

Abstract

Abstract:

While natural language processing systems are beginning to see clinical use, it remains unclear whether they can be disseminated effectively through the health care community. MedLEE, a general-purpose natural language processor developed for Columbia-Presbyterian Medical Center, was compared to physicians' ability to detect seven clinical conditions in 200 Brigham and Women's Hospital chest radiograph reports. Using the system on the new institution's reports resulted in a small but measurable drop in performance (it was distinguishable from physicians at p = 0.011). By making adjustments to the interpretation of the processor's coded output (without changing the processor itself), local behavior was better accommodated, and performance improved so that it was indistinguishable from the physicians. Pairs of physicians disagreed on at least one condition for 22% of reports; the source of disagreement appeared to be interpretation of findings, gauging likelihood and degree of disease, and coding errors.

Keywords

Natural Language Processing - Computerized Medical Record Systems - Thoracic Radiography; Evaluation

Full Text

References

REFERENCES
1 Spyns P. Natural language processing in medicine: an overview. Meth Inform Med 1996; 35: 285-301.
2 Friedman C, Alderson PO, Austin JHM, Cimino JJ, Johnson SB. A general natural-language text processor for clinical radiology. Amer Med Inform Ass 1994; 1: 161-74.
3 Sager N, Lyman M, Bucknall C, Nhan N, Tick LJ. Natural language processing and the representation of clinical data. Amer Med Inform Ass 1994; 1: 142-60.
4 Baud RH, Rassinoux AM, Scherrer JR. Natural language processing and semantical representation of medical texts. Meth Inform Med 1992; 31: 117-25.
5 Rassinoux AM, Michel PA, Juge C, Baud R, Scherrer JR. Natural language processing of medical texts within the HELIOS environraent. Comput Meth Progr Biomed 1994; 45 (Suppl) S79-S96.
6 Zweigenbaum P. MENELAS: an access system for medical records using natural language. Comput Meth Progr Biomed 1994; 45: 117-20.
7 Haug PJ, Ranum DL, Frederick PR. Computerized extraction of coded findings from free-text radiologic reports: work in progress. Radiology 1990; 174: 543-8.
8 Gundersen ML, Haug PJ, Pryor TA, et al. Development and evaluation of a computerized admission diagnoses encoding system. Comput Biomed Res 1996; 29: 351-72.
9 Zingmond D, Lenert LA. Monitoring freetext data using medical language processing. Comput Biomed Res 1993; 26: 467-81.
10 Lenert LA, Tovar M. Automated linkage of free-text descriptions of patients with a practice guideline. Proceedings of the Annual Symposium on Computer Applications in Medical Care 1993; 274-8.
11 Gabrieli ER. Computer-assisted assessment of patient care in the hospital. J Med Systems 1988; 12: 135-46.
12 Hripcsak G, Friedman C, Alderson PO, Du-Mouchel W, Johnson SB, Clayton PD. Unlocking clinical data from narrative reports: a study of natural language processing. Ann Intern Med 1995; 122: 681-8.
13 Friedman C, Hripcsak G, DuMouchel W, Johnson SB, Clayton PD. Natural language processing in an operational clinical information system. Natural Language Engin 1995; 1: 83-108.
14 Jain NL, Knirsch C, Friedman C, Hripcsak G. Identification of suspected tuberculosis patients based on natural language processing of chest radiograph reports. Proceedings of the Annual Symposium on Computer Applications in Medical Care 1996; 542-6.
15 Knirsch CA, Jain NL, Pablos-Mendez A, Friedman C, Hripcsak G. Respiratory isolation of tuberculosis patients using clinical guidelines and an automated clinical decision support system. Infect Control Hosp Epidemiol. 1998 (in press).
16 Teich JM, Spurr CD, Flammini SJ, et al. Response to a trial of physician-based inpatient order entry. Proceedings of the Annual Symposium on Computer Applications in Medical Care 1993; 316-20.
17 Hripcsak G, Ludemann P, Pryor TA, Wigertz OB, Clayton PD. Rationale for the Arden Syntax. Comput Biomed Res 1994; 27: 291-324.
18 Efron B, Tibshirani RJ. An Introduction to the Bootstrap. New York: Chapman & Hall; 1993
19 DuMouchel W, Friedman C, Hripcsak G, Johnson SB, Clayton PD. Two applications of statistical modelling to natural language processing. In: AI and Statistics V. Fisher D, Lenz H. eds. New York: Springer-Verlag,; 1996: 413-21.
20 Flanagan JC. The critical incident technique. Psychol Bull 1954; 51: 327-58.
21 Bundens WP, Bergan JJ, Halasz NA, Murray J, Drehobl M. The superficial femoral vein: a potentially lethal misnomer. JAMA 1995; 274: 1296-8.
22 Yerushalmy J. The statistical assessment of the variability in observer perception and description of roentgenographic pulmonary shadows. Radiologic Clinics of North America 1969; 7: 381-92.
23 Jollis JG, Ancukiewicz M, DeLong ER, Pryor DB, Muhlbaier LH, Mark DB. Discordance of databases designed for claims payment versus clinical information systems: implications for outcomes research. Ann Intern Med 1993; 119: 844-50.