Download PDF
Methods Inf Med 1998; 37(01): 01-07
DOI: 10.1055/s-0038-1634499
Original Article
Schattauer GmbH

Extracting Findings from Narrative Reports: Software Transferability and Sources of Physician Disagreement

G. Hripcsak
1   Department of Medical Informatics, Columbia University, New York, USA
,
G.J. Kuperman
2   Department of Information Systems, Brigham and Women's Hospital, Boston, USA
,
C. Friedman
3   Department of Computer Science, Queens College, New York, USA
1   Department of Medical Informatics, Columbia University, New York, USA
› Author Affiliations
Further Information

Publication History

Publication Date:
07 February 2018 (online)

    Permissions and Reprints

Abstract:

While natural language processing systems are beginning to see clinical use, it remains unclear whether they can be disseminated effectively through the health care community. MedLEE, a general-purpose natural language processor developed for Columbia-Presbyterian Medical Center, was compared to physicians' ability to detect seven clinical conditions in 200 Brigham and Women's Hospital chest radiograph reports. Using the system on the new institution's reports resulted in a small but measurable drop in performance (it was distinguishable from physicians at p = 0.011). By making adjustments to the interpretation of the processor's coded output (without changing the processor itself), local behavior was better accommodated, and performance improved so that it was indistinguishable from the physicians. Pairs of physicians disagreed on at least one condition for 22% of reports; the source of disagreement appeared to be interpretation of findings, gauging likelihood and degree of disease, and coding errors.

Keywords

Natural Language Processing - Computerized Medical Record Systems - Thoracic Radiography; Evaluation
 

  • REFERENCES

  • 1 Spyns P. Natural language processing in medicine: an overview. Meth Inform Med 1996; 35: 285-301.
    PubMedSearch in Google Scholar
  • 2 Friedman C, Alderson PO, Austin JHM, Cimino JJ, Johnson SB. A general natural-language text processor for clinical radiology. Amer Med Inform Ass 1994; 1: 161-74.
    PubMedSearch in Google Scholar
  • 3 Sager N, Lyman M, Bucknall C, Nhan N, Tick LJ. Natural language processing and the representation of clinical data. Amer Med Inform Ass 1994; 1: 142-60.
    PubMedSearch in Google Scholar
  • 4 Baud RH, Rassinoux AM, Scherrer JR. Natural language processing and semantical representation of medical texts. Meth Inform Med 1992; 31: 117-25.
    PubMedSearch in Google Scholar
  • 5 Rassinoux AM, Michel PA, Juge C, Baud R, Scherrer JR. Natural language processing of medical texts within the HELIOS environraent. Comput Meth Progr Biomed 1994; 45 (Suppl) S79-S96.
    CrossrefPubMedSearch in Google Scholar
  • 6 Zweigenbaum P. MENELAS: an access system for medical records using natural language. Comput Meth Progr Biomed 1994; 45: 117-20.
    CrossrefPubMedSearch in Google Scholar
  • 7 Haug PJ, Ranum DL, Frederick PR. Computerized extraction of coded findings from free-text radiologic reports: work in progress. Radiology 1990; 174: 543-8.
    CrossrefPubMedSearch in Google Scholar
  • 8 Gundersen ML, Haug PJ, Pryor TA, et al. Development and evaluation of a computerized admission diagnoses encoding system. Comput Biomed Res 1996; 29: 351-72.
    CrossrefPubMedSearch in Google Scholar
  • 9 Zingmond D, Lenert LA. Monitoring freetext data using medical language processing. Comput Biomed Res 1993; 26: 467-81.
    CrossrefPubMedSearch in Google Scholar
  • 10 Lenert LA, Tovar M. Automated linkage of free-text descriptions of patients with a practice guideline. Proceedings of the Annual Symposium on Computer Applications in Medical Care 1993; 274-8.
    PubMedSearch in Google Scholar
  • 11 Gabrieli ER. Computer-assisted assessment of patient care in the hospital. J Med Systems 1988; 12: 135-46.
    CrossrefPubMedSearch in Google Scholar
  • 12 Hripcsak G, Friedman C, Alderson PO, Du-Mouchel W, Johnson SB, Clayton PD. Unlocking clinical data from narrative reports: a study of natural language processing. Ann Intern Med 1995; 122: 681-8.
    CrossrefPubMedSearch in Google Scholar
  • 13 Friedman C, Hripcsak G, DuMouchel W, Johnson SB, Clayton PD. Natural language processing in an operational clinical information system. Natural Language Engin 1995; 1: 83-108.
    PubMedSearch in Google Scholar
  • 14 Jain NL, Knirsch C, Friedman C, Hripcsak G. Identification of suspected tuberculosis patients based on natural language processing of chest radiograph reports. Proceedings of the Annual Symposium on Computer Applications in Medical Care 1996; 542-6.
    PubMedSearch in Google Scholar
  • 15 Knirsch CA, Jain NL, Pablos-Mendez A, Friedman C, Hripcsak G. Respiratory isolation of tuberculosis patients using clinical guidelines and an automated clinical decision support system. Infect Control Hosp Epidemiol. 1998 (in press).
    PubMedSearch in Google Scholar
  • 16 Teich JM, Spurr CD, Flammini SJ, et al. Response to a trial of physician-based inpatient order entry. Proceedings of the Annual Symposium on Computer Applications in Medical Care 1993; 316-20.
    PubMedSearch in Google Scholar
  • 17 Hripcsak G, Ludemann P, Pryor TA, Wigertz OB, Clayton PD. Rationale for the Arden Syntax. Comput Biomed Res 1994; 27: 291-324.
    CrossrefPubMedSearch in Google Scholar
  • 18 Efron B, Tibshirani RJ. An Introduction to the Bootstrap. New York: Chapman & Hall; 1993
  • 19 DuMouchel W, Friedman C, Hripcsak G, Johnson SB, Clayton PD. Two applications of statistical modelling to natural language processing. In: AI and Statistics V. Fisher D, Lenz H. eds. New York: Springer-Verlag,; 1996: 413-21.
    Search in Google Scholar
  • 20 Flanagan JC. The critical incident technique. Psychol Bull 1954; 51: 327-58.
    CrossrefPubMedSearch in Google Scholar
  • 21 Bundens WP, Bergan JJ, Halasz NA, Murray J, Drehobl M. The superficial femoral vein: a potentially lethal misnomer. JAMA 1995; 274: 1296-8.
    CrossrefPubMedSearch in Google Scholar
  • 22 Yerushalmy J. The statistical assessment of the variability in observer perception and description of roentgenographic pulmonary shadows. Radiologic Clinics of North America 1969; 7: 381-92.
    PubMedSearch in Google Scholar
  • 23 Jollis JG, Ancukiewicz M, DeLong ER, Pryor DB, Muhlbaier LH, Mark DB. Discordance of databases designed for claims payment versus clinical information systems: implications for outcomes research. Ann Intern Med 1993; 119: 844-50.
    CrossrefPubMedSearch in Google Scholar