Download PDF
Methods Inf Med 1998; 37(04/05): 477-490
DOI: 10.1055/s-0038-1634549
Original Article
Schattauer GmbH

Evaluation of a Semantic Data Model for Chest Radiology: Application of aNew Methodology

R. A. Rocha
1   Departamento de Cirurgia, Universidade Federal do Parana, PR, Brazil
,
S. M. Huff
2   Department of Medical Informatics, University of Utah, Salt Lake City, UT, USA
,
P. J. Haug
2   Department of Medical Informatics, University of Utah, Salt Lake City, UT, USA
,
D. A. Evans
3   CLARITECH Corporation Pittsburgh, PA, USA
,
B. E. Bray
2   Department of Medical Informatics, University of Utah, Salt Lake City, UT, USA
› Author Affiliations
Further Information

Publication History

Publication Date:
15 February 2018 (online)

    Permissions and Reprints

Abstract

An essential step toward the effective processing of the medical language is the development of representational models that formalize the language semantics. These models, also known as semantic data models, help to unlock the meaning of descriptive expressions, making them accessible to computer systems. The present study tries to determine the quality of a semantic data model created to encode chest radiology findings . The evaluation methodology relied on the ability of physicians to extract information from textual and encoded representations of chest X-ray reports, whilst answering questions associated with each report. The evaluation demonstrated that the encoded reports seemed to have the same information content of the original textual reports. The methodology generated useful data regarding the quality of the data model, demonstrating that certain segments were creating ambiguous representations and that some details were not being represented.

Keywords

Data Model - Evaluation - Information Extraction - Text Encoding
 

  • REFERENCES

  • 1 Haug PJ, Henderson SE, Maack BB. Text Management in a Healthcare Information System. Journal of AHIMA 1994; 65 (12) 32-8.
    PubMedGoogle Scholar
  • 2 Haug P, Koehler S, Lau LM, Wang P, Rocha R, Huff S. A Natural Language Understanding System Combining Syntactic and Semantic Techniques. Journal of the American Medical Informatics Association 1994; 1 (symposium supplement) 247-51.
    PubMedGoogle Scholar
  • 3 Friedman C, Alderson PO, Austin JHM, Cimino JJ, Johnson SB. A General Natural-language Text Processor for Clinical Radiology. Journal of the American Medical Informatics Association 1994; 1 (2) 161-74.
    CrossrefPubMedGoogle Scholar
  • 4 Rassinoux A-M, Michel P-A, Juge C, Baud R, Scherrer J-R. Natural Language Processing of Medical Texts within the HELIOS Environment. Computer Methods and Programs in Biomedicine 1994; 45 (Suppl.) S79-96.
    CrossrefPubMedGoogle Scholar
  • 5 Zingmond D, Lenert LA. Monitoring FreeText Data Using Medical Language Processing. Computers and Biomedical Research 1993; 26 (5) 467-81.
    CrossrefPubMedGoogle Scholar
  • 6 Zweigenbaum P. MENELAS: an access system for medical records using natural language. Computers Methods and Programs in Biomedicine 1994; 45: 117-20.
    PubMedGoogle Scholar
  • 7 Sager N, Friedman C, Lyman MS. Medical Language Processing – Computer Management of Narrative Data. Reading, MA: Addison-Wesley Publishing Company; 1987
    Google Scholar
  • 8 Grishman R, Kittredge R. eds. Analyzing Language in Restricted Domains: Sublanguage Description and Processing. Hillsdale, New Jersey: Lawrence Erlbaum Associates, Inc.; 1986
  • 9 Friedman C, Huff SM, Hersh WR, Pattison-Gordon E, Cimino JJ. The Canon Group's Effort: Working Toward a Merged Model. Journal of the American Medical Informatics Association 1995; 2 (1) 4-18.
    CrossrefPubMedGoogle Scholar
  • 10 Friedman C. Automatic Structuring of Sublanguage Information: Application to Medical Narrative. In: Analyzing Language in Restricted Domains: Sublanguage Description and Processing. Grishman R, Kittredge R. eds. Hillsdale, New Jersey: Lawrence Erlbaum Associates, Inc.; 1986: 85-102.
    Google Scholar
  • 11 Friedman C. Information Structures in Clinical Narrative. In: Medical Language Processing – Computer Management of Narrative Data. Sager N, Friedman C, Lyman MS. eds. Reading, MA: Addison-Wesley Publishing Company; 1987: 61-80.
    Google Scholar
  • 12 Sager N, Lyman M, Bucknall C, Nhan N, Tick LJ. Natural Language Processing and the Representation of Clinical Data. Journal of the American Medical Informatics Associ ation 1994; 1 (2) 142-60.
    PubMedGoogle Scholar
  • 13 Evans DA, Miller RA. Final Task Report (Task 2) – Unified Medical Language System (UMLS) Project: Initial Phase in Developing Representations for Mapping Medical Knowledge: INTERNIST-I/QMR, HELP, and MeSH. Technical Report No CMU-LCL-87-1, Laboratory for Computational Linguistics. Carnegie Mellon University; Pittsburgh, PA: 1987, iv+37pp., and appendices.
    Google Scholar
  • 14 Rector AL, Nowlan WA, Kay S. Foundations for an Electronic Medical Record. Methods of Information in Medicine 1991; 30 (3) 179-86.
    Article in Thieme ConnectPubMedGoogle Scholar
  • 15 Rector AL, Glowinski WA, Nowlan WA, Rossi-Mori A. Medical-concept Models and Medical Records: An Approach Based on GALEN and PEN&PAD. Journal of the American Medical Informatics Association 1995; 2 (1) 19-35.
    CrossrefPubMedGoogle Scholar
  • 16 Rector AL, Nowlan WA, Kay S, Goble CA. Howkins TJ. A Framework for Modelling the Electronic Medical Record. Methods of Information in Medicine 1993; 32 (2) 109-19.
    PubMedGoogle Scholar
  • 17 Rossi Mori A, Bernauer J, Pakarinen V, et al. Model for Representation of Terminologies and Coding Systems in Medicine. Seminar Opportunities for European and US Cooperation in Standardization in Health Care Informatics. Geneva: 1992
    Google Scholar
  • 18 Baud R, Rassinoux A-M, Scherrer J-R. Natural Language Processing and Semantical Representation of Medical Texts. Methods of Information in Medicine 1992; 31 (2) 117-25.
    Article in Thieme ConnectPubMedGoogle Scholar
  • 19 Evans DA, Cimino JJ, Hersh WR, Huff SM, Bell DS. Toward a Medical-concept Representation Language. Journal of the American Medical Informatics Association 1994; 1 (3) 207-17.
    CrossrefPubMedGoogle Scholar
  • 20 Masarie FE, Miller RA, Bouhaddou O, Giuse NB, Warner HR. An Interlingua for Electronic Interchange of Medical Information: Using Frames to Map between Clinical Vocabularies. Computers and Biomedical Research 1991; 24: 379-400.
    CrossrefPubMedGoogle Scholar
  • 21 Rector AL, Nowlan WA, Glowinski A. Goals for Concept Representation in the GALEN project. In: Seventeenth Symposium on Computer Applications in Medical Care. Safran C. ed. Washington, D.C.: McGraw-Hill, Inc.; New York, NY; 1993: 414-8.
    Google Scholar
  • 22 Shiffman S, Detmer W, Lane CD, Fagan LM. A Continuous-speech Interface to a Decision Support System: I. Techniques to Accommodate for Misrecognized Input. Journal of the American Medical Informatics Association 1995; 2 (1) 36-45.
    PubMedGoogle Scholar
  • 23 White GM. Natural Language Understanding and Speech Recognition. Communications of the ACM 1990; 33 (8) 72-82.
    CrossrefPubMedGoogle Scholar
  • 24 Hobbs JR. The Generic Information Extraction System. Fifth Message Understanding; Conference (MUC-5). Baltimore, Maryland: Morgan Kaufman Publishers, Inc.; 1993: 87-91.
  • 25 Do Amaral MB, Satomura Y. Structuring medical information into a language-independent database. Medical Informatics 1994; 19 (3) 269-82.
    CrossrefPubMedGoogle Scholar
  • 26 Nangle B, Keane MT. Effective retrieval in Hospital Information Systems: The use of context in answering queries to Patient Discharge Summaries. Artificial Intelligence in Medicine 1994; 6: 207-27.
    CrossrefPubMedGoogle Scholar
  • 27 Berrut C. Indexing Medical Reports: The RIME Approach. Information Processing & Management 1990; 26 (1) 93-109.
    CrossrefPubMedGoogle Scholar
  • 28 Bell DS, Greenes RA, Doubilet P. Form-based clinical input from a structured vocabulary: Initial application in ultrasound reporting. In: Sixteenth Annual Symposium on Computers Applications in Medical Care. Frisse ME. ed. Washington, DC: McGraw-Hill, Inc.; 1992: 789-91.
    Google Scholar
  • 29 Benoit R, Canfield K, Teitelbaum S, et al. Frame-based Clinical Data Entry: Design Issues for Physician Acceptance. In: Seventh World Congress on Medical Informatics, MEDINFO 92. Lun KC, Degoulet P, Pierre T, Rienhoff O. eds. Geneva, Switzerland: Elsevier Science Publishers; 1992: 1280-85.
    Google Scholar
  • 30 Kuhn K, Zemmler T, Reichert M, Heinlein C, Roesner D. Structured Data Collection and Knowledge-Based User Guidance for Abdominal Ultrasound Reporting. In: Seventeenth Annual Symposium on Computer Applications in Medical Care. Safran C. ed. Washington DC: McGraw-Hill, Inc.; 1993: 311-5.
    Google Scholar
  • 31 Rocha RA, Huff SM, Haug PJ, Warner HR. Designing a Controlled Medical Vocabulary Server: The VOSER Project. Computers and Biomedical Research 1994; 27 (6) 472-507.
    CrossrefPubMedGoogle Scholar
  • 32 Campbell KE, Das AK, Musen MA. A Logical Foundation for Representation of Clinical Data. Journal of the American Medical Informatics Association 1994; 1 (3) 218-32.
    CrossrefPubMedGoogle Scholar
  • 33 Rassinoux A-M, Baud RH, Scherrer J-R. Conceptual graphs model extension for knowledge representation of medical texts. In: Seventh World Congress on Medical Informatics, MEDINFO 92. Lun KC, Degoulet P, Pierre T, Reinhoff O. eds. Geneva, Switzerland: Elsevier Science: Publishers; 1992: 1368-74.
    Google Scholar
  • 34 Hripcsak G, Friedman C, Alderson PO, DuMouchel W, Johnson SB, Clayton PD. Unlocking Clinical Data from Narrative Reports: A Study of Natural Language Processing. Annals of Internal Medicine 1995; 122 (9) 681-8.
    CrossrefPubMedGoogle Scholar
  • 35 Brigl B, Mieth M, Haux R, Glück E. The LBI-method for automated indexing of diagnosis by using SNOMED. Part 2. Evaluation. International Journal of Bio Medical Computing 1995; 38 (2) 101-8.
    PubMedGoogle Scholar
  • 36 Sundheim BM. Tipster/MUC-5 Information Extraction System Evaluation. Fifth Message Understanding Conference (MUC-5). Baltimore, Maryland: Morgan Kaufman Publishers, Inc.; 1993: 27-44.
  • 37 Chinchor N, Hirschman L, Lewis DD. Evaluating Message Understanding Systems: An Analysis of the Third Message Understanding Conference. Computational Lin-Iguistics 1993; 19 (3) 409-49.
    PubMedGoogle Scholar
  • 38 van Slype G. Conception d'une méthodologie genérale d'évaluation de la traduction autok. matique. Multilingua 1982; 1 (4) 221-37.
    PubMedGoogle Scholar
  • 39 Kiuchi T, Kaihara S. Quantitative Evaluation of English-Japanese Machine Translation of Medical Literature. Methods of Information in Medicine 1991; 30 (3) 199-205.
    Article in Thieme ConnectPubMedGoogle Scholar
  • 40 Kuichi T, Kaihara S. Machine translation and medical linguistics. In: Seventh World Congress on Medical Informatics, MEDINFO 92. Lun KC, Degoulet P, Pierre T, Reinhoff O. eds. Geneva, Switzerland: Elsevier Science Publishers; 1992: 1350-55.
    Google Scholar
  • 41 Nirenburg S, Carbonell J, Tomita M, Goodman K. Machine Translation: A Knowledge-based Approach. San Mateo, CA: Morgan Kaufmann Publishers, Inc.; 1992
    Google Scholar
  • 42 Huff SM, Rocha RA, Bray BE, Warner HR, Haug PJ. An Event Model of Medical Information Representation. Journal of American Medical Informatics Association 1995; 2 (2) 116-34.
    CrossrefPubMedGoogle Scholar
  • 43 International Organization for Standardization. International Standard ISO/IEC 8824: Specification of Abstract Syntax Notation One (ASN.l). (Second ed.). Geneva, Switzerland: International Organization for Standardization; 1990
    Google Scholar
  • 44 Rocha RA. Development and Evaluation of a Semantic Data Model for Chest Radiology [Dissertation]. Salt Lake City: University of Utah; 1996: 268 p.
    Google Scholar
  • 45 Huff SM, Warner HR, Bray BE, Sorenson DK, Haug PJ. Assessing the Accuracy and Completeness of Structured Clinical Databases Using Questions and Answers and Expert Systems (abstract). In: 1992 Spring Congress of the American Medical Informatics Association. Ackerman MJ. ed. Pordand, OR: American Medical Informatics Association; 1992: 35.
    Google Scholar
  • 46 Pryor TA, Gardner RM, Clayton PD, Warner HR. The HELP System. Journal of Medical Systems 1983; 7 (2) 87-101.
    CrossrefPubMedGoogle Scholar
  • 47 Pryor TA, Warner HR, Gardner RM, Clayton PD, Haug PJ. The HELP System Development Tools. In: Implementing Health Care Information Systems. Orthner HF, Blum BI. eds. New York, NY: Springer Verlag; 1989: 365-83.
    Google Scholar
  • 48 Kuperman GJ, Gardner RM, Pryor TA. HELP: A Dynamic Hospital Information System. New York, NY: Springer-Verlag; 1991
    Google Scholar
  • 49 Goodman D. The Complete HYPERCARD 2.2 Handbook, (fourth ed.). New York, NY: Random House, Inc.; 1993
    Google Scholar
  • 50 Ostell J. Asn Tool: Abstract Syntax Notation 1 Tool. 2.0. ed. Bethesda, MD: National Center for Biotechnology Information; 1992
    Google Scholar
  • 51 Chinchor N. MUC-5 Evaluation Metrics. Fifth Message Understanding Conference (MUC-5). Baltimore, Maryland: Morgan Kaufman Publishers, Inc.; 1993: 69-78.
  • 52 Keppel G. Design and Analysis: A Researcher's Handbook, (third ed.). Englewood Cliffs, NJ: Prentice-Hall, Inc.; 1991
    Google Scholar
  • 53 Will CA. Comparing Human and Machine Performance for Natural Language Information Extraction: Results for English Microelectronics from the MUC-5 Evaluation. Fifth Message Understanding Conference (MUC-5). Baltimore, Maryland: Morgan Kaufman Publishers, Inc.; 1993: 53-67.
  • 54 McDonald CJ. Protocol-based computer reminders, the quality of care and the non-perfectibility of man. New England Journal of Medicine 1976; 295: 1351-5.
    CrossrefPubMedGoogle Scholar