A Framework for Characterizing Terminological Systems

Summary

Objectives: The notion of a terminological system (TS) is complex due to the broad range of systems, applications, and clinical domains. A uniform approach to describe the characteristics of TSs is lacking. This impedes furthering understanding, applicability, mutual comparison and development of TSs. For these reasons we propose a terminological systems characterization framework.

Methods: Relevant issues pertaining to TSs and terminology servers have been extracted from literature describing requirements and functionality of TSs. From these issues, features have been distilled and further refined. A categorization has been developed to provide a convenient arrangement of these features.

Results: The framework distinguishes between application-dependent and application-independent features of TSs. Definitions are provided for measures of content coverage, which was identified as the only application-dependent feature. Application-independent features are categorized along two axes: their respective type of TS and the particular element within that system, i.e. the formalism, the content, or the functionality. For each feature we provide an explicit question, the answer to which yields a feature value. The framework has been applied to SNOMED CT and the CLUE browser.

Conclusions: We present and apply a framework to support a feature-based characterization of terminological systems. Standardized methods for content coverage studies reduce the effort of assessing the applicability of a TS for a specific clinical setting. A two-axial categorization provides a convenient arrangement of the large number of application-independent features. Application of the framework increases comparability of terminological systems. This framework may also help TS developers determine how their system can be improved.

Keywords

Terminological systems - evaluation - methods - definitions

• References

• 1 de Keizer NF, Abu-Hanna A, Zwetsloot-Schonk JH. Understanding terminological systems. I: Terminology and typology. Methods Inf Med 2000; 39 (01) 16-21.
• 2 Rossi Mori A, Consorti F, Galeazzi E. Standards to support development of terminological systems for healthcare telematics. Methods Inf Med 1998; 37 (04) (05) 551-63.
• 3 International Statistical Classification of Diseases and Related Health Problems, 1989 Revision Geneva: World Health Organization. 1992
• 4 Rogers FB. Medical subject headings. Bulletin of the Medical Library Association 1963; 51: 114-6.
• 5 Brown EG, Wood L, Wood S. The medical dictionary for regulatory activities (MedDRA). Drug safety: an international journal of medical toxicology and drug experience 1999; 20 (02) 109-17.
• 6 McDonald CJ, Huff SM, Suico JG, Hill G, Leavelle D, Aller R. et al LOINC, a universal standard for identifying laboratory observations: a 5-year update. Clinical Chemistry 2003; 49 (04) 624-33.
• 7 Coté RA, Rothwell DJ, Palotay JL, Beckett RS, Brochu L. editors SNOMED International: the systematized nomenclature of human and veterinary medicine. Vols I-IV. Northfield, IL: : College of American Pathologists; 1993
• 8 Spackman K. SNOMED RT and SNOMED CT. Promise of an international clinical terminology. MD Computing 2000; 17 (06) 29
• 9 Rector AL, Nowlan WA. The GALEN project. Computer Methods and Programs in Biomedicine 1994; 45 (01) (02) 75-8.
• 10 Harris MA, Clark J, Ireland A, Lomax J, Ashburner M, Foulger R. et al The Gene Ontology (GO) database and informatics resource. Nucleic Acids Research 2004; 32 (Database issue) D 258-61.
• 11 Rosse C, Mejino JLV. V. A reference ontology for biomedical informatics: the Foundational Model of Anatomy. Journal of Biomedical Informatics 2003; 36 (06) 478-500.
• 12 Schuyler PL, Hole WT, Tuttle MS, Sherertz DD. The UMLS Metathesaurus: representing different views of biomedical concepts. Bulletin of the Medical Library Association 1993; 81 (02) 217-22.
• 13 Ingenerf J, Giere W. Concept-oriented standardization and statistics-oriented classification: continuing the classification versus nomenclature controversy. Methods Inf Med 1998; 37 (04) (05) 527-39.
• 14 Cimino JJ. Desiderata for controlled medical vocabularies in the twenty-first century. Methods Inf Med 1998; 37 (04) [phone].
• 15 ISO/TC215 WG 3 Standard Specification for Quality Indicators for Controlled Health Vocabularies; 2000 July. Report No.: TS17117
• 16 Hales JW, Schoeffler KM. Barriers to Evaluation of Clinical Vocabularies. In Cesnik B, McCray AT, Scherrer JR. editors Medinfo1998: Proceedings of the 9th World Congress on Medical Informatics;1998; Seoul, Korea. Amsterdam, The Netherlands: IOS Press 1998; 680-4.
• 17 NCVHS SSS. Summary and Analysis of Terminology Questionnaires Submitted by Developers of Candidate Terminologies for PMRI Standards. draft 2003; April 17
• 18 OMG Lexicon Query Service Specification: Object Management Group; 2000 July. Report No.: 00-06-31.pdf
• 19 Smith B. Beyond Concepts: Ontology as Reality Representation. In Varzi A, Vieu L. editors Proceedings of FOIS 2004. International Conference on Formal Ontology and Information Systems; 2004; Turin: Amsterdam, The Netherlands: IOS Press 2004; 73-84.
• 20 Bodenreider O, Smith B, Burgun A. The Ontology- Epistemology Divide: A Case Study in Medical Terminology. In Varzi A, Vieu L. editors Proceedings of FOIS 2004. International Conference on Formal Ontology and Information Systems; 2004; Turin. Amsterdam, The Netherlands: IOS Press 2004; 185-95.
• 21 Supekar K. A Peer-review Approach for Ontology Evaluation. In Noy NF. editor. 8th International Protégé Conference; July 18-21, 2005. Madrid, Spain 2005; 77-9.
• 22 Supekar K, Patel C, Lee Y. Characterizing quality of knowledge on semantic web. In Barr V, Markov Z. editors Proceedings of the Seventeenth International FLAIRS Conference; May 2004 Miami, FL: AAAI Press; 2004: 220-8.
• 23 Cornet R. Towards Structured Requirements for Terminological Systems and Servers. In Patel V, Rogers R, Haux R. editors Medinfo 2001: Proceedings of the 10th World Congress on Medical Informatics; 2001; London, UK: IOS Press, Amsterdam, The Netherlands 2001; 295
• 24 CEN/TC251 Medical Informatics – Categorical structures of systems of concepts – Model for representation of semantics. European Prestandard. Brussels, Belgium: European Committee for Standardization; October 1997. Report No.: ENV 12264
• 25 Guarino N. Formal Ontology in Information Systems. In Guarino N. editor Proceedings of FOIS’98; June 1998; Trento, Italy: IOS Press, Amsterdam, The Netherlands 1998; 3-15.
• 26 Arts DGT, Cornet R, de Jonge E, de Keizer NF. Methods for Evaluation of Medical Terminological Systems. Methods Inf Med 2005; 44: 616-25.
• 27 Bodenreider O, Burgun A, Botti G, Fieschi M, Le Beux P, Kohler F. Evaluation of the Unified Medical Language System as a medical knowledge source. Journal of the American Medical Informatics Association 1998; 5 (01) 76-87.
• 28 Bakken Henry S, Holzemer WL, Reilly CA, Campbell KE. Terms used by nurses to describe patient problems: can SNOMED III represent nursing concepts in the patient record?. Journal of the American Medical Informatics Association 1994; 1 (01) 61-74.
• 29 Bodenreider O, Mitchell JA, McCray AT. Evaluation of the UMLS as a terminology and knowledge resource for biomedical informatics. In Kohane IS. Proceedings of the 2002 AMIA Annual Symposium; 2002; San Antonio, TX, USA. Philadelphia, PA: Hanley and Belfus Inc., USA 2002; 61-5.
• 30 Bowles KH. Application of the Omaha System in acute care. Research in Nursing & Health 2000; 23 (02) 93-105.
• 31 Campbell JR, Carpenter P, Sneiderman C, Cohn S, Chute CG, Warren J. Phase II Evaluation of Clinical Coding Schemes: Completeness, Taxonomy, Mapping, Definitions, and Clarity. Journal of the American Medical Informatics Association 1997; 4 (03) 238-51.
• 32 Chute CG, Cohn SP, Campbell KE, Oliver DE, Campbell JR. For The Computer-Based Patient Record Institute’s Work Group on Codes & Structures. The content coverage of clinical classifications. Journal of the American Medical Informatics Association 1996; 3 (03) 224-33.
• 33 Cimino JJ, Patel VL, Kushniruk AW. Studying the human-computer-terminology interface. Journal of the American Medical Informatics Association 2001; 8 (02) 163-73.
• 34 Hardiker NR, Rector AL. Structural validation of nursing terminologies. Journal of the American Medical Informatics Association 2001; 8 (03) 212-21.
• 35 Humphreys BL, McCray AT, Cheh ML. Evaluating the coverage of controlled health data terminologies: report on the results of the NLM/AHCPR large scale vocabulary test. Journal of the American Medical Informatics Association 1997; 4 (06) 484-500.
• 36 Warnekar P, Carter J. HIV Terms Coverage by a Commercial Nomenclature. In Musen MA, Friedman CP, Teich JM. editors Proceedings of the 2003 AMIA Annual Symposium; 2003; Washington, DC, USA. Philadelphia, PA, USA: Hanley and Belfus Inc 2003; 1046
• 37 Wasserman H, Wang J. An Applied Evaluation of SNOMED CT as a Clinical Vocabulary for the Computerized Diagnosis and Problem. List. In Musen MA, Friedman CP, Teich JM. editors Proceedings of the 2003 AMIA Annual Symposium; 2003; Washington, DC, USA. Philadelphia, PA, USA: Hanley and Belfus Inc 2003; 699-703.
• 38 Arts D, De Keizer N, De Jonge E, Cornet R. Comparison of methods for evaluation of a medical terminological system. In Fieschi M, Coiera E, Li J. editors Proceedings from Medinfo 2004, San Francisco, CA, USA. Amsterdam The Netherlands: IOS Press; 2004: 467-71.
• 39 Rector AL. Clinical terminology: why is it so hard?. Methods Inf Med 1999; 38 (04) (05) 239-52.