Application of an Ontology for Characterizing Data Quality for a Secondary Use of EHR Data

Summary

Objective

The goal of this study is to apply an ontology based assessment process to electronic health record (EHR) data and determine its usefulness in characterizing data quality for calculating an example eMeasure (CMS178).

Methods

The process uses a data quality ontology that references separate data quality, domain and task ontologies to compute measures based on proportions of constraints that are satisfied. These quantities indicate how well the data conforms to the domain and how well it fits the task.

Results

The process was performed on a de-identified 200,000 encounter sample from a hospital EHR. CodingConsistency was poor (44%) but DomainConsistency (97%) and TaskRelevance (95%) were very good. Improvements in the data quality Measures correlated with improvements in the eMeasure.

Conclusion

This approach can encourage the development of new detailed Domain ontologies that can be reused for data quality purposes across different organizations’ EHR data. Automating the data quality assessment process using this method can enable sharing of data quality metrics that may aid in making research results that use EHR data more transparent and reproducible.

Keywords

Data quality - electronic health record - data validation and verification - ontology

• References

• 1 Kayyali B, Knott D, van Kuiken S. The big-data revolution in US health careS: Accelerating value and innovation. McKinsey & Company; 2013: 1-6.
• 2 Blumenthal D, Tavenner M. The “Meaningful Use” Regulation for Electronic Health Records. N. Engl. J. Med. Massachusetts Medical Society 2010; 363: 501-504.
• 3 King J, Patel V, Furukawa MF. Physician Adoption of Electronic Health Record Technology to Meet Meaningful Use Objectives: 2009–2012. ONC Data Brief 07: 2012.
• 4 Safran C, Bloomrosen M, Hammond WE, Labkoff S, Markel-Fox S, Tang PC, Detmer DE. Toward a national framework for the secondary use of health data: an American Medical Informatics Association white paper. J Am Med Informatics Assoc 2007; 14 (01) 1-9.
• 5 Holve E, Segal C, Hamilton MLopez. Opportunities and Challenges for Comparative Effectiveness Research (CER) With Electronic Clinical Data. Med Care 2012; 50: S11-S18.
• 6 Mirnezami R, Nicholson J, Darzi A. Preparing for Precision Medicine. N Engl J Med 2012; 366: 489-491.
• 7 Hersh WR, Weiner MG, Embi PJ, Logan JR, Payne PR, Bernstam EV, Lehmann HP, Hripcsak G, Hartzog TH, Cimino JJ, Saltz JH. Caveats for the use of operational electronic health record data in comparative effectiveness research. Med Care 2013; 51: S30-S37.
• 8 Botsis T, Hartvigsen G, Chen F, Weng C. Secondary Use of EHR: Data Quality Issues and Informatics Opportunities. AMIA Proc from Summit Transl Sci 2010; 2010: 1-5.
• 9 Arts D, Keizer N, Scheffer G-J. Defining and Improving Data Quality in Medical RegistriesS: A Literature Review, Case Study, and Generic Framework. J Am Med Inf Assoc 2002; 09: 600-611.
• 10 Liaw S, Rahimi A, Ray P, Taggart J, Dennis S, de Lusignan S, Jalaludin B, Yeo AET, Talaei-Khoei A. Towards an ontology for data quality in integrated chronic disease management: a realist review of the literature. Int J Med Inform 2013; 82: 10-24.
• 11 Weiskopf NG, Weng C. Methods and dimensions of electronic health record data quality assessment: enabling reuse for clinical research. J Am Med Informatics Assoc 2012; 2-8.
• 12 Kahn MG, Raebel MA, Glanz JM, Riedlinger K, Steiner JF. ANALYTIC METHODS: A Pragmatic Framework for Single-site and Multisite Data Quality Assessment in Electronic Health. Med Care 2012; 50: 21-29.
• 13 Canadian Institute of Health. The CIHI Data Quality Framework. 2009
• 14 bObservational Medical Outcomes Partnership (OMOP) [Internet]. [cited 2015 Jul 15]. Available from: http://omop.org/
• 15 Overhage JM, Ryan PB, Reich CG, Hartzema AG, Stang PE. Validation of a common data model for active safety surveillance research. J Am Med Informatics Assoc 2012; 19: 54-60.
• 16 Platt R, Carnahan R, Brown J, Chrischilles E, Curtis L, Hennessy S, Nelson JC, Racoosin JA, Robb M, Schneeweiss S, Toh S, Weiner MG. The U.S. Food and Drug Administration’s Mini-Sentinel Program. Pharmacoepidemiol Drug Saf 2012; 21: 1-8.
• 17 Kahn MG, Brown JS, Chun AT, Davidson BN, Meeker D, Ryan PB, Schilling LM, Weiskopf NG, Williams AE, Zozus MN. Transparent Reporting of Data Quality in Distributed Data Networks. eGEMs (Generating Evid. Methods to Improv. patient outcomes) 2015; 03: 1-12.
• 18 Johnson SG, Speedie S, Simon G, Kumar V, Westra BL. A Data Quality Ontology for the Secondary Use of EHR Data. AMIA 2015 Annu Symp Proc American Medical Informatics Association 2015; 1937-1946.
• 19 Studer R, Benjamins R, Fensel D. Knowledge engineering: Principles and methods. Data Knowl Eng 1998; 25: 161-198.
• 20 Staab S, Studer R. Handbook on Ontologies. Springer; 2010
• 21 Centers for Medicare & Medicaid Services. Proposed Clinical Quality Measures for 2014 CMS EHR Incentive Programs for Eligible Hospitals & CAHs [Internet]. 2014; [cited 2015 Aug 1]. 1-20. Available from: http://www.cms.gov/Regulations-and-Guidance/Legislation/EHRIncentivePrograms/Eligible_Hospi tal_Information.html
• 22 CMS Clinical Quality eMeasure Logic and Implementation Guidance v1.3 [Internet]. 2014 [cited 2015 Aug 1]. Available from: https://www.cms.gov/Regulations-and-Guidance/Legislation/EHRIncentivePro grams/Downloads/2014_eCQM_Measure_Logic_Guidancev13_April2013.pdf
• 23 Centers for Medicare & Medicaid Services (CMS). Urinary catheter removed on Postoperative Day 1 (POD 1) or Postoperative Day 2 (POD 2) with day of surgery being day zero [Internet]. 2014 [cited 2015 Aug 1]. Available from: https://ecqi.healthit.gov/system/files/ecqm/2014/EH/measures/CMS178v5_1.html#toc.
• 24 Stéphan F, Sax H, Wachsmuth M, Hoffmeyer P, Clergue F, Pittet D. Reduction of urinary tract infection and antibiotic use after surgery: a controlled, prospective, before-after intervention study. Clin Infect Dis 2006; 42: 1544-1551.
• 25 Pipino LL, Lee YW, Wang RY. Data quality assessment. Commun ACM 2002; 45: 211.
• 26 Heinrich B, Kaiser M, Klier M. How to Measure Data Quality?. A Metric Based Approach [Internet]. 2007; [cited 2015 Aug 1]. 1-15. Available from: http://epub.uni-regensburg.de/23633/1/heinrich.pdf