Lessons Learned from Developing a Drug Evidence Base to Support Pharmacovigilance

Summary

Objectives: This work identified challenges associated with extraction and representation of medication-related information from publicly available electronic sources.

Methods: We gained direct observational experience through creating and evaluating the Drug Evidence Base (DEB), a repository of drug indications and adverse effects (ADEs), and supplemented this through literature review. We extracted DEB content from the National Drug File Reference Terminology, from aggregated MEDLINE co-occurrence data, and from the National Library of Medicine’s DailyMed. To understand better the similarities, differences and problems with the content of DEB and the SIDER Side Effect Resource, and Vanderbilt’s MEDI Indication Resource, we carried out statistical evaluations and human expert reviews.

Results: While DEB, SIDER, and MEDI often agreed on medication indications and side effects, cross-system shortcomings limit their current utility. The drug information resources we evaluated frequently employed multiple, disparate vaguely related UMLS concepts to represent a single specific clinical drug indication or adverse effect. Thus, evaluations comparing drug-indication and drug-ADE coverage for such resources will encounter substantial numbers of false negative and false positive matches. Furthermore, our review found that many indication and ADE relationships are too complex – logically and temporally – to represent within existing systems.

Conclusion: To enhance applicability and utility, future drug information systems deriving indications and ADEs from public resources must represent clinical concepts uniformly and as precisely as possible. Future systems must also better represent the inherent complexity of indications and ADEs.

Citation: Smith JC, Denny JC, Chen Q, Nian H, Spickard III A, Rosenbloom ST, Miller RA. Lessons learned from developing a drug evidence base to support pharmacovigilance. Appl Clin Inf 2013; 4: 596–617

http://dx.doi.org/10.4338/ACI-2013-08-RA-0062

Keywords

Drug therapy - adverse effects - knowledge bases - drug product labeling - unified medical language system

 

Conflict of interest

The authors declare that they have no conflicts of interest in the research.

• References

• 1 Miller RA, Gardner RM, Johnson KB, Hripcsak G. Clinical decision support and electronic prescribing systems: a time for responsible thought and action. J Am Med Inform Assoc 2005; 12 (Suppl. 04) 403-409.
• 2 Wang X, Chase HS, Li J, Hripcsak G, Friedman C. Integrating heterogeneous knowledge sources to acquire executable drug-related knowledge. AMIA Annu Symp Proc 2010; 2010: 852-856.
• 3 Li Y, Salmasian H, Harpaz R, Chase H, Friedman C. Determining the Reasons for Medication Prescriptions in the EHR using Knowledge and Natural Language Processing. AMIA Annu Symp Proc 2011; 2011: 768-776.
• 4 LePendu P, Iyer SV, Bauer-Mehren A, Harpaz R, Mortensen JM, Podchiyska T, Ferris TA, Shah NH. Pharmacovigilance Using Clinical Notes. Clin Pharmacol Ther [Internet]. 2013 Mar 4. [cited 2013 Apr 11]; Available from: http://www.nature.com/doifinder/10.1038/clpt.2013.47
• 5 Sharp M, Bodenreider O, Wacholder N. A framework for characterizing drug information sources. AMIA Annu Symp Proc 2008; 2008: 662-666.
• 6 FDA’s Sentinel Initiative [Internet]. [cited 2012 Apr 3]. Available from: http://www.fda.gov/Safety/FDAs SentinelInitiative/default.htm
• 7 Mini-Sentinel [Internet]. [cited 2013 Aug 23]. Available from: http://www.mini-sentinel.org/default.aspx
• 8 Wang X, Hripcsak G, Markatou M, Friedman C. Active computerized pharmacovigilance using natural language processing, statistics, and electronic health records: A feasibility study. J Am Med Inform Assoc 2009; 16 (Suppl. 03) 328-337.
• 9 Wang X, Chase H, Markatou M, Hripcsak G, Friedman C. Selecting information in electronic health records for knowledge acquisition. J Biomed Inform 2010; 43 (Suppl. 04) 595-601.
• 10 Tatonetti NP, Ye PP, Daneshjou R, Altman RB. Data-driven prediction of drug effects and interactions. Sci Transl Med 2012; 4 (125) 125ra31.
• 11 Wei W-Q, Cronin RM, Xu H, Lasko TA, Bastarache L, Denny JC. Development and evaluation of an ensemble resource linking medications to their indications. J Am Med Inform Assoc [Internet]. 2013 Apr 10. [cited 2013 Apr 11]; Available from: http://jamia.bmj.com/cgi/doi/10.1136/amiajnl-2012–001431
• 12 Coloma PM, Avillach P, Salvo F, Schuemie MJ, Ferrajolo C, Pariente A, Fourrier-Réglat A, Molokhia M, Patadia V, Van der Lei J, Sturkenboom M, Trifirò G. A reference standard for evaluation of methods for drug safety signal detection using electronic healthcare record databases. Drug Saf 2013; 36 (Suppl. 01) 13-23.
• 13 Harpaz R, DuMouchel W, Shah NH, Madigan D, Ryan P, Friedman C. Novel data-mining methodologies for adverse drug event discovery and analysis. Clin Pharmacol Ther 2012; 91 (Suppl. 06) 1010-1021.
• 14 2011 AA National Drug File –Reference Terminology Source Information [Internet]. [cited 2012 Feb 8]. Available from: http://www.nlm.nih.gov/research/umls/sourcereleasedocs/current/NDFRT
• 15 MEDLINE Fact Sheet [Internet]. [cited 2013 Apr 3]. Available from: http://www.nlm.nih.gov/pubs/fact sheets/medline.html
• 16 Unified Medical Language System (UMLS) –Home [Internet]. [cited 2013 Apr 3]. Available from: http://www.nlm.nih.gov/research/umls
• 17 DailyMed [Internet]. [cited 2012 Feb 8]. Available from: http://dailymed.nlm.nih.gov/dailymed/about.cfm
• 18 Micromedex [Internet]. [cited 2012 Apr 23]. Available from: http://www.micromedex.com
• 19 Drug Data | FDB (First Databank) [Internet]. [cited 2012 May 4]. Available from: http://www.fdbhealth. com/
• 20 Point of Care Medical Applications | Epocrates [Internet]. [cited 2013 Mar 19]. Available from: http://www.epocrates.com
• 21 UpToDate Inc. [Internet]. [cited 2012 May 4]. Available from: http://www.uptodate.com/index
• 22 Schadow G. Assessing the impact of HL7/FDA Structured Product Label (SPL) content for medication knowledge management. AMIA Annu Symp Proc. 2007; 2007: 646-50
• 23 Pathak J, Chute CG. Analyzing categorical information in two publicly available drug terminologies: RxNorm and NDF-RT. J Am Med Inform Assoc 2010; 17 (Suppl. 04) 432-439.
• 24 Nelson SJ, Zeng K, Kilbourne J, Powell T, Moore R. Normalized names for clinical drugs: RxNorm at 6 years. J Am Med Inform Assoc 2011; 18 (Suppl. 04) 441-448.
• 25 UMLS® Reference Manual –NCBI Bookshelf [Internet]. National Library of Medicine; 2009 [cited 2012 Feb 8]. Available from: http://www.ncbi.nlm.nih.gov/books/NBK9676
• 26 MEDLINE Backfiles Source Information [Internet]. [cited 2012 Apr 10]. Available from: http://www.nlm.nih.gov/research/umls/sourcereleasedocs/current/MBD/
• 27 Wang W, Haerian K, Salmasian H, Harpaz R, Chase H, Friedman C. A drug-adverse event extraction algorithm to support pharmacovigilance knowledge mining from PubMed citations. AMIA Annu Symp Proc 2011; 2011: 1464-1470.
• 28 Zeng Q, Cimino JJ. Automated knowledge extraction from the UMLS. Proc AMIA Symp 1998; 1998: 568-572.
• 29 Duda S, Aliferis C, Miller R, Statnikov A, Johnson K. Extracting drug-drug interaction articles from MED-LINE to improve the content of drug databases. AMIA Annu Symp Proc 2005; 2005: 216-220.
• 30 Avillach P, Dufour J-C, Diallo G, Salvo F, Joubert M, Thiessard F, Mougin F, Trifirò G, Fourrier-Réglat A, Pariente A, Fieschi M. Design and validation of an automated method to detect known adverse drug reactions in MEDLINE: a contribution from the EU-ADR project. J Am Med Inform Assoc 2013; 20 (Suppl. 03) 446-452.
• 31 Shetty KD, Dalal SR. Using information mining of the medical literature to improve drug safety. J Am Med Inform Assoc 2011; 18 (Suppl. 05) 668-674.
• 32 Chen ES, Hripcsak G, Xu H, Markatou M, Friedman C. Automated acquisition of disease drug knowledge from biomedical and clinical documents: an initial study. J Am Med Inform Assoc 2008; 15 (Suppl. 01) 87-98.
• 33 SIDER Side Effect Resource [Internet]. [cited 2012 Feb 22]. Available from: http://sideeffects.embl.de
• 34 Kuhn M, Campillos M, Letunic I, Jensen LJ, Bork P. A side effect resource to capture phenotypic effects of drugs. Mol Syst Biol 2010; 6: 343.
• 35 Structured Product Labeling > SPL –Downloadable Data [Internet]. [cited 2012 Feb 8]. Available from: http://www.fda.gov/ForIndustry/DataStandards/StructuredProductLabeling/ucm240580.htm
• 36 Denny JC, Smithers JD, Miller RA, Spickard A. “Understanding” medical school curriculum content using KnowledgeMap. J Am Med Inform Assoc 2003; 10 (Suppl. 04) 351-362.
• 37 Denny JC, Miller RA, Waitman LR, Arrieta MA, Peterson JF. Identifying QT prolongation from ECG impressions using a general-purpose Natural Language Processor. Int J Med Inform 2009; 78 (Suppl. 01) S34-S42.
• 38 Denny JC, Choma NN, Peterson JF, Miller RA, Bastarache L, Li M, Peterson NB. Natural language processing improves identification of colorectal cancer testing in the electronic medical record. Med Decis Making 2012; 32 (Suppl. 01) 188-197.
• 39 Fleiss JL, Cohen J, Everitt BS. Large sample standard errors of kappa and weighted kappa. Psychol Bull 1969; 72 (Suppl. 05) 323-327.
• 40 Fleiss JL. Measuring nominal scale agreement among many raters. Psychol Bull 1971; 76 (Suppl. 05) 378-382.
• 41 The R Project for Statistical Computing [Internet]. [cited 2013 Apr 14]. Available from: http://www.r-project.org
• 42 Ghazvinian A, Noy NF, Musen MA. Creating mappings for ontologies in biomedicine: simple methods work. AMIA Annu Symp Proc 2009; 2009: 198-202.
• 43 Shah NH, Shah N, Muse MA, Musen M. UMLS-Query: a perl module for querying the UMLS. AMIA Annu Symp Proc 2008; 652-656.
• 44 McInnes BT, Pedersen T, Pakhomov SVS. UMLS-Interface and UMLS-Similarity : open source software for measuring paths and semantic similarity. AMIA Annu Symp Proc 2009; 2009: 431-435.
• 45 Xiang Y, Lu K, James SL, Borlawsky TB, Huang K, Payne PRO. k-Neighborhood decentralization: a comprehensive solution to index the UMLS for large scale knowledge discovery. J Biomed Inform 2012; 45 (Suppl. 02) 323-336.
• 46 Pivovarov R, Elhadad N. A hybrid knowledge-based and data-driven approach to identifying semantically similar concepts. J Biomed Inform 2012; 45 (Suppl. 03) 471-481.