The contribution of the Vaccine adverse event Text Mining system to the classification of possible Guillain-Barré Syndrome reports

Summary

Background: We previously demonstrated that a general purpose text mining system, the Vaccine adverse event Text Mining (VaeTM) system, could be used to automatically classify reports of anaphylaxis for post-marketing safety surveillance of vaccines.

Objective: To evaluate the ability of VaeTM to classify reports to the Vaccine Adverse Event Reporting System (VAERS) of possible Guillain-Barré Syndrome (GBS).

Methods: We used VaeTM to extract the key diagnostic features from the text of reports in VAERS. Then, we applied the Brighton Collaboration (BC) case definition for GBS, and an information retrieval strategy (i.e. the vector space model) to quantify the specific information that is included in the key features extracted by VaeTM and compared it with the encoded information that is already stored in VAERS as Medical Dictionary for Regulatory Activities (MedDRA) Preferred Terms (PTs). We also evaluated the contribution of the primary (diagnosis and cause of death) and secondary (second level diagnosis and symptoms) diagnostic VaeTM-based features to the total VaeTM-based information.

Results: MedDRA captured more information and better supported the classification of reports for GBS than VaeTM (AUC: 0.904 vs. 0.777); the lower performance of VaeTM is likely due to the lack of extraction by VaeTM of specific laboratory results that are included in the BC criteria for GBS. On the other hand, the VaeTM-based classification exhibited greater specificity than the MedDRA-based approach (94.96% vs. 87.65%). Most of the VaeTM-based information was contained in the secondary diagnostic features.

Conclusion: For GBS, clinical signs and symptoms alone are not sufficient to match MedDRA coding for purposes of case classification, but are preferred if specificity is the priority.

Citation: Botsis T, Woo EJ, Ball R. The contribution of the vaccine adverse event text mining system to the classification of possible Guillain-Barré syndrome reports. Appl Clin Inf 2013; 4: 88–99

http://dx.doi.org/10.4338/ACI-2012-11-RA-0049

Keywords

Biosurveillance and case reporting - data repositories - data mining - natural language processing - data access - integration - analysis

• References

• 1 Varricchio F, Iskander J, Destefano F, Ball R, Pless R, Braun MM, Chen RT. Understanding vaccine safety information from the vaccine adverse event reporting system. The Pediatric Infectious Disease Journal 2004; 23 (04) 287-294.
• 2 Brown EG, Wood L, Wood S. The medical dictionary for regulatory activities (MedDRA). Drug Safety 1999; 20 (02) 109-117.
• 3 Brown EG. Using MedDRA: implications for risk management. Drug Safety 2004; 27 (08) 591-602.
• 4 Food and Drug Administration.. Guidance for Industry. US Department of Health and Human Services. Available from http://www.fda.gov/downloads/RegulatoryInformation/Guidances/UCM126834.pdf Last accessed: 30 October 2012
• 5 Bousquet C, Henegar C, Louet ALL, Degoulet P, Jaulent MC. Implementation of automated signal generation in pharmacovigilance using a knowledge-based approach. International Journal of Medical Informatics 2005; 74 7-8 563-571.
• 6 Bonhoeffer J, Kohl K, Chen R, Duclos P, Heijbel H, Heininger U, Jefferson T, Loupi E. The Brighton Collaboration: addressing the need for standardized case definitions of adverse events following immunization (AEFI). Vaccine 2002; 21 3-4 298-302.
• 7 Botsis T, Woo EJ, Ball R. Application of Information Retrieval Approaches to Case Classification in the Vaccine Adverse Event Reporting System. Drug Safety (in press).
• 8 Botsis T, Buttolph T, Nguyen MD, Winiecki S, Woo EJ, Ball R. Vaccine adverse event text mining system for extracting features from vaccine safety reports. Journal of the American Medical Informatics Association 2012; 19 (06) 1011-1018.
• 9 Botsis T, Nguyen MD, Woo EJ, Markatou M, Ball R. Text mining for the Vaccine Adverse Event Reporting System: medical text classification using informative feature selection. Journal of the American Medical Informatics Association 2011; 18 (05) 631-638.
• 10 Brighton Collaboration.. ABC tool: Electronic Concultant. Available from: https://brightoncollaboration org/public/what-we-do/capacity/abc html. Last accessed: 7 July 2012
• 11 Sejvar JJ, Kohl KS, Gidudu J, Amato A, Bakshi N, Baxter R, Burwen DR, Cornblath DR, Cleerbout J, Edwards KM. Guillain-Barré syndrome and Fisher syndrome: case definitions and guidelines for collection, analysis, and presentation of immunization safety data. Vaccine 2011; 29 (03) 599-612.
• 12 UMLS® Reference Manual [Internet].. Bethesda (MD): National Library of Medicine (US); 2009 Sep. Available from: http://www.ncbi.nlm.nih.gov/books/NBK9676.
• 13 Manning CD, Raghavan P, Schutze H. Introduction to information retrieval. 1st edition. Cambridge University Press; Cambridge: 2008
• 14 Harpaz R, DuMouchel W, Shah NH, Madigan D, Ryan P, Friedman C. Novel Data-Mining Methodologies for Adverse Drug Event Discovery and Analysis. Clinical Pharmacology & Therapeutics 2012; 91 (06) 1010-1021.
• 15 Ali AK, Hartzema AG. Assessing the association between omalizumab and arteriothrombotic events through spontaneous adverse event reporting. Journal of Asthma and Allergy 2012; 5: 1-9.
• 16 Harpaz R, Haerian K, Chase HS, Friedman C. Statistical Mining of Potential Drug Interaction Adverse Effects in FDA’s Spontaneous Reporting System. AMIA Annual Symposum Proceedings 2010: 281-285.
• 17 Kadoyama K, Sakaeda T, Tamon A, Okuno Y. Adverse event profile of tigecycline: data mining of the public version of the u.s. Food and drug administration adverse event reporting system. Biological & Pharmaceutical Bulletin 2012; 35 (06) 967-970.
• 18 Trifiro G, Patadia V, Schuemie MJ, Coloma PM, Gini R, Herings R, Hippisley-Cox J, Mazzaglia G, Giaquinto C, Scotti L. et al. EU-ADR healthcare database network vs. spontaneous reporting system database: preliminary comparison of signal detection. Studies in Health Technology and Informatics 2011; 166: 25-30.
• 19 Pearson RK, Hauben M, Goldsmith DI, Gould AL, Madigan D, O’Hara DJ, Reisinger SJ, Hochberg AM. Influence of the MedDRA hierarchy on pharmacovigilance data mining results. International Journal of Medical Informatics 2009; 78 (12) e97-e103.
• 20 Tolentino H, Matters M, Walop W, Law B, Tong W, Liu F, Fontelo P, Kohl K, Payne D. Concept negation in free text components of vaccine safety reports. AMIA Annual Symposium Proceedings; 2006: 1122.
• 21 Tolentino HD, Matters MD, Walop W, Law B, Tong W, Liu F, Fontelo P, Kohl K, Payne DC. A UMLS-based spell checker for natural language processing in vaccine safety. BMC Medical Informatics & Decision Making 2007; 7: 3.