Automatic Identification of Self-Reported COVID-19 Vaccine Information from Vaccine Adverse Events Reporting System

Abstract

Background The short time frame between the coronavirus disease 2019 (COVID-19) pandemic declaration and the vaccines authorization led to concerns among public regarding the safety and efficacy of the vaccines. The Food and Drug Administration uses the Vaccine Adverse Events Reporting System (VAERS) where general population can report their vaccine side effects in the text box. This information could be utilized to determine self-reported vaccine side effects.

Objective To develop a supervised and unsupervised natural language processing (NLP) pipeline to extract self-reported COVID-19 vaccination side effects, location of the side effects, medications, and possibly false/misinformation seeking further investigation in a structured format for analysis and reporting.

Methods We utilized the VAERS dataset of COVID-19 vaccine reports from November 2020 to August 2022 of 725,246 individuals. We first developed a gold-standard (GS) dataset of randomly selected 1,500 records. Second, the GS was split into training, testing, and validation sets. The training dataset was used to develop the NLP applications (supervised and unsupervised) and testing and validation datasets were used to test the performances of the NLP application.

Results The NLP application automatically extracted vaccine side effects, body locations of the side effects, medication, and possibly misinformation with moderate to high accuracy (84% sensitivity, 82% specificity, and 83% F-1 measure). We found that 23% people (386,270) faced arm soreness, 31% body swelling (226,208), 23% fatigue/body weakness (168,160), and 22% (159,873) cold/flue-like symptoms. Most of the complications occurred in the body locations such as the arm, back, chest, neck, face, and head. Over-the-counter pain medications such as Tylenol and Ibuprofen and allergy medication like Benadryl were most reported self-reported medications. Death due to COVID-19, changes in the DNA, and infertility were possible false/misinformation reported by people.

Conclusion Some self-reported side effects such as syncope, arthralgia, and blood clotting need further clinical investigations. Our NLP application may help in extracting information from big free-text electronic datasets to help policy makers and other researchers with decision making.

Keywords

COVID-19 - Vaccine Adverse Events Reporting System - COVID-19 vaccine side effects - vaccine misinformation

Ethical Approval Statement

The study was performed using VAERS publicly available de-identified data; hence it does not require ethical committee approval.

Publication History

Received: 26 June 2022

Accepted: 21 November 2022

Article published online:
09 January 2023

• References

• 1 “Coronavirus in the US: latest map and case count.” The New York Times. Accessed November 12, 2021 at: https://www.nytimes.com/interactive/2021/us/covid-cases.html
• 2 “Understanding emergency use authorization and full licensure of COVID-19 vaccines | University of Utah Health.” University of Utah Health. Accessed November 12, 2021 at: https://healthcare.utah.edu/healthfeed/postings/2021/08/eua-and-full-licensure-covid19-vaccines.php
• 3 “Johnson & Johnson COVID-19 vaccine authorized by US FDA for emergency use | Johnson & Johnson.” Johnson & Johnson. Accessed November 12, 2021 at: https://www.jnj.com/johnson-johnson-covid-19-vaccine-authorized-by-u-s-fda-for-emergency-usefirst-single-shot-vaccine-in-fight-against-global-pandemic
• 4 “Moderna COVID-19 Vaccine | FDA.” U.S. Food and Drug Administration, FDA. Accessed November 12, 2021 at: https://www.fda.gov/emergency-preparedness-and-response/coronavirus-disease-2019-covid-19/moderna-covid-19-vaccine#
• 5 “FDA authorizes Moderna Covid-19 Vaccine.” The Medical Letter on Drugs and Therapeutics. Accessed January 24, 2022 at: https://pubmed.ncbi.nlm.nih.gov/33512345/
• 6 “FDA authorizes Johnson & Johnson Covid-19 vaccine.” The Medical Letter on Drugs and Therapeutics. Accessed January 24, 2022 at: https://pubmed.ncbi.nlm.nih.gov/33976088/
• 7 Baden LR, El Sahly HM, Essink B. et al; COVE Study Group. efficacy and safety of the mRNA-1273 SARS-CoV-2 vaccine. N Engl J Med 2021; 384 (05) 403-416
• 8 Polack FP, Thomas SJ, Kitchin N. et al; C4591001 Clinical Trial Group. Safety and efficacy of the BNT162b2 mRNA Covid-19 vaccine. N Engl J Med 2020; 383 (27) 2603-2615
• 9 Sadoff J, Gray G, Vandebosch A. et al; ENSEMBLE Study Group. Safety and efficacy of single-dose Ad26.COV2.S vaccine against COVID-19. N Engl J Med 2021; 384 (23) 2187-2201
• 10 Centers for Disease Control and Prevention. “VAERS | Vaccine Safety | CDC.” Accessed November 12, 2021 at: https://www.cdc.gov/vaccinesafety/ensuringsafety/monitoring/vaers/index.html
• 11 “Data mining at the center for biologics evaluation and research | FDA.” US Food and Drug Administration, FDA. Accessed November 12, 2021 at: https://www.fda.gov/science-research/data-mining/data-mining-center-biologics-evaluation-and-research
• 12 “Data Mining at FDA–White Paper.” US Food and Drug Administration, FDA. Accessed November 10, 2021 at: https://www.fda.gov/science-research/data-mining/data-mining-fda-white-paper
• 13 “VAERS Home.” US Department of Health and Human Services. Accessed December 8, 2022 at: https://vaers.hhs.gov/reportevent.html
• 14 Botsis T, Jankosky C, Arya D. et al. Decision support environment for medical product safety surveillance. J Biomed Inform 2016; 64: 354-362
• 15 Botsis T, Nguyen MD, Woo EJ, Markatou M, Ball R. Text mining for the Vaccine Adverse Event Reporting System: medical text classification using informative feature selection. J Am Med Inform Assoc 2011; 18 (05) 631-638
• 16 Brumfiel, Geoff. “Anti-vaccine activists use a federal database to spread fear about Covid vaccines.” NPR, NPR. Published June 14, 2021. Accessed December 8, 2022 at: https://www.npr.org/sections/health-shots/2021/06/14/1004757554/anti-vaccine-activists-use-a-federal-database-to-spread-fear-about-covid-vaccine
• 17 npr. Anti-vaccine activists use a federal database to spread fear about COVID vaccines. Accessed December 8, 2022 at: https://www.npr.org/sections/health-shots/2021/06/14/1004757554/anti-vaccine-activists-use-a-federal-database-to-spread-fear-about-covid-vaccine
• 18 Wadman M. Antivaccine activists use a government database on side effects to scare the public. Science. Accessed November 12, 2021 at: https://www.sciencemag.org/news/latest-news?_ga=2.57532202.1250888674.1585020615-97107484.1585020615&page=3
• 19 South BR, Shen S, Leng J, Forbush TB, DuVall SL, Chapman WW. A Prototype Tool Set to Support Machine-Assisted Annotation. Stroudsburg, PA: Association for Computational Linguistics; 2012
• 20 Sureshbhai Patel J, Rao R, Brandon R. et al. (2022). Develop a natural language processing pipeline to automate extraction of periodontal disease information from electronic dental clinical notes. Paper presented at: 2022 6th International Conference on Medical and Health Informatics; Kyoto, Japan; May 13–15, 2022:61–68
• 21 Widiastuti NI. Convolution neural network for text mining and natural language processing. IOP Conf Ser Mater Sci Eng 2019; 662: 0520101
• 22 Kenter T, Borisov A, de Rijke M. Siamese CBOW: optimizing word embeddings for sentence representations. arXiv.org. Accessed October 30, 2022 at: https://arxiv.org/abs/1606.04640
• 23 Yogish D, Manjunath TN, Hegadi RS. Review on natural language processing trends and techniques using NLTK. Paper presented at: RTIP2R 2018; Solapur, India; December 21–22, 2018; Communications in Computer and Information Science, vol 1037.
• 24 Hripcsak G, Rothschild AS. Agreement, the f-measure, and reliability in information retrieval. J Am Med Inform Assoc 2005; 12 (03) 296-298
• 25 Centers for Disease Control and Prevention. “Selected adverse events reported after COVID-19 vaccination.” Accessed January 20, 2022 at: https://www.cdc.gov/coronavirus/2019-ncov/vaccines/safety/adverse-events.html
• 26 Luo C, Du J, Cuker A. et al. Comparability of clinical trials and spontaneous reporting data regarding COVID-19 vaccine safety. Sci Rep 2022; 12 (01) 10946
• 27 Centers for Disease Control and Prevention. Possible side effects after getting a COVID-19 vaccine. Accessed December 8, 2022 at: https://www.cdc.gov/coronavirus/2019-ncov/vaccines/expect/after.html
• 28 Male V. Menstrual changes after covid-19 vaccination. BMJ 2021; 374: n2211
• 29 National Institutes of Health. COVID-19 vaccines and the menstrual cycle. Accessed November 12, 2021 at: https://covid19.nih.gov/news-and-stories/covid-19-vaccines-and-menstrual-cycle
• 30 Centers for Disease Control and Prevention. How to address COVID-19 vaccine misinformation. Accessed December 8, 2022 at: https://www.cdc.gov/vaccines/covid-19/health-departments/addressing-vaccine-misinformation.html