Developing Validated Tools to Identify Pulmonary Embolism in Electronic Databases: Rationale and Design of the PE-EHR+ Study

Abstract

Background Contemporary pulmonary embolism (PE) research, in many cases, relies on data from electronic health records (EHRs) and administrative databases that use International Classification of Diseases (ICD) codes. Natural language processing (NLP) tools can be used for automated chart review and patient identification. However, there remains uncertainty with the validity of ICD-10 codes or NLP algorithms for patient identification.

Methods The PE-EHR+ study has been designed to validate ICD-10 codes as Principal Discharge Diagnosis, or Secondary Discharge Diagnoses, as well as NLP tools set out in prior studies to identify patients with PE within EHRs. Manual chart review by two independent abstractors by predefined criteria will be the reference standard. Sensitivity, specificity, and positive and negative predictive values will be determined. We will assess the discriminatory function of code subgroups for intermediate- and high-risk PE. In addition, accuracy of NLP algorithms to identify PE from radiology reports will be assessed.

Results A total of 1,734 patients from the Mass General Brigham health system have been identified. These include 578 with ICD-10 Principal Discharge Diagnosis codes for PE, 578 with codes in the secondary position, and 578 without PE codes during the index hospitalization. Patients within each group were selected randomly from the entire pool of patients at the Mass General Brigham health system. A smaller subset of patients will also be identified from the Yale-New Haven Health System. Data validation and analyses will be forthcoming.

Conclusions The PE-EHR+ study will help validate efficient tools for identification of patients with PE in EHRs, improving the reliability of efficient observational studies or randomized trials of patients with PE using electronic databases.

Keywords

pulmonary embolism - validity - electronic health records - International Classification of Diseases - natural language processing

• References

• 1 Virani SS, Alonso A, Aparicio HJ. et al; American Heart Association Council on Epidemiology and Prevention Statistics Committee and Stroke Statistics Subcommittee. Heart Disease and Stroke Statistics-2021 Update: a report from the American Heart Association. Circulation 2021; 143 (08) e254-e743
• 2 Cohen AT, Agnelli G, Anderson FA. et al; VTE Impact Assessment Group in Europe (VITAE). Venous thromboembolism (VTE) in Europe. The number of VTE events and associated morbidity and mortality. Thromb Haemost 2007; 98 (04) 756-764
• 3 Heit JA, Cohen AT, Anderson FJ. Estimated annual number of incident and recurrent, non-fatal and fatal venous thromboembolism (VTE) events in the US. Blood 2005; 106: 1
• 4 Bikdeli B, Bikdeli B. Updates on advanced therapies for acute pulmonary embolism. Int J Cardiovasc Pract 2016; 1: 47-50
• 5 Barco S, Mahmoudpour SH, Valerio L. et al. Trends in mortality related to pulmonary embolism in the European Region, 2000-15: analysis of vital registration data from the WHO Mortality Database. Lancet Respir Med 2020; 8 (03) 277-287
• 6 Barco S, Valerio L, Ageno W. et al. Age-sex specific pulmonary embolism-related mortality in the USA and Canada, 2000-18: an analysis of the WHO Mortality Database and of the CDC Multiple Cause of Death database. Lancet Respir Med 2021; 9 (01) 33-42
• 7 Konstantinides SV, Meyer G, Becattini C. et al; ESC Scientific Document Group. 2019 ESC guidelines for the diagnosis and management of acute pulmonary embolism developed in collaboration with the European Respiratory Society (ERS). Eur Heart J 2020; 41 (04) 543-603
• 8 Giri J, Sista AK, Weinberg I. et al. Interventional therapies for acute pulmonary embolism: current status and principles for the development of novel evidence: a scientific statement from the American Heart Association. Circulation 2019; 140 (20) e774-e801
• 9 Ortel TL, Neumann I, Ageno W. et al. American Society of Hematology 2020 guidelines for management of venous thromboembolism: treatment of deep vein thrombosis and pulmonary embolism. Blood Adv 2020; 4 (19) 4693-4738
• 10 Aujesky D, Long JA, Fine MJ, Ibrahim SA. African American race was associated with an increased risk of complications following venous thromboembolism. J Clin Epidemiol 2007; 60 (04) 410-416
• 11 Baglin T, Bauer K, Douketis J, Buller H, Srivastava A, Johnson G. SSC of the ISTH. Duration of anticoagulant therapy after a first episode of an unprovoked pulmonary embolus or deep vein thrombosis: guidance from the SSC of the ISTH. J Thromb Haemost 2012; 10 (04) 698-702
• 12 Barnes GD, Muzikansky A, Cameron S. et al. Comparison of 4 acute pulmonary embolism mortality risk scores in patients evaluated by pulmonary embolism response teams. JAMA Netw Open 2020; 3 (08) e2010779
• 13 Cushman M, Barnes GD, Creager MA. et al; American Heart Association Council on Peripheral Vascular Disease; Council on Arteriosclerosis, Thrombosis and Vascular Biology; Council on Cardiovascular and Stroke Nursing; Council on Clinical Cardiology; Council on Epidemiology and Prevention; and the International Society on Thrombosis and Haemostasis. Venous thromboembolism research priorities: a scientific statement from the American Heart Association and the International Society on Thrombosis and Haemostasis. Circulation 2020; 142 (06) e85-e94
• 14 Bikdeli B, Jimenez D, Hawkins M. et al; RIETE Investigators. Rationale, Design and Methodology of the Computerized Registry of Patients with Venous Thromboembolism (RIETE). Thromb Haemost 2018; 118 (01) 214-224
• 15 Weitz JI, Haas S, Ageno W. et al. Global Anticoagulant Registry in the Field - Venous Thromboembolism (GARFIELD-VTE). Rationale and design. Thromb Haemost 2016; 116 (06) 1172-1179
• 16 Wiener RS, Schwartz LM, Woloshin S. Time trends in pulmonary embolism in the United States: evidence of overdiagnosis. Arch Intern Med 2011; 171 (09) 831-837
• 17 Stein PD, Beemath A, Olson RE. Trends in the incidence of pulmonary embolism and deep venous thrombosis in hospitalized patients. Am J Cardiol 2005; 95 (12) 1525-1526
• 18 Bikdeli B, Wang Y, Jimenez D. et al. Pulmonary embolism hospitalization, readmission, and mortality rates in US older adults, 1999-2015. JAMA 2019; 322 (06) 574-576
• 19 Barco S, Valerio L, Gallo A. et al. Global reporting of pulmonary embolism-related deaths in the World Health Organization mortality database: vital registration data from 123 countries. Res Pract Thromb Haemost 2021; 5 (05) e12520
• 20 Lehnert P, Lange T, Møller CH, Olsen PS, Carlsen J. Acute pulmonary embolism in a national Danish cohort: increasing incidence and decreasing mortality. Thromb Haemost 2018; 118 (03) 539-546
• 21 Patient Safety Indicator 12 (PSI 12) perioperative pulmonary embolism or deep vein thrombosis rate. Agency for Healthcare Research and Quality, 2016. Agency for Healthcare Research and Quality. Accessed October 25, 2021 at: https://www.qualityindicators.ahrq.gov/Downloads/Modules/PSI/V60-ICD10/TechSpecs/PSI_12_Perioperative_Pulmonary_Embolism_or_Deep_Vein_Thrombosis_Rate.pdf
• 22 Spyropoulos AC, Ashton V, Chen YW, Wu B, Peterson ED. Rivaroxaban versus warfarin treatment among morbidly obese patients with venous thromboembolism: comparative effectiveness, safety, and costs. Thromb Res 2019; 182: 159-166
• 23 Guo JD, Hlavacek P, Rosenblatt L. et al. Safety and effectiveness of apixaban compared with warfarin among clinically-relevant subgroups of venous thromboembolism patients in the United States Medicare population. Thromb Res 2021; 198: 163-170
• 24 Marquis-Gravel G, Roe MT, Robertson HR. et al. Rationale and Design of the Aspirin Dosing-A Patient-Centric Trial Assessing Benefits and Long-term Effectiveness (ADAPTABLE) Trial. JAMA Cardiol 2020; 5 (05) 598-607
• 25 Chen PH. Essential elements of natural language processing: what the radiologist should know. Acad Radiol 2020; 27 (01) 6-12
• 26 Wu S, Roberts K, Datta S. et al. Deep learning in clinical natural language processing: a methodical review. J Am Med Inform Assoc 2020; 27 (03) 457-470
• 27 Pham AD, Névéol A, Lavergne T. et al. Natural language processing of radiology reports for the detection of thromboembolic diseases and clinically relevant incidental findings. BMC Bioinformatics 2014; 15 (01) 266
• 28 Raja AS, Ip IK, Prevedello LM. et al. Effect of computerized clinical decision support on the use and yield of CT pulmonary angiography in the emergency department. Radiology 2012; 262 (02) 468-474
• 29 Tian Z, Sun S, Eguale T, Rochefort CM. Automated extraction of VTE events from narrative radiology reports in electronic health records: a validation study. Med Care 2017; 55 (10) e73-e80
• 30 Selby LV, Narain WR, Russo A, Strong VE, Stetson P. Autonomous detection, grading, and reporting of postoperative complications using natural language processing. Surgery 2018; 164 (06) 1300-1305
• 31 Chen MC, Ball RL, Yang L. et al. Deep learning to classify radiology free-text reports. Radiology 2018; 286 (03) 845-852
• 32 Ascent of machine learning in medicine. Nat Mater 2019; 18 (05) 407
• 33 Burles K, Innes G, Senior K, Lang E, McRae A. Limitations of pulmonary embolism ICD-10 codes in emergency department administrative data: let the buyer beware. BMC Med Res Methodol 2017; 17 (01) 89
• 34 Casez P, Labarère J, Sevestre MA. et al. ICD-10 hospital discharge diagnosis codes were sensitive for identifying pulmonary embolism but not deep vein thrombosis. J Clin Epidemiol 2010; 63 (07) 790-797
• 35 Alotaibi GS, Wu C, Senthilselvan A, McMurtry MS. The validity of ICD codes coupled with imaging procedure codes for identifying acute venous thromboembolism using administrative data. Vasc Med 2015; 20 (04) 364-368
• 36 Lawrence K, Joos C, Jones AE, Johnson SA, Witt DM. Assessing the accuracy of ICD-10 codes for identifying acute thromboembolic events among patients receiving anticoagulation therapy. J Thromb Thrombolysis 2019; 48 (02) 181-186
• 37 Prat M, Derumeaux H, Sailler L, Lapeyre-Mestre M, Moulis G. Positive predictive values of peripheral arterial and venous thrombosis codes in French hospital database. Fundam Clin Pharmacol 2018; 32 (01) 108-113
• 38 Johnson SA, Signor EA, Lappe KL. et al. A comparison of natural language processing to ICD-10 codes for identification and characterization of pulmonary embolism. Thromb Res 2021; 203: 190-195
• 39 Verma AA, Masoom H, Pou-Prom C. et al. Developing and validating natural language processing algorithms for radiology reports compared to ICD-10 codes for identifying venous thromboembolism in hospitalized medical patients. Thromb Res 2022; 209: 51-58
• 40 Andersson T, Isaksson A, Khalil H, Lapidus L, Carlberg B, Söderberg S. Validation of the Swedish National Inpatient Register for the diagnosis of pulmonary embolism in 2005. Pulm Circ 2022; 12 (01) e12037
• 41 Kreimeyer K, Foster M, Pandey A. et al. Natural language processing systems for capturing and standardizing unstructured clinical information: a systematic review. J Biomed Inform 2017; 73: 14-29
• 42 Wong A, Plasek JM, Montecalvo SP, Zhou L. Natural language processing and its implications for the future of medication safety: a narrative review of recent advances and challenges. Pharmacotherapy 2018; 38 (08) 822-841
• 43 Zeng Z, Deng Y, Li X, Naumann T, Luo Y. Natural language processing for EHR-based computational phenotyping. IEEE/ACM Trans Comput Biol Bioinformatics 2019; 16 (01) 139-153
• 44 Virani SS, Alonso A, Benjamin EJ. et al; American Heart Association Council on Epidemiology and Prevention Statistics Committee and Stroke Statistics Subcommittee. Heart Disease and Stroke Statistics-2020 Update: a report from the American Heart Association. Circulation 2020; 141 (09) e139-e596
• 45 Minges KE, Bikdeli B, Wang Y, Attaran RR, Krumholz HM. National and regional trends in deep vein thrombosis hospitalization rates, discharge disposition, and outcomes for medicare beneficiaries. Am J Med 2018; 131 (10) 1200-1208
• 46 Klok FA, Kruip MJHA, van der Meer NJM. et al. Incidence of thrombotic complications in critically ill ICU patients with COVID-19. Thromb Res 2020; 191: 145-147
• 47 Bikdeli B, Madhavan MV, Jimenez D. et al; Global COVID-19 Thrombosis Collaborative Group, Endorsed by the ISTH, NATF, ESVM, and the IUA, Supported by the ESC Working Group on Pulmonary Circulation and Right Ventricular Function. COVID-19 and thrombotic or thromboembolic disease: implications for prevention, antithrombotic therapy, and follow-up: JACC state-of-the-art review. J Am Coll Cardiol 2020; 75 (23) 2950-2973
• 48 Bikdeli B. Anticoagulation in COVID-19: randomized trials should set the balance between excitement and evidence. Thromb Res 2020; 196: 638-640
• 49 Nopp S, Janata-Schwatczek K, Prosch H, Shulym I, Königsbrügge O, Pabinger I, Ay C. Pulmonary embolism during the COVID-19 pandemic: decline in diagnostic procedures and incidence at a university hospital. Res Pract Thromb Haemost 2020; 4 (05) 835-841
• 50 Bikdeli B, Carrier M, Bates SM. Subsegmental pulmonary embolism: may not be a killer but indicates significant risk. Thromb Res 2020; 185: 180-182
• 51 Baumgartner C, Go AS, Fan D. et al. Administrative codes inaccurately identify recurrent venous thromboembolism: the CVRN VTE study. Thromb Res 2020; 189: 112-118
• 52 Hutchinson BD, Navin P, Marom EM, Truong MT, Bruzzi JF. Overdiagnosis of pulmonary embolism by pulmonary CT angiography. AJR Am J Roentgenol 2015; 205 (02) 271-277
• 53 Miller Jr WT, Marinari LA, Barbosa Jr E. et al. Small pulmonary artery defects are not reliable indicators of pulmonary embolism. Ann Am Thorac Soc 2015; 12 (07) 1022-1029
• 54 Tapson VF, Platt DM, Xia F. et al. Monitoring for pulmonary hypertension following pulmonary embolism: the INFORM study. Am J Med 2016; 129 (09) 978.e2-985.e2
• 55 Vinson DR, Drenten CE, Huang J. et al; Kaiser Permanente Clinical Research on Emergency Services and Treatment (CREST) Network. Impact of relative contraindications to home management in emergency department patients with low-risk pulmonary embolism. Ann Am Thorac Soc 2015; 12 (05) 666-673
• 56 Jung RG, Simard T, Hibbert B. et al. Association of annual volume and in-hospital outcomes of catheter-directed thrombolysis for pulmonary embolism. Catheter Cardiovasc Interv 2022; 99 (02) 440-446
• 57 Elbadawi A, Mahtta D, Elgendy IY. et al. Trends and outcomes of fibrinolytic therapy for STEMI: insights and reflections in the COVID-19 era. JACC Cardiovasc Interv 2020; 13 (19) 2312-2314
• 58 Otite FO, Saini V, Sur NB. et al. Ten-year trend in age, sex, and racial disparity in tPA (Alteplase) and thrombectomy use following stroke in the United States. Stroke 2021; 52 (08) 2562-2570
• 59 Guez D, Hansberry DR, Eschelman DJ. et al. Inferior vena cava filter placement and retrieval rates among radiologists and nonradiologists. J Vasc Interv Radiol 2018; 29 (04) 482-485
• 60 Gayou EL, Makary MS, Hughes DR. et al. Nationwide trends in use of catheter-directed therapy for treatment of pulmonary embolism in medicare beneficiaries from 2004 to 2016. J Vasc Interv Radiol 2019; 30 (06) 801-806
• 61 Pasrija C, Kronfli A, Rouse M. et al. Outcomes after surgical pulmonary embolectomy for acute submassive and massive pulmonary embolism: A single-center experience. J Thorac Cardiovasc Surg 2018; 155 (03) 1095.e2-1106.e2
• 62 Tamariz L, Harkins T, Nair V. A systematic review of validated methods for identifying venous thromboembolism using administrative and claims data. Pharmacoepidemiol Drug Saf 2012; 21 (Suppl. 01) 154-162