Scholarly Impact of Quality Improvement Reports in Neonatology

Abstract

Objective Neonatology quality improvement (QI) projects can improve the safety and value of health care, but the scholarly impact of published QI projects is unclear. We measured scholarly citation and media attention garnered by published neonatology QI projects and analyzed project or publication characteristics associated with increased impact metrics.

Study Design We identified publications between 2016 and 2019 using mapping review methodology. We correlated project characteristics with measures of scholarly citation in Scopus and Google Scholar, and media attention as measured by Altmetrics. We collected Citation and Altmetric data in 2023.

Results The search identified 148 eligible articles, with a median citation count of 7 based on Scopus (or 12, based on Google Scholar) and a median Altmetric score of 2. Notably, 66% of articles published in a journal with an Impact Factor (IF) had more citations per year than would be expected from the IF value. Higher scientific citations were associated with articles reporting process and cost outcomes; implementing interventions that addressed family education or organizational change; and using regression analysis. Higher media attention was associated with multicenter projects, longer intervention periods, and projects scoring higher on the Quality Improvement Minimum Quality Criteria Set (QI-MQCS) rubric.

Conclusion Published neonatology QI projects are well cited in subsequent scientific publications, with the choice of project outcome, interventions, and analytic strategy influencing citation metrics. Adherence to QI-MQCS guidelines was favorably associated with media attention, but not with scholarly citations.

Key Points

• Neonatology QI publications are frequently cited.

• Projects with cost data receive more citations.

• Citation and media mention predictors differ.

Keywords

scholarly - impact - quality - improvement - neonatology

Publication History

Received: 19 September 2023

Accepted: 16 May 2024

Article published online:
10 June 2024

• References

• 1 Tumin D, Akpan US, Kohler Sr JA, Uffman JC. Publication bias among conference abstracts reporting on pediatric quality improvement projects. Am J Med Qual 2020; 35 (03) 274-280
• 2 Pearlman SA, Swanson JR. A practical guide to publishing a quality improvement paper. J Perinatol 2021; 41 (06) 1454-1458
• 3 Ellsbury DL, Clark RH. Does quality improvement work in neonatology improve clinical outcomes?. Curr Opin Pediatr 2017; 29 (02) 129-134
• 4 Healy H, Croonen LEE, Onland W, van Kaam AH, Gupta M. A systematic review of reports of quality improvement for bronchopulmonary dysplasia. Semin Fetal Neonatal Med 2021; 26 (01) 101201
• 5 Lachman P, Jayadev A, Rahi M. The case for quality improvement in the neonatal intensive care unit. Early Hum Dev 2014; 90 (11) 719-723
• 6 Geyer ED, Miller R, Kim SS, Tobias JD, Nafiu OO, Tumin D. Quality and impact of survey research among anesthesiologists: a systematic review. Adv Med Educ Pract 2020; 11: 587-599
• 7 Sacks OA, Lambour AJ, Wilcox AR, Nammalwar S, Wong SL. Quality assessment of the literature on surgical quality improvement. Surgery 2019; 166 (05) 764-768
• 8 Walshe K, Freeman T. Effectiveness of quality improvement: learning from evaluations. Qual Saf Health Care 2002; 11 (01) 85-87
• 9 Patthi B, Prasad M, Gupta R. et al. Altmetrics - A collated adjunct beyond citations for scholarly impact: a systematic review. J Clin Diagn Res 2017; 11 (06) ZE16-ZE20
• 10 Ruscio J, Seaman F, D'Oriano C, Stremlo E, Mahalchik K. Measuring scholarly impact using modern citation-based indices. Measurement 2012; 10 (03) 123-146
• 11 Mullins CH, Boyd CJ, Lindeman B. Factors associated with the highest and lowest cited research articles in general surgery journals. J Surg Res 2020; 250: 39-44
• 12 Kim HSJ, Wahid M, Choi C, Das P, Jung S, Khosa F. Bibliometric analysis of manuscript characteristics that influence citations: a comparison of ten major dermatology journals. Burns 2020; 46 (07) 1686-1692
• 13 James K, Randall N, Haddaway N. A methodology for systematic mapping in environmental sciences. Environ Evid 2016; 5: 7
• 14 SQUIRE | SQUIRE 2.0 Guidelines. Accessed May 9, 2022 at: http://squire-statement.org/index.cfm?fuseaction=Page.ViewPage&PageID=471
• 15 McGowan J, Sampson M, Salzwedel DM, Cogo E, Foerster V, Lefebvre C. PRESS Peer Review of Electronic Search Strategies: 2015 Guideline Statement. J Clin Epidemiol 2016; 75: 40-46
• 16 Nathan AT, Kaplan HC. Tools and methods for quality improvement and patient safety in perinatal care. Semin Perinatol 2017; 41 (03) 142-150
• 17 Ellsbury DL, Ursprung R. A primer on quality improvement methodology in neonatology. Clin Perinatol 2010; 37 (01) 87-99
• 18 Page MJ, McKenzie JE, Bossuyt PM. et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ 2021; 372 (71) n71
• 19 Hempel S, Shekelle PG, Liu JL. et al. Development of the Quality Improvement Minimum Quality Criteria Set (QI-MQCS): a tool for critical appraisal of quality improvement intervention publications. BMJ Qual Saf 2015; 24 (12) 796-804
• 20 SCOPUS. Elsevier BV. Accessed May 20, 2022 at: https://www.scopus.com/home.uri
• 21 Williams A. Altmetrics: an overview and evaluation. Online Inf Rev 2017; 41 (03) 311-317
• 22 Setti G. Bibliometric indicators: why do we need more than one?. IEEE Access 2013; 1: 232-246
• 23 Bornmann L, Schier H, Marx W, Daniel HD. What factors determine citation counts of publications in chemistry besides their quality?. J Informetrics 2012; 6 (01) 11-18
• 24 Onodera N, Yoshikane F. Factors affecting citation rates of research articles. J Assoc Inf Sci Technol 2015; 66 (04) 739-764
• 25 Kunze KN, Manzi JE, Polce EM, Vadhera A, Bhandari M, Piuzzi NS. High social media attention scores are not reflective of study quality: an altmetrics-based content analysis. Intern Emerg Med 2022; 17 (05) 1363-1374
• 26 Monzani A, Tagliaferri F, Bellone S, Genoni G, Rabbone I. A global overview of COVID-19 research in the pediatric field: bibliometric review. JMIR Pediatr Parent 2021; 4 (03) e24791
• 27 Groff ML, Offringa M, Emdin A, Mahood Q, Parkin PC, Cohen E. Publication trends of pediatric and adult randomized controlled trials in general medical journals, 2005-2018: a citation analysis. Children (Basel) 2020; 7 (12) 293
• 28 McDowell DT, Darani A, Shun A, Thomas G, Holland AJA. A bibliometric analysis of pediatric liver transplantation publications. Pediatr Transplant 2017 ;21(4):
• 29 Shah A. Using data for improvement. BMJ 2019; 364 (8188) l189
• 30 Chang J, Desai N, Gosain A. Correlation between altmetric score and citations in pediatric surgery core journals. J Surg Res 2019; 243: 52-58
• 31 Patel PA, Boyd CJ. Altmetric analysis of the most mentioned articles online in pediatric ophthalmology. J Pediatr Ophthalmol Strabismus 2023; 60 (05) 378-379
• 32 Kunze KN, Polce EM, Vadhera A. et al. What is the predictive ability and academic impact of the Altmetrics score and social media attention?. Am J Sports Med 2020; 48 (05) 1056-1062
• 33 Suelzer EM, Jackson JL. Measures of impact for journals, articles, and authors. J Gen Intern Med 2022; 37 (07) 1593-1597