Agreement between an Early-Onset Neonatal Sepsis Risk Calculator and the Colombian Clinical Practice Guideline in Three Tertiary-Care Centers in Bogotá, Colombia

Keywords

neonatal sepsis - newborn - Early-Onset Sepsis Calculator - practice guideline

Early-onset neonatal sepsis (EONS) is a clinical condition responsible for a large number of hospital admissions worldwide.[1] The etiological organism most frequently reported is Streptococcus agalactiae, a member of the group B streptococci (GBS). However, its incidence has decreased since the implementation of maternal screening with rectovaginal cultures.[2] Although GBS remains the most frequent microorganisms, the use of these strategies has shifted the global epidemiology toward germs such as Escherichia coli and Listeria monocytogenes.[3]

The positivity rate for blood cultures confirming EONS is low, with an incidence of approximately 0.5 per 1,000 live births, as reported by the Centers for Disease Control and Prevention (CDC). The incidence of this condition in Colombia is currently unknown, but admission rates to neonatal intensive care units (NICUs) due to suspected EONS are high, leading to negative outcomes such as bacterial resistance, medication-related adverse events, unnecessary procedures, and prolonged hospital stays, among others. This situation is not uncommon worldwide, which is why in 2011, Dhudasia et al used maternal factors and features of the newborn's clinical condition to predict EONS, leading to the development of the Early-Onset Sepsis Calculator (EOSCalc).[4]

The need to standardize the clinical approach to EONS has led to the development of clinical practice guidelines. Both the United Kingdom's National Institute for Health and Care Excellence (NICE) guideline and the Colombian Early-Onset Neonatal Sepsis Clinical Practice Guidelines (CPG) discourage the use of risk assessment tools in managing EONS.[5] [6] However, various studies have been performed to evaluate the clinical, economic, and social outcomes derived from using EOSCalc in different populations, with favorable results such as reductions in the rates of NICU admissions, procedures, and antibiotic use.[1] [7] [8] [9] [10] [11] [12] [13] [14] [15] [16] [17] [18] [19] Other studies highlight the concern that EOSCalc may miss EONS cases, leading to delays in diagnosis and negative outcomes.[1] [19] [20] [21] [22] [23] The aim of this study was to calculate the incidence of EONS in Bogotá and to calculate the agreement in clinical recommendations according to the CPG and EOSCalc.

Materials and Methods

A retrospective cohort study was performed in three tertiary-care centers in Bogotá. Data were taken from electronic health records (EHRs) spanning January 2017 to January 2020. We selected EHRs of newborns with gestational age ≥34 weeks in whom complete blood counts, blood cultures, C-reactive protein (CRP), erythrocyte sedimentation rate, or procalcitonin was ordered within 72 hours after delivery to identify patients with a presumptive diagnosis of EONS.[24] We excluded records from patients with chromosomal anomalies, major congenital defects, or perinatal hypoxia.

We calculated the sample size required for an interrater reliability study (GPC and EOSCalc) with a binary outcome of antibiotic use recommendation. We estimated different sample sizes for various proportions of correct recommendations, defined as a rater giving an antibiotic use recommendation in patients in whom blood cultures were ultimately positive, and for different alternate hypotheses on Cohen's kappa. Using an alternate hypothesis for Cohen's kappa of 80% and a correct recommendation rate of 5%, we calculated a sample size of approximately 185 patients.[25]

The study variables were selected from perinatal risk factors, the neonate's clinical status, any ancillary testing performed, and antibiotic use. The normal range for leukocyte count was considered to be between 5,000 and 33,000 cells/uL, and CRP > 5 mg/L was considered abnormal. Using these data as input, the EOSCalc classified each neonate into one of three groups: clinically ill-appearing, equivocal-appearing, and well-appearing. Clinically ill patients had a persistent requirement of oxygen therapy with high-flow nasal cannulas, continuous positive airway pressure (CPAP), or mechanical ventilation, hemodynamic instability despite vasoactive support, neonatal encephalopathy, or a requirement of oxygen therapy to maintain oxygen saturations of >90% for ≥2 hours. Equivocal-appearing newborns displayed isolated tachycardia, tachypnea, fever, or respiratory distress for >4 hours, or a combination of at least two of the above for >2 hours. On data extraction from 10 positive blood cultures, 3 were judged to be contaminated because of the specific bacterial isolate, the opinion of the pediatric infectious disease specialist, and the patient's clinical status diverging considerably from the expected evolution for the reported isolate.

The Colombian GPC recommends making a diagnosis of EONS if three or more of the following criteria are present: increased oxygen requirements, apnea (>20 seconds), tachypnea, bradycardia, arterial hypotension, perfusion alteration (capillary refill time > 3 seconds), altered mental status, irritability, neonatal seizure, abnormal temperature (outside the 36–37.5 °C range), persistent vomiting, abdominal distension (>3 cm increase from baseline), jaundice, glucose intolerance (>120 mg/dL), metabolic acidosis.

The results of EOSCalc were interpreted under two scenarios, given the wording assigned to risk scores by the calculator. The calculator outputs a recommendation of “empirical antibiotics” if calculated risk exceeds 3 per 1,000 live births and “consider antibiotics” if the calculated risk is below that value but clinical examination findings suggest clinical illness. Since these two outputs are open to interpretation, we analyzed the results considering an “unequivocal approach,” in which antibiotics would be administered only to children with a recommendation of “empirical antibiotics,” and a “cautious approach” in which antibiotics would have been administered to any child with at least a “consider antibiotics” recommendation.

The protocol for this study was approved by the Ethics in Research Committee of the Fundación Universitaria Sanitas (grant no.: CEIFUS 879–20) and the Ethics in Research Committee of the Universidad del Rosario (approval no.: DVO0051488-CV1365). Anonymized data were entered into a centralized database for analysis. All analyses were performed using R. Risk assessments were obtained from the Kaiser Permanente website using the CDC-reported incidence of 0.5/1,000 newborns since the incidence in Colombia was unknown.[24] [26] [27] Sensitivity analyses using incidences of 0.3/1,000 newborns and 0.6/1,000 newborns were performed. The final clinical recommendations were based on the clinical variables included in the EOSCalc model. This was compared with the recommendations of the 2013 Colombian EONS CPG. A descriptive analysis of length of hospital stay among the different risk groups according to the EOSCalc model was performed.

Results

Of the 23,497 newborns with gestational age ≥ 34 weeks, 470 (1.71%) were admitted to the NICU with a presumptive diagnosis of EONS. Females made up 41.7% (n = 196) of the sample, with a median gestational age of 37.14 weeks (interquartile range [IQR]: 3.43 weeks) and a median birth weight of 2,700 g (IQR: 716 g; [Table 1]). A single death occurred in the group of patients with culture-confirmed EONS, for a lethality of 14.3%. Although antepartum screening is considered the standard of care in Colombia, the results were often not available to physicians. Centers #2 and #3 were part of the same network within a Health Management Organization (HMO) and results for these patients were always available to physicians. However, Center no.: 1 catered to a variety of patients from different HMOs and test results were not shared with the center, which operated as a private institution outside any of the networks.

Table 1 Sociodemographic and clinical features of included subjects

Variable
Center, n (%)
Center 1	245 (52.12)
Center 2	169 (35.96)
Center 3	56 (11.92)
Four or more prenatal consults, n (%)	418 (88.94)
GBS screening, n (%)
Unknown	212 (45.11)
Negative	191 (40.64)
Positive	67 (14.26)
APP, n (%)
APP ≥ 4 h	99 (21.06)
APP 2–3.9 h	21 (4.47)
APP > 2 h	22 (4.68)
None or any Abx < 2 h	328 (69.79)
Female sex, n (%)	196 (41.7)
Neonatal respiratory distress, n (%)
No	315 (67.02)
Moaning	7 (1.49)
Intercostal retractions	67 (14.26)
Both	81 (17.23)
O2 requirement, n (%)
No	198 (42.13)
Nasal cannula > 2 h to keep SatO2 > 90%	106 (22.55)
CPAP	92 (19.57)
HFNC	11 (2.34)
MV	63 (13.4)
Need for vasoactive support, n (%)	11 (2.34)
Need for invasive procedures, n (%)	95 (20.21)
EOSCalc, n (%)
Clinically ill-appearing	241 (51.28)
Well-appearing	185 (39.36)
Equivocal	44 (9.36)
Blood cultures, n (%)
Negative	460 (97.87)
Positive	7 (1.49)
Contamination	3 (0.64)
Maternal age (y), median (Q1–Q3)	27 (23–32)
Antepartum maternal temperature (°C), median (Q1–Q3)	36.6 (36.4–37)
Rupture of membranes (h), median (Q1–Q3)	8 (3–17)
Gestational age (wk), median (Q1–Q3)	37.14 (35.57–39)
Birth weight (g), median (Q1–Q3)	2,700 (2,344–3,060)
Birth length (cm), median (Q1–Q3)	48 (46–50)
Time to first progress note (h), median (Q1–Q3)	5 (1.5–9)
Heart rate (BPM), median (Q1–Q3)	145 (135–154.75)
Respiratory rate (BPM), median (Q1–Q3)	54 (50–60)
Temperature (°C), median (Q1–Q3)	36.7 (36.5–36.9)
Oxygen saturation (%), median (Q1–Q3)	94 (91.25–96)
Approximate risk after physical examination (×1,000 NB), median (Q1–Q3)	0.5 (0.05–1.92)
Hospital stay (d), median (Q1–Q3)	5 (4–8)

Abbreviations: Abx, antibiotics; APP, antepartum prophylaxis; BPM, beats per minute; CPAP, continuous positive airway pressure; EOSCalc, Early-Onset Sepsis Calculator; GBS, group B streptococcus; HFNC, high-flow nasal cannula; MV, mechanical ventilation; NB, newborns.

In newborns, the normal range for absolute leukocyte counts is broad. Hence, 13 patients (2.76%) had leukocytosis and 4 (0.85%) had leukopenia, of which only 1 had positive blood cultures. Positive CRP (>5 g/L) was found in 33.41% (n = 156) of patients. Chest X-ray results were available for 272 (81.24%) patients, all of which had respiratory distress. Of these, normal findings were reported in 18.75% (n = 51).

Incidence of Early-Onset Neonatal Sepsis

Of 473 blood cultures, 10 were positive, 7 of which were pathological. The overall incidence of EONS in the study sample was 0.298 per 1,000 newborns (95% confidence interval [CI] > 0.120/1,000–0.614/1,000). The overall incidence per center in the 2017–2019 time period ranged from 0 cases in Center 3 to 4 cases in Center 1. The yearly incidence ranged from 0.233 per 1,000 newborns in 2017 to 0.436 per 1,000 newborns in 2019. Center 3 had no confirmed EONS cases in the study period, probably due to the low number of deliveries ([Supplementary Table S1], available in the online version only).

Theoretical Impact of EOSCalc versus the Colombian CPG

All 470 newborns with a presumptive diagnosis of EONS had data available for retrospective analysis and the theoretical calculation of risk using EOSCalc. Empirical antibiotic was administered to 354 (75.32%) newborns following the Colombian CPG recommendations.

Retrospectively calculating risk using EOSCalc with an incidence of 0.5/1,000 newborns, “empirical antibiotics” were recommended in 76/470 newborns with a presumptive diagnosis of EONS (16.17%). If a more cautious approach were adopted and antibiotics were administered to newborns receiving a recommendation of “consider antibiotics” as well, the number would rise to 248/470 (52.77%; [Table 2]).

Table 2 Comparison of EONS management recommendations from EOSCalc and the Colombian CPG

EOSCalc recommendation	CPG recommendation			Total
	No	Observation	Yes
No additional management, vital signs every 4 h for 24 h	7	62	144	213
Blood cultures, vital signs every 4 h for 24 h	0	2	7	9
Consider antibiotic, vital signs monitoring in NICU	0	36	136	172
Empirical antibiotic, NICU admission	0	9	67	76
Total	7	109	354	470

Abbreviation: NICU, neonatal intensive care unit.

On the other hand, considering only the 354 patients in whom the CPG suggested antibiotics, only 67 (18.9%) would obtain that recommendation under EOSCalc, which would in turn lead to a reduction in antibiotic use of 81.1%. If including the patients in whom antibiotic was suggested, the number of patients rises to 203 (57.34%), which would lead to an antibiotic use reduction of 42.66%. When dichotomizing the results to take into account only unequivocal antibiotic recommendations (CPG: yes vs. no and observation; EOSCalc: empirical antibiotic versus all other recommendations), the kappa coefficient would be 6.2%. If the recommendations were split considering a more cautious approach (CPG: yes vs. no and observation; EOSCalc: empirical antibiotic and consider antibiotic versus other categories), the kappa rises to 14.2%.

The overall median hospital stay of newborns included in the study did not differ from that of newborns with confirmed EONS (5 days; IQR: 4 days). If patients are split according to the risk assessment result from EOSCalc, the median hospital stay tends to increase ([Fig. 1]).

Fig. 1 Distribution of hospital stay duration by EOSCalc recommendation.

Confirmed EONS Cases

Of the seven newborns with confirmed EONS, EOSCalc recommended “Empirical antibiotics and NICU admission” for three of them and “consider antibiotics, vital signs monitoring in NICU” for the remaining four ([Table 3]). The Colombian CPG recommended antibiotics in six of the seven newborns with confirmed EONS. None of the recommendations changed when sensitivity analyses were performed using incidences of 0.3 and 0.6/1,000 newborns.

Table 3 EOSCalc recommendations to each of the seven confirmed EONS cases

ID	Calculator predictors						Calculator results			Bacterial isolate
	Gestational age (wk + d)	Maternal temperature (°C)	Rupture of membranes (h)	GBS screening	Antibiotic prophylaxis	Clinical assessment	Risk at birth	Risk after clinical assessment	EOSCalc recommendation
1	386/7	36	30	Negative	BSA > 4 h antepartum	Clinically ill	0.01	0.31	Consider antibiotics, vital signs monitoring in NICU	Streptococcus agalactiae
2	342/7	36	Intrapartum	Unknown	None	Clinically ill	0.09	1.95	Consider antibiotics, vital signs monitoring in NICU	Listeria monocytogenes
3	403/7	36.5	8.5	Positive	BSA > 4 h antepartum	Clinically ill	0.02	0.72	Consider antibiotics, vital signs monitoring in NICU	Escherichia coli
4	355/7	36.1	Intrapartum	Unknown	None	Clinically ill	2.98	0.74	Consider antibiotics, vital signs monitoring in NICU	S. agalactiae
5	396/7	39	8	Unknown	None	Clinically ill	2.98	59.62	Empirical antibiotics, NICU admission	S. agalactiae
6	350/7	37	Intrapartum	Unknown	None	Clinically ill	0.24	5.04	Empirical antibiotics, NICU admission	L. monocytogenes
7	361/7	36.7	9	Unknown	None	Clinically ill	0.23	4.89	Empirical antibiotics, NICU admission	S. agalactiae

Abbreviations: BSA, broad spectrum antibiotics; EOSCalc, Early-Onset Sepsis Calculator; GBS, group B streptococcus; NICU, neonatal intensive care unit.

Discussion

This study is the first study in Colombia to establish an incidence for EONS, estimated to be 0.298 per 1,000 newborns in three centers in Bogotá between 2017 and 2019. Additionally, this is the first Latin American study to evaluate the agreement between the management recommendations from the local CPGs and EOSCalc. Seven confirmed cases of EONS were reported in this period, all of which received a recommendation to start or consider antibiotics according to EOSCalc. However, one of these cases did not receive this recommendation according to the Colombian CPG, which might suggest that EOSCalc displays greater sensitivity for EONS.[1] [12] [28] [29]

Regarding the bacterial isolates in the confirmed EONS cases, GBS continues to be the most frequently isolated group, followed by L. monocytogenes and E. coli. These findings are similar to those reported in the literature,[1] [28] which describes a shift away from EONS caused by GBS due to the implementation of screening and antepartum prophylaxis.[3] [9] On analysis of the seven EONS confirmed cases, five of them (71.4%) had had no GBS screening and did not receive adequate GBS prophylaxis, corroborating that these two measures have likely been key to reducing the incidence of EONS worldwide. Given the severity of GBS-related EONS, a high lethality is to be expected, and one of seven cases in this study had this outcome. On the other hand, the point estimate for the frequency of L. monocytogenes exceeds the values reported elsewhere, which ranges between 4 and 6% for Western Europe and Australasia.[30] [31] However, the low frequency of EONS found in this study leads to considerable uncertainty around the point estimates, with CIs that overlap with the values reported worldwide (28%; 95% CI: 3.67–70.96).

The results of ancillary testing (complete blood count, CRP 12 hours after delivery) showed that the leukocyte count is a poor predictor of EONS. Furthermore, as expected, elevated CRP values display high sensitivity and low specificity for the diagnosis of EONS, limiting its clinical application. This suggests that ancillary testing in newborns is unlikely to provide useful information when deciding whether antibiotics should be administered empirically.[5] However, once a presumptive diagnosis of EONS is made, the Colombian CPG suggests obtaining these values. Similarly, 57.88% of our patients had chest X-ray results available, and alterations were reported in 221 of them of which only 1 was confirmed to be an EONS case. It should be noted that chest X-rays are not recommended by any major CPG worldwide as part of ancillary testing for newborns with suspected EONS, given its poor diagnostic performance.[5]

The primary aim of this study was to evaluate the agreement between the recommendations of EOSCalc and those of the Colombian EONS CPG. Risk categories are different for each of these tools, with the Colombian CPG issuing three recommendations regarding the use of empirical antibiotics (start, observation, and do not start), while EOSCalc issues four recommendations (no additional management, take blood cultures, consider antibiotic, and start empirical antibiotics). Hence, we opted to split the recommendations under two interpretation scenarios. In the first, which we referred to as the unequivocal scenario, we compared the recommendations to start empirical antibiotics against all other recommendations by each tool. In the second scenario, which we refer to as “cautious,” we compared the categories that would at least justify the use of antibiotics against all others. For EOSCalc, for example, the recommendations to both start and consider empirical antibiotics were grouped and compared with the others. Theoretically, this would allow for greater homogeneity in the interpretation of the recommendations. However, Cohen's kappa for the recommendation did not exceed 15% (6 vs. 14% for each scenario, respectively), which is considered poor.[32] This disagreement could have significant effects on the frequency with which antibiotics are used.

The NICU admission rate for suspected EONS was 1.71% (470 newborns) in newborns with gestational ages ≥ 34 weeks, of which 75.32% (354 newborns) received empirical antibiotics as recommended by the local CPG. The use of EOSCalc would have allowed for an 81.1% reduction in the use of empirical antibiotics under the unequivocal scenario and 42.66% under the cautious scenario. This is similar to the results reported by Kimpton et al in the United Kingdom, who projected a reduction in the use of empirical antibiotics of up to 90.8% when comparing EOSCalc with the local guidelines.[1]

Another theoretical impact of the implementation of EOSCalc is a potential reduction in the duration of hospital stays. The median hospital stay for newborns with confirmed EONS was 5 days (IQR: 4 days). The hospital stay tended to be longer in children whose risk profile led to EOSCalc recommendations of NICU admission or close monitoring versus those with recommendations of expectant management. Studies in North America and Europe have evaluated the outcome of more accurate risk assessment in newborns, concluding that it reduces the duration of hospital stays and the mother–child separation time.[1] [8]

Limitations and Strengths

Among the limitations of this study, we found that its retrospective nature had a significant impact on the homogeneity of the quality of the EHRs, since several records were incomplete or reported the clinical status of the newborns up to 24 hours after delivery. This limited the data collection of variables related to the evolution of symptoms, which may have modified the risk assessment by EOSCalc. Additionally, the theoretical, retrospective, risk assessment by EOSCalc requires a prospective study for confirmation of the expected benefits.

Before undertaking this study, we found no local or national reports of EONS incidence. Although we used an incidence of EONS of 0.5/1,000 newborns to estimate the EOSCalc risk, the clinical recommendations did not change significantly after using a value of 0.6/1,000 newborns. This could mean that the change in the model's intercept had no significant impact on the clinical recommendations in this sample. However, Benitz and Achten, in 2021, cautioned against using calculators with different incidence values since the model itself may require a formal readjustment for the intended population.[33] The lack of reports on the incidence of EONS in Colombia from academic or administrative sources suggests that there is a pressing need to carry out public health monitoring for this condition.

The results presented here should not be interpreted as a suggestion to disregard local guidelines or expert recommendations, which can help inform the decision-making process of physicians in high-risk clinical scenarios. Risk assessment calculators should simply be an additional tool to be used in clinical practice. A potential reduction in unnecessary antibiotic use could mitigate the effects of alterations in the intestinal microbiota, such as the development of chronic illnesses.[8] [9] [10] [11] [12] [13] [14] [15] [16] [17] [18] [20] [21] [22] [34] This potential benefit justifies further prospective and controlled studies that can explore the effects of EOSCalc's use on these outcomes.

Conclusion

Thanks to a more profound understanding of the pathophysiology of EONS, different risk assessment tools with good predictive power (EOSCalc) have been developed. The adoption of this tool into clinical practice in Colombia has the potential to reduce the unnecessary use of intensive care and antibiotics in newborns. Although the use of this calculator could be clinically and administratively advantageous, further prospective studies on the safety of its implementation in the Colombian context are necessary. Lastly, the local CPG on the diagnosis and management of EONS is potentially outdated, particularly with regard to the use of risk assessment tools, and an update is likely to be necessary.

• References

• 1 Kimpton JA, Verma A, Thakkar D. et al; Neonatal Trainee-Led Research and Improvement Projects (NeoTRIPs). Comparison of NICE Guideline CG149 and the Sepsis Risk Calculator for the management of early-onset sepsis on the postnatal ward. Neonatology 2021; 118 (05) 562-568
• 2 Good PI, Hooven TA. Evaluating newborns at risk for early-onset sepsis. Pediatr Clin North Am 2019; 66 (02) 321-331
• 3 Benitz WE, Long SS. The holy grail of ascertainment of early-onset neonatal sepsis. J Pediatr 2019; 213: 10-12
• 4 Dhudasia MB, Mukhopadhyay S, Puopolo KM. Implementation of the sepsis risk calculator at an academic birth hospital. Hosp Pediatr 2018; 8 (05) 243-250
• 5 Ministerio de Salud y Protección Social de la República de Colombia, Colciencias, Instituto de Evaluación Tecnológica en Salud. Guía de Práctica Clínica (GPC) Reciín Nacido: Sepsis Neonatal Temprana. 2013. Accessed May 22, 2022. https://www.minsalud.gov.co/sites/rid/Lists/BibliotecaDigital/RIDE/INEC/IETS/GPC_Completa_Sepsis.pdf
• 6 National Institute for Health and Care Excellence (NICE). Neonatal Infection (Early Onset): Antibiotics for Prevention and Treatment. NICE; 2012:222. Accessed July 14, 2022 at: https://www.nice.org.uk/guidance/cg149
• 7 Ji H, Bridges M, Pesek E, Graham K, Tan L, Chabra S. Acute funisitis correlates with the risk of early-onset sepsis in term newborns assessed using the Kaiser Sepsis Calculator. Pediatr Dev Pathol 2019; 22 (06) 523-531
• 8 Eason J, Ward H, Danko O, Richardson K, Vaitkute R, McKeon-Carter R. Early-onset sepsis: can we screen fewer babies safely?. Arch Dis Child 2021; 106 (01) 86-88
• 9 Leonardi BM, Binder M, Griswold KJ, Yalcinkaya GF, Walsh MC. Utilization of a neonatal early-onset sepsis calculator to guide initial newborn management. Pediatr Qual Saf 2019; 4 (05) e214
• 10 Akangire G, Simpson E, Weiner J, Noel-MacDonnell J, Petrikin J, Sheehan M. Implementation of the neonatal sepsis calculator in early-onset sepsis and maternal chorioamnionitis. Adv Neonatal Care 2020; 20 (01) 25-32
• 11 Hershkovich-Shporen C, Ujirauli N, Oren S. et al. Not all newborns born to mothers with clinical chorioamnionitis need to be treated. J Matern Fetal Neonatal Med 2021; 34 (12) 1949-1954
• 12 Goel N, Shrestha S, Smith R. et al. Screening for early onset neonatal sepsis: NICE guidance-based practice versus projected application of the Kaiser Permanente sepsis risk calculator in the UK population. Arch Dis Child Fetal Neonatal Ed 2020; 105 (02) 118-122
• 13 Sharma V, Adkisson C, Gupta K. Managing infants exposed to maternal chorioamnionitis by the use of early-onset sepsis calculator. Glob Pediatr Health 2019; 6: X19833711
• 14 Bridges M, Pesek E, McRae M, Chabra S. Use of an early onset-sepsis calculator to decrease unnecessary NICU admissions and increase exclusive breastfeeding. J Obstet Gynecol Neonatal Nurs 2019; 48 (03) 372-382
• 15 Helmbrecht AR, Marfurt S, Chaaban H. Systematic review of the effectiveness of the neonatal early-onset sepsis calculator. J Perinat Neonatal Nurs 2019; 33 (01) 82-88
• 16 Strunk T, Buchiboyina A, Sharp M, Nathan E, Doherty D, Patole S. Implementation of the neonatal sepsis calculator in an Australian tertiary perinatal centre. Neonatology 2018; 113 (04) 379-382
• 17 Achten NB, Visser DH, Tromp E, Groot W, van Goudoever JB, Plötz FB. Early onset sepsis calculator implementation is associated with reduced healthcare utilization and financial costs in late preterm and term newborns. Eur J Pediatr 2020; 179 (05) 727-734
• 18 Warren S, Garcia M, Hankins C. Impact of neonatal early-onset sepsis calculator on antibiotic use within two tertiary healthcare centers. J Perinatol 2017; 37 (04) 394-397
• 19 Kuzniewicz MW, Puopolo KM, Fischer A. et al. A quantitative, risk-based approach to the management of neonatal early-onset sepsis. JAMA Pediatr 2017; 171 (04) 365-371
• 20 Deshmukh M, Mehta S, Patole S. Sepsis calculator for neonatal early onset sepsis - a systematic review and meta-analysis. J Matern Fetal Neonatal Med 2021; 34 (11) 1832-1840
• 21 Sloane AJ, Coleman C, Carola DL. et al. Use of a modified early-onset sepsis risk calculator for neonates exposed to chorioamnionitis. J Pediatr 2019; 213: 52-57
• 22 Al-Lawama M, AlZaatreh A, Elrajabi R, Abdelhamid S, Badran E. Prolonged Rupture of membranes, neonatal outcomes and management guidelines. J Clin Med Res 2019; 11 (05) 360-366
• 23 Beavers JB, Bai S, Perry J, Simpson J, Peeples S. Implementation and evaluation of the early-onset sepsis risk calculator in a high-risk university nursery. Clin Pediatr (Phila) 2018; 57 (09) 1080-1085
• 24 Escobar GJ, Li DK, Armstrong MA. et al. Neonatal sepsis workups in infants >/=2000 grams at birth: a population-based study. Pediatrics 2000; 106 (2 Pt 1): 256-263
• 25 Allan D. Sample size requirements for interval estimation of the intraclass kappa statistic. Commun Stat Simul Comput 1999; 28 (02) 415-429
• 26 Kuzniewicz MW, Walsh EM, Li S, Fischer A, Escobar GJ. Development and Implementation of an early-onset sepsis calculator to guide antibiotic management in late preterm and term neonates. Jt Comm J Qual Patient Saf 2016; 42 (05) 232-239
• 27 Puopolo KM, Draper D, Wi S. et al. Estimating the probability of neonatal early-onset infection on the basis of maternal risk factors. Pediatrics 2011; 128 (05) e1155-e1163
• 28 Morris R, Jones S, Banerjee S. et al. Comparison of the management recommendations of the Kaiser Permanente neonatal early-onset sepsis risk calculator (SRC) with NICE guideline CG149 in infants ≥34 weeks' gestation who developed early-onset sepsis. Arch Dis Child Fetal Neonatal Ed 2020; 105 (06) 581-586
• 29 Rallis D, Balomenou F, Karantanou K, Kappatou K, Tzoufi M, Giapros V. A comparison between risk-factor guidance for neonatal early-onset sepsis and Kaiser Permanente sepsis risk calculator in a Greek cohort. Early Hum Dev 2021; 155: 105331
• 30 Okike IO, Lamont RF, Heath PT. Do we really need to worry about Listeria in newborn infants?. Pediatr Infect Dis J 2013; 32 (04) 405-406
• 31 Wang Z, Tao X, Liu S, Zhao Y, Yang X. An update review on Listeria infection in pregnancy. Infect Drug Resist 2021; 14: 1967-1978
• 32 Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics 1977; 33 (01) 159-174
• 33 Benitz WE, Achten NB. Technical assessment of the neonatal early-onset sepsis risk calculator. Lancet Infect Dis 2021; 21 (05) e134-e140
• 34 Mukhopadhyay S, Taylor JA, Von Kohorn I. et al. Variation in sepsis evaluation across a national network of nurseries. Pediatrics 2017; 139 (03) e20162845