External Validation of an Electronic Phenotyping Algorithm Detecting Attention to High Body Mass Index in Pediatric Primary Care

Abstract

Objectives The lack of feasible and meaningful measures of clinicians' behavior hinders efforts to assess and improve obesity management in pediatric primary care. In this study, we examined the external validity of a novel algorithm, previously validated in a single geographic region, using structured electronic health record (EHR) data to identify phenotypes of clinicians' attention to elevated body mass index (BMI) and weight-related comorbidities.

Methods We extracted structured EHR data for 300 randomly selected 6- to 12-year-old children with elevated BMI seen for well-child visits from June 2018 to May 2019 at pediatric primary care practices affiliated with Yale. Using diagnosis codes, laboratory orders, referrals, and medications adapted from the original algorithm, we categorized encounters as having evidence of attention to BMI only, weight-related comorbidities only, or both BMI and comorbidities. We evaluated the algorithm's sensitivity and specificity for detecting any attention to BMI and/or comorbidities using chart review as the reference standard.

Results The adapted algorithm yielded a sensitivity of 79.2% and specificity of 94.0% for identifying any attention to high BMI/comorbidities in clinical documentation. Of 86 encounters labeled as “no attention” by the algorithm, 83% had evidence of attention in free-text components of the progress note. The likelihood of classification as “any attention” by both chart review and the algorithm varied by BMI category and by clinician type (p < 0.001).

Conclusion The electronic phenotyping algorithm had high specificity for detecting attention to high BMI and/or comorbidities in structured EHR inputs. The algorithm's performance may be improved by incorporating unstructured data from clinical notes.

Keywords

data validation and verification - primary care - pediatrics - specific conditions - quality - EHR systems

Data Availability Statement

To protect the privacy of individuals included in the study, the data underlying this article cannot be shared publicly. The data can be shared on reasonable request to the corresponding author.

Protection of Human and Animal Subjects

This study was approved by the Yale Human Research Protection Program.

Publication History

Received: 15 November 2023

Accepted: 13 February 2024

Article published online:
28 August 2024

• References

• 1 Hales CM, Fryar CD, Carroll MD, Freedman DS, Ogden CL. Trends in obesity and severe obesity prevalence in US youth and adults by sex and age, 2007-2008 to 2015-2016. JAMA 2018; 319 (16) 1723-1725
• 2 Skinner ACRS, Ravanbakht SN, Skelton JA, Perrin EM, Armstrong SC. Prevalence of obesity and severe obesity in US children, 1999-2016. Pediatrics 2018; 141 (03) e20173459
• 3 Hu K, Staiano AE. Trends in obesity prevalence among children and adolescents aged 2 to 19 years in the US from 2011 to 2020. JAMA Pediatr 2022; 176 (10) 1037-1039
• 4 Grossman DC, Bibbins-Domingo K, Curry SJ. et al; US Preventive Services Task Force. Screening for obesity in children and adolescents: US Preventive Services Task Force recommendation statement. JAMA 2017; 317 (23) 2417-2426
• 5 Hampl SE, Hassink SG, Skinner AC. et al. Clinical practice guideline for the evaluation and treatment of children and adolescents with obesity. Pediatrics 2023; 151 (02) e2022060640
• 6 Ackard DM, Neumark-Sztainer D. Health care information sources for adolescents: age and gender differences on use, concerns, and needs. J Adolesc Health 2001; 29 (03) 170-176
• 7 Kao TA, Ling J, Hawn R, Vu C. The effects of motivational interviewing on children's body mass index and fat distributions: a systematic review and meta-analysis. Obes Rev 2021; 22 (10) e13308
• 8 Imoisili OE, Goodman AB, Dooyema CA, Harrison MR, Belay B, Park S. Screening and referral for childhood obesity: adherence to the U.S. Preventive Services Task Force recommendation. Am J Prev Med 2019; 56 (02) 179-186
• 9 Sharifi M, Rifas-Shiman SL, Marshall R. et al. Evaluating the implementation of expert committee recommendations for obesity assessment. Clin Pediatr (Phila) 2013; 52 (02) 131-138
• 10 Reyes I. An evaluation of the identification and management of overweight and obesity in a pediatric clinic. J Pediatr Health Care 2015; 29 (05) e9-e14
• 11 Lingren T, Thaker V, Brady C. et al. Developing an algorithm to detect early childhood obesity in two tertiary pediatric medical centers. Appl Clin Inform 2016; 7 (03) 693-706
• 12 Camp NL, Robert RC, Dawes CS, Kelly KP. Identification of overweight and obesity in low-income minority children by pediatric providers and child characteristics associated with underrecognition. J Pediatr Health Care 2019; 33 (02) 162-168
• 13 Taveras EM, Marshall R, Kleinman KP. et al. Comparative effectiveness of childhood obesity interventions in pediatric primary care: a cluster-randomized clinical trial. JAMA Pediatr 2015; 169 (06) 535-542
• 14 Taveras EM, Marshall R, Sharifi M. et al. Comparative effectiveness of clinical-community childhood obesity interventions: a randomized clinical trial. JAMA Pediatr 2017; 171 (08) e171325
• 15 Weight Assessment and Counseling for Nutrition and Physical Activity for Children/Adolescents (WCC). Accessed May 20, 2020 at: https://www.ncqa.org/hedis/measures/weight-assessment-and-counseling-for-nutrition-and-physical-activity-for-children-adolescents/
• 16 Banda JM, Seneviratne M, Hernandez-Boussard T, Shah NH. Advances in electronic phenotyping: from rule-based definitions to machine learning models. Annu Rev Biomed Data Sci 2018; 1: 53-68
• 17 Hewner S, Smith E, Sullivan SS. Identifying high-need primary care patients using nursing knowledge and machine learning methods. Appl Clin Inform 2023; 14 (03) 408-417
• 18 Joseph A, Mullett C, Lilly C. et al. Coronary artery disease phenotype detection in an academic hospital system setting. Appl Clin Inform 2021; 12 (01) 10-16
• 19 Samal L, Wu E, Aaron S. et al. Refining clinical phenotypes to improve clinical decision support and reduce alert fatigue: a feasibility study. Appl Clin Inform 2023; 14 (03) 528-537
• 20 Turer CB, Skinner CS, Barlow SE. Algorithm to detect pediatric provider attention to high BMI and associated medical risk. J Am Med Inform Assoc 2019; 26 (01) 55-60
• 21 Turer CB, Barlow SE, Sarwer DB. et al. Association of clinician behaviors and weight change in school-aged children. Am J Prev Med 2019; 57 (03) 384-393
• 22 Flegal KM, Wei R, Ogden CL, Freedman DS, Johnson CL, Curtin LR. Characterizing extreme values of body mass index-for-age by using the 2000 Centers for Disease Control and Prevention growth charts. Am J Clin Nutr 2009; 90 (05) 1314-1320
• 23 Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics 1977; 33 (01) 159-174
• 24 Lydecker JA, Grilo CM. The missed diagnosis and misdiagnosis of pediatric obesity. Psychother Psychosom 2017; 86 (03) 173-174
• 25 Young EL. Increasing diagnosis and treatment of overweight and obese pediatric patients. Clin Pediatr (Phila) 2015; 54 (14) 1359-1365
• 26 Reed M, Cygan H, Lui K, Mullen M. Identification, prevention, and management of childhood overweight and obesity in a pediatric primary care center. Clin Pediatr (Phila) 2016; 55 (09) 860-866
• 27 Kennell Jr TI, Willig JH, Cimino JJ. Clinical informatics researcher's desiderata for the data content of the next generation electronic health record. Appl Clin Inform 2017; 8 (04) 1159-1172
• 28 Zayas-Cabán T, Okubo TH, Posnack S. Priorities to accelerate workflow automation in health care. J Am Med Inform Assoc 2022; 30 (01) 195-201