Download PDF
CC BY-NC-ND 4.0 · Appl Clin Inform 2023; 14(01): 091-107
DOI: 10.1055/s-0042-1760631
Special Section on Patient Engagement in Informatics

Implementing an Electronic Patient-Reported Outcome and Decision Support Tool in Early Intervention

Sabrin Rizk‡   †
1   Children's Participation in Environment Research Lab, College of Applied Health Sciences, University of Illinois Chicago, Chicago, Illinois, United States
2   Department of Occupational Therapy, University of Illinois Chicago, Chicago, Illinois, United States
,
Vera C. Kaelin‡   †
1   Children's Participation in Environment Research Lab, College of Applied Health Sciences, University of Illinois Chicago, Chicago, Illinois, United States
3   Program in Rehabilitation Sciences, College of Applied Health Sciences, University of Illinois Chicago, Chicago, Illinois, United States
,
Julia Gabrielle C. Sim‡   †
1   Children's Participation in Environment Research Lab, College of Applied Health Sciences, University of Illinois Chicago, Chicago, Illinois, United States
,
Natalie J. Murphy
4   Department of Health Systems, Management, and Policy, University of Colorado, Aurora, Colorado, United States
,
Beth M. McManus
4   Department of Health Systems, Management, and Policy, University of Colorado, Aurora, Colorado, United States
,
Natalie E. Leland
5   Department of Occupational Therapy, University of Pittsburgh, Pittsburgh, Pennsylvania, United States
,
Ashley Stoffel
2   Department of Occupational Therapy, University of Illinois Chicago, Chicago, Illinois, United States
,
Lesly James
6   Department of Occupational Therapy, Lenoir-Rhyne University, Columbia, South Carolina, United States
,
Kris Barnekow
7   Department of Occupational Therapy, University of Wisconsin Milwaukee, Milwaukee, Wisconsin, United States
,
Elizabeth Lerner Papautsky‡   ††
8   Department of Biomedical and Health Information Sciences, University of Illinois Chicago, Chicago, Illinois, United States
,
Mary A. Khetani‡   ††
1   Children's Participation in Environment Research Lab, College of Applied Health Sciences, University of Illinois Chicago, Chicago, Illinois, United States
2   Department of Occupational Therapy, University of Illinois Chicago, Chicago, Illinois, United States
3   Program in Rehabilitation Sciences, College of Applied Health Sciences, University of Illinois Chicago, Chicago, Illinois, United States
9   CanChild Centre for Childhood Disability Research, School of Rehabilitation Science, McMaster University, Hamilton, Ontario, Canada
› Author Affiliations Funding This work is dedicated to our esteemed colleague Kelly Kearns, an amazing service coordinator, who took part in PROSPECT. Kelly passed away from cancer in May 2022. She significantly contributed to this work and will be remembered for her commitment and passion to ensure that family priorities are centered in EI services. This work is funded by American Occupational Therapy Foundation (grant no.: AOTFIR20KHETANI), with additional support from the Agency for Healthcare Research and Quality (grant no.: 5R01HS027583-02), and the Dean's Scholar Fellowship (V. Kaelin), Bridge to Faculty Scholar Program (S. Rizk) and the Honors College Research Grant and Chancellor's Undergraduate Research Award (J. Sim) from the University of Illinois Chicago. The project described, including use of Research Electronic Data Capture (REDCap), was supported by the National Center for Advancing Translational Sciences, National Institutes of Health, through Grant UL1TR002003. The content is solely the responsibility of the authors and does not necessarily represent the official views of the funding agencies. The authors thank members of the High Value Early Intervention Research Group and the American Occupational Therapy Association's Early Childhood Community of Practice for their expertise in shaping the protocol for data collection, and to Vivian Villegas, Zurisadai Salgado, and Marlene Angulo for feedback on preliminary results and figures/tables.
› Further Information
   Permissions and Reprints

Abstract

Objective The aim of the study is to identify and prioritize early intervention (EI) stakeholders' perspectives of supports and barriers to implementing the Young Children's Participation and Environment Measure (YC-PEM), an electronic patient-reported outcome (e-PRO) tool, for scaling its implementation across multiple local and state EI programs.

Methods An explanatory sequential (quan > QUAL) mixed-methods study was conducted with EI families (n = 6), service coordinators (n = 9), and program leadership (n = 7). Semi-structured interviews and focus groups were used to share select quantitative pragmatic trial results (e.g., percentages for perceived helpfulness of implementation strategies) and elicit stakeholder perspectives to contextualize these results. Three study staff deductively coded transcripts to constructs in the Consolidated Framework for Implementation Research (CFIR). Data within CFIR constructs were inductively analyzed to generate themes that were rated by national early childhood advisors for their relevance to longer term implementation.

Results All three stakeholder groups (i.e., families, service coordinators, program leadership) identified thematic supports and barriers across multiple constructs within each of four CFIR domains: (1) Six themes for “intervention characteristics,” (2) Six themes for “process,” (3) three themes for “inner setting,” and (4) four themes for “outer setting.” For example, all stakeholder groups described the value of the YC-PEM e-PRO in forging connections and eliciting meaningful information about family priorities for efficient service plan development (“intervention characteristics”). Stakeholders prioritized reaching families with diverse linguistic preferences and user navigation needs, further tailoring its interface with automated data capture and exchange processes (“process”); and fostering a positive implementation climate (“inner setting”). Service coordinators and program leadership further articulated the value of YC-PEM e-PRO results for improving EI access (“outer setting”).

Conclusion Results demonstrate the YC-PEM e-PRO is an evidence-based intervention that is viable for implementation. Optimizations to its interface are needed before undertaking hybrid type-2 and 3 multisite trials to test these implementation strategies across state and local EI programs with electronic data capture capabilities and diverse levels of organizational readiness and resources for implementation.

Keywords

early intervention - assessment - goal setting - implementation strategies - family engagement - facilitators and barriers - children - patient-reported outcomes

Protection of Human and Animal Subjects

This manuscript does not include any research on human subjects.


† These authors have contributed equally to this work and share first authorship.


†† These authors have contributed equally to this work and share senior authorship.


Supplementary Material



Publication History

Received: 14 July 2022

Accepted: 09 December 2022

Article published online:
01 February 2023

© 2023. The Author(s). This is an open access article published by Thieme under the terms of the Creative Commons Attribution-NonDerivative-NonCommercial License, permitting copying and reproduction so long as the original work is given appropriate credit. Contents may not be used for commercial purposes, or adapted, remixed, transformed or built upon. (https://creativecommons.org/licenses/by-nc-nd/4.0/)

Georg Thieme Verlag KG
Rüdigerstraße 14, 70469 Stuttgart, Germany

 

  • References

  • 1 Individuals with Disabilities Education Act, Reauthorization. Public Law 108–446. 2004: 99–457
    PubMed
  • 2 US Department of Education. 2020 Part C SPP/ APR indicator analysis booklet. Accessed June 30, 2022, at: https://sites.ed.gov/idea/files/PartC-IndicatorAnalysis-FFY2018.pdf
    PubMedGoogle Scholar
  • 3 Magnusson DM, Khetani MA. Early intervention. In: Blum NJ, Jimenez M. eds. Developmental-Behavioral Pediatrics. 5th ed.. Elsevier;
    Google Scholar
  • 4 Magnusson DM, Mistry KB. Racial and ethnic disparities in unmet need for pediatric therapy services: the role of family-centered care. Acad Pediatr 2017; 17 (01) 27-33
    CrossrefPubMedGoogle Scholar
  • 5 McManus BM, Richardson Z, Schenkman M. et al. Child characteristics and early intervention referral and receipt of services: a retrospective cohort study. BMC Pediatr 2020; 20 (01) 84
    CrossrefPubMedGoogle Scholar
  • 6 Richardson ZS, Khetani MA, Scully E, Dooling-Litfin J, Murphy NJ, McManus BM. Social and functional characteristics of receipt and service use intensity of core early intervention services. Acad Pediatr 2019; 19 (07) 722-732
    CrossrefPubMedGoogle Scholar
  • 7 Jimenez ME, Fiks AG, Shah LR. et al. Factors associated with early intervention referral and evaluation: a mixed methods analysis. Acad Pediatr 2014; 14 (03) 315-323
    CrossrefPubMedGoogle Scholar
  • 8 Boop C, Cahill SM, Davis C. et al. Occupational therapy practice framework: domain and process 4th ed. Am J Occup Ther 2020; 74 (02) 7412410010p1-7412410010p87
    CrossrefPubMedGoogle Scholar
  • 9 Ronen GM, Rosenbaum PL, Streiner DL. Patient perspectives in pediatric neurology: a critical shift in the paradigm of outcome measurement. Dev Med Child Neurol 2022; 64 (02) 149-155
    CrossrefPubMedGoogle Scholar
  • 10 Stern BZ. Clinical potential of patient-reported outcome measures in occupational therapy. Am J Occup Ther 2022; 76 (02) 1-5
    CrossrefPubMedGoogle Scholar
  • 11 Keeney T, Kumar A, Erler KS, Karmarkar AM. Making the case for patient-reported outcome measures in big-data rehabilitation research: implications for optimizing patient-centered care. Arch Phys Med Rehabil 2022; 103 (5S): S140-S145
    CrossrefPubMedGoogle Scholar
  • 12 Austin E, LeRouge C, Hartzler AL, Chung AE, Segal C, Lavallee DC. Opportunities and challenges to advance the use of electronic patient-reported outcomes in clinical care: a report from AMIA workshop proceedings. JAMIA Open 2019; 2 (04) 407-410
    CrossrefPubMedGoogle Scholar
  • 13 Jarvis JM, Fink EL. More than a feeling: understanding function and health related quality of life after pediatric neurocritical illness. Neurocrit Care 2021; 35 (02) 308-310
    CrossrefPubMedGoogle Scholar
  • 14 Hsiao CJ, Dymek C, Kim B, Russell B. Advancing the use of patient-reported outcomes in practice: understanding challenges, opportunities, and the potential of health information technology. Qual Life Res 2019; 28 (06) 1575-1583
    CrossrefPubMedGoogle Scholar
  • 15 Kwok EYL, Rosenbaum P, Thomas-Stonell N, Cunningham BJ. Strengths and challenges of the COSMIN tools in outcome measures appraisal: a case example for speech-language therapy. Int J Lang Commun Disord 2021; 56 (02) 313-329
    CrossrefPubMedGoogle Scholar
  • 16 Khetani MA, Coster W, Law MC, Bedell GM. Young Children's Participation and Environment Measure. Colorado State University (copyright to authors); 2013
    Google Scholar
  • 17 Khetani MA, McManus BM, Arestad K. et al. Technology-based functional assessment in early childhood intervention: a pilot study. Pilot Feasibility Stud 2018; 4 (01) 65
    CrossrefPubMedGoogle Scholar
  • 18 Albrecht EC, Kaelin VC, Rigau BL. et al; High Value Early Intervention Research Group. Pilot implementation of an electronic patient-reported outcome measure for planning and monitoring participation-focused care in early intervention. BMC Med Inform Decis Mak 2020; 20 (01) 199
    CrossrefPubMedGoogle Scholar
  • 19 Khetani MA, Graham JE, Davies PL, Law MC, Simeonsson RJ. Psychometric properties of the young children's participation and environment measure. Arch Phys Med Rehabil 2015; 96 (02) 307-316
    CrossrefPubMedGoogle Scholar
  • 20 Lim CY, Law M, Khetani M, Rosenbaum P, Pollock N. Psychometric evaluation of the young children's participation and environment measure (YC-PEM) for use in Singapore. Phys Occup Ther Pediatr 2018; 38 (03) 316-328
    CrossrefPubMedGoogle Scholar
  • 21 Khetani MA. Validation of environmental content in the Young Children's Participation and Environment Measure. Arch Phys Med Rehabil 2015; 96 (02) 317-322
    CrossrefPubMedGoogle Scholar
  • 22 Benjamin TE, Lucas-Thompson RG, Little LM, Davies PL, Khetani MA. Participation in early childhood educational environments for young children with and without developmental disabilities and delays: a mixed methods study. Phys Occup Ther Pediatr 2017; 37 (01) 87-107
    CrossrefPubMedGoogle Scholar
  • 23 Khetani MA, Cohn ES, Orsmond GI, Law MC, Coster WJ. Parent perspectives of participation in home and community activities when receiving Part C early intervention services. Top Early Child Spec Educ 2013; 32 (04) 234-245
    CrossrefPubMedGoogle Scholar
  • 24 Khetani M, Marley J, Baker M. et al. Validity of the participation and environment measure for children and youth (PEM-CY) for health impact assessment (HIA) in sustainable development projects. Disabil Health J 2014; 7 (02) 226-235
    CrossrefPubMedGoogle Scholar
  • 25 Gurga A, Greif A, Sidorowych C, Magasi S, Khetani MA. Success Story: Designing a Participation-Focused Report Yield. Accessed July 1, 2022, at: https://www.canchild.ca/en/resources/223-young-children-s-participation-and-environment-measure-ycpem
    PubMedGoogle Scholar
  • 26 Harrison JD, Auerbach AD, Anderson W. et al. Patient stakeholder engagement in research: a narrative review to describe foundational principles and best practice activities. Health Expect 2019; 22 (03) 307-316
    CrossrefPubMedGoogle Scholar
  • 27 Martinez J, Wong C, Piersol CV, Bieber DC, Perry BL, Leland NE. Stakeholder engagement in research: a scoping review of current evaluation methods. J Comp Eff Res 2019; 8 (15) 1327-1341
    CrossrefPubMedGoogle Scholar
  • 28 Chaudoir SR, Dugan AG, Barr CHI. Measuring factors affecting implementation of health innovations: a systematic review of structural, organizational, provider, patient, and innovation level measures. Implement Sci 2013; 8 (01) 22
    CrossrefPubMedGoogle Scholar
  • 29 Khetani MA, Cliff AB, Schelly C, Daunhauer L, Anaby D. Decisional support algorithm for collaborative care planning using the Participation and Environment Measure for Children and Youth (PEM-CY): a mixed methods study. Phys Occup Ther Pediatr 2014; 35 (03) 231-252
    CrossrefPubMedGoogle Scholar
  • 30 Khetani MA, Lim HK, Corden ME. Caregiver input to optimize the design of a pediatric care planning guide for rehabilitation: descriptive study. JMIR Rehabil Assist Technol 2017; 4 (02) e10
    CrossrefPubMedGoogle Scholar
  • 31 Jarvis JM, Gurga AR, Lim H. et al. Caregiver strategy use to promote children's home participation after pediatric critical illness. Arch Phys Med Rehabil 2019; 100 (11) 2144-2150
    CrossrefPubMedGoogle Scholar
  • 32 Jarvis JM, Kaelin VC, Anaby D, Teplicky R, Khetani MA. Electronic participation-focused care planning support for families: a pilot study. Dev Med Child Neurol 2020; 62 (08) 954-961
    CrossrefPubMedGoogle Scholar
  • 33 Rigau BL, Scully EA, Dooling-Litfin JK, Murphy NJ, McManus BM, Khetani MA. Community engagement to pilot electronic patient-reported outcomes (e-PROs) in early intervention: lessons learned. J Clin Transl Sci 2018; 2 (01) 20-26
    CrossrefPubMedGoogle Scholar
  • 34 Anaby D, Khetani M, Piskur B. et al. Towards a paradigm shift in pediatric rehabilitation: accelerating the uptake of evidence on participation into routine clinical practice. Disabil Rehabil 2022; 44 (09) 1746-1757
    CrossrefPubMedGoogle Scholar
  • 35 Elbaum B, Celimli-Aksoy S. Developmental outcomes of children served in a Part C early intervention program. Infants Young Child 2022; 35 (01) 3-19
    CrossrefPubMedGoogle Scholar
  • 36 Khetani MA, Bosak D, Jarvis JM, Teplicky R. Young Children' s Participation and Environment User Guide (Version 1.2). Accessed July 1, 2022 at: https://www.canchild.ca/en/shop/23-yc-pem-young-children-s-participation-and-environment-measure
    PubMedGoogle Scholar
  • 37 McManus BM, Murphy NJ, Richardson Z, Khetani MA, Schenkman M, Morrato EH. Family-centred care in early intervention: examining caregiver perceptions of family-centred care and early intervention service use intensity. Child Care Health Dev 2020; 46 (01) 1-8
    CrossrefPubMedGoogle Scholar
  • 38 Papautsky EL, Patterson ES. Patients are knowledge workers in the clinical information space. Appl Clin Inform 2021; 12 (01) 133-140
    Article in Thieme ConnectPubMedGoogle Scholar
  • 39 Papautsky EL, Crandall B, Grome A, Greenberg JM. A case study of source triangulation. J Cogn Eng Decis Mak 2015; 9 (04) 347-358
    CrossrefPubMedGoogle Scholar
  • 40 Kaelin V, Villegas V, Chen YF. et al; High Value Early Intervention Research Group. Effectiveness and scalability of an electronic patient-reported outcome measure and decision support tool for family-centred and participation-focused early intervention: PROSPECT hybrid type 1 trial protocol. BMJ Open 2022; 12 (01) e051582
    CrossrefPubMedGoogle Scholar
  • 41 Creswell J, Clark V. Designing and Conducting Mixed Methods Research. 2nd ed.. Sage Publications, Inc.; 2011
    Google Scholar
  • 42 Curran GM, Bauer M, Mittman B, Pyne JM, Stetler C. Effectiveness-implementation hybrid designs: combining elements of clinical effectiveness and implementation research to enhance public health impact. Med Care 2012; 50 (03) 217-226
    CrossrefPubMedGoogle Scholar
  • 43 Landes SJ, McBain SA, Curran GM. Reprint of: an introduction to effectiveness-implementation hybrid designs. Psychiatry Res 2020; 283: 112630
    CrossrefPubMedGoogle Scholar
  • 44 Lambert SD, Loiselle CG. Combining individual interviews and focus groups to enhance data richness. J Adv Nurs 2008; 62 (02) 228-237
    CrossrefPubMedGoogle Scholar
  • 45 QSR International Pty Ltd NVivo. . Accessed July 1, 2022, at: https://www.qsrinternational.com/nvivo-qualitative-data-analysis-software/home
    PubMed
  • 46 Mayring P. Qualitative Content Analysis: Theoretical Foundation, Basic Procedures and Software Solution. Klagenfurt; 2014
    Google Scholar
  • 47 Damschroder LJ, Aron DC, Keith RE, Kirsh SR, Alexander JA, Lowery JC. Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implement Sci 2009; 4 (01) 50
    CrossrefPubMedGoogle Scholar
  • 48 Nilsen P. Making sense of implementation theories, models and frameworks. Implement Sci 2015; 10 (01) 53
    CrossrefPubMedGoogle Scholar
  • 49 Krueger RA, Casey MA. Focus Groups: A Practical Guide for Applied Research. 5th ed.. Sage Publications, Inc.; 2015
    Google Scholar
  • 50 US Department of Housing and Urban Development. Estimated Median Family Incomes for Fiscal Year (FY) 2021. Accessed July 1, 2022 at: https://www.huduser.gov/portal/datasets/il/il21/Medians2021.pdf
    PubMed
  • 51 Khetani MA, Kaelin VC, Kuznicki L, Bane J, Banks S. Academic-community partnership to improve early intervention. Collaborative Endeavors. Accessed July 1, 2022 at: https://collaborative-endeavors.simplecast.com/episodes/khetani-txJ9ZBT0
    PubMedGoogle Scholar
  • 52 Galvin HK, Petersen C, Subbian V, Solomonides A. Patients as agents in behavioral health research and service provision: recommendations to support the learning health system. Appl Clin Inform 2019; 10 (05) 841-848
    Article in Thieme ConnectPubMedGoogle Scholar
  • 53 Russ AL, Fairbanks RJ, Karsh BT, Militello LG, Saleem JJ, Wears RL. The science of human factors: separating fact from fiction. BMJ Qual Saf 2013; 22 (10) 802-808
    CrossrefPubMedGoogle Scholar
  • 54 Ratwani RM, Fairbanks RJ, Hettinger AZ, Benda NC. Electronic health record usability: analysis of the user-centered design processes of eleven electronic health record vendors. J Am Med Inform Assoc 2015; 22 (06) 1179-1182
    CrossrefPubMedGoogle Scholar
  • 55 Institute of Medicine (US) Committee on Quality of Health Care in America. Crossing the Quality Chasm: A New Health System for the 21st Century. National Academies Press: 2001
    Google Scholar
  • 56 Swoboda CM, DePuccio MJ, Fareed N, McAlearney AS, Walker DM. Patient portals: useful for whom and for what? A cross-sectional analysis of national survey data. Appl Clin Inform 2021; 12 (03) 573-581
    Article in Thieme ConnectPubMedGoogle Scholar
  • 57 van Leeuwen D, Mittelman M, Fabian L, Lomotan EA. Nothing for me or about me, without me: codesign of clinical decision support. Appl Clin Inform 2022; 13 (03) 641-646
    Article in Thieme ConnectPubMedGoogle Scholar
  • 58 Di Tosto G, Walker DM, Sieck CJ. et al. Examining the relationship between health literacy, health numeracy, and patient portal use. Appl Clin Inform 2022; 13 (03) 692-699
    Article in Thieme ConnectPubMedGoogle Scholar
  • 59 Tomas V, Srinivasan R, Kulkarni V, Teplicky R, Anaby D, Khetani M. A guiding process to culturally adapt assessments for participation-focused pediatric practice: the case of the Participation and Environment Measures (PEM). Disabil Rehabil 2022; 44 (21) 6497-6509
    CrossrefPubMedGoogle Scholar
  • 60 Lyman J. The participation and environment measure-plus: a useful new tool for caregivers and professionals. Dev Med Child Neurol 2020; 62 (08) 890-890
    CrossrefPubMedGoogle Scholar
  • 61 Rigau BL, Dooling-Litfin J, Scully E. et al. Building organizational capacity for research in early intervention. Zero Three 2019; 39 (06) 54-61
    PubMedGoogle Scholar
  • 62 King G, Servais M, Kertoy M. et al. A measure of community members' perceptions of the impacts of research partnerships in health and social services. Eval Program Plann 2009; 32 (03) 289-299
    CrossrefPubMedGoogle Scholar
  • 63 Graham F, Timothy E, Williman J, Levack W. Participation-focused practices in paediatric rehabilitation for children with neurodisability in New Zealand: an observational study using MAPi audit tool. Child Care Health Dev 2020; 46 (05) 552-562
    CrossrefPubMedGoogle Scholar
  • 64 Kolehmainen N, Marshall J, Hislop J. et al. Implementing participation-focused services: a study to develop the method for using audit and feedback in participation implementation (MAPi). Child Care Health Dev 2020; 46 (01) 37-45
    CrossrefPubMedGoogle Scholar
  • 65 Menon A, Cafaro T, Loncaric D. et al. Creation and validation of the PERFECT: a critical incident tool for evaluating change in the practices of health professionals. J Eval Clin Pract 2010; 16 (06) 1170-1175
    CrossrefPubMedGoogle Scholar
  • 66 Charles and Lynn Schusterman Family Foundation. More than numbers. A guide toward diversity, equity, and inclusion (DEI) in data collection. Accessed July 1, 2022 at: https://cloudspn.tfaforms.net/266232
    PubMedGoogle Scholar
  • 67 Nguyen OT, Hong YR, Alishahi Tabriz A, Hanna K, Turner K. Prevalence and factors associated with patient-requested corrections to the medical record through use of a patient portal: findings from a national survey. Appl Clin Inform 2022; 13 (01) 242-251
    Article in Thieme ConnectPubMedGoogle Scholar