An Exploratory, Population-Based, Mixed-Methods Program Evaluation of User Satisfaction of Services Provided by a Regional Extension Center (REC)

Summary

Objectives: To evaluate user satisfaction among practices receiving services provided by the Arizona Regional Extension Center (REC).

Methods: This program evaluation utilized a mixed-methods approach including: 1) a mail-based survey targeting all 489 REC member practices; and 2) a series of telephone-based focus groups using a convenience sample of rural and urban REC member practices. Targeted respondents were key contacts who handle interactions with the REC at each practice. Program evaluators at the University of Arizona and experts at Arizona Health-e Connection (AzHeC) created the questionnaires, focus group script, participant invitation and follow up documents via a collaborative process. Regression and Rasch analyses were used to identify key factors associated with satisfaction with REC and to assess questionnaire validity, respectively.

Results: Responses from both the focus groups and survey revealed that most of the respondents were satisfied with the current services, despite the presence of satisfaction gaps between practices of various characteristics: respondents that were clinicians, practices using web-based electronic health record systems (EHRs), and practices that had achieved Stage 1 Meaningful Use had a higher level of satisfaction compared with their respective counterparts. Focus group participants provided suggestions for improving REC services.

Conclusions: Most respondents reported being satisfied with REC services. Specialized representatives may be needed for practices at different stages of Meaningful Use to further improve REC satisfaction in order to facilitate more efficient adoption of EHRs.

Citation: Tang D, Rutala M, Ihde C, Bills A, Mollon L, Warholak T. An exploratory, population-based, mixedmethods program evaluation of user satisfaction of services provided by a regional extension center (REC). Appl Clin Inf 2014; 5: 1–24 http://dx.doi.org/10.4338/ACI-2013-06-RA-0037

Keywords

Health information technology - user satisfaction - mixed-methods study - regional extension center - meaningful use

• References

• 1 Chaudhry B, Wang J, Wu S. et al. Systematic review: impact of health information technology on quality, efficiency, and costs of medical care. Ann Intern Med 2006; 144: 742-752.
• 2 Parente ST, McCullough JS. Health information technology and patient safety: evidence from panel data. Health Aff 2009; 28: 357-360.
• 3 McCullough JS, Casey M, Moscovice I, Prasad S. The effect of health information technology on quality in U. S. hospitals. Health Aff 2010; 29: 647-654.
• 4 Davis K, Doty M, Shea K, Stremikis K. Health information technology and physician perceptions of quality of care and satisfaction. Health Policy 2009; 90: 239-246.
• 5 Goldzweig CL, Towfigh A, Maglione M, Shekelle PG. Costs and benefits of health information technology: new trends from the literature. Health Aff 2009; 28: w282-w293.
• 6 Jha AK. Meaningful use of electronic health records: The road ahead. JAMA 2010; 304: 1709-1710.
• 7 Centers for Medicare & Medicaid Services.. EHR Incentive Programs. Available at: http://www.cms.gov/Regulations-and-Guidance/Legislation/EHRIncentivePrograms/Eligible_Hospital_Information.html. Accessed October 13, 2013
• 8 Blumenthal D, Tavenner M. The meaningful use regulation for electronic health records. N Engl J Med 2010; 363: 501-504.
• 9 Hogan SO, Kissam SM. Measuring meaningful use. Health Aff 2010; 29: 601-606.
• 10 Buntin MB, Jain SH, Blumenthal D. Health information technology: laying the infrastructure for national health reform. Health Aff 2010; 29: 1214-1219.
• 11 Price JH, Dake JA, Murman J, Dimmig J, Akpanudo S. Power analysis in survey research: importance and use for health educators. Am J Health Educ 2005; 36: 202-207.
• 12 Coughlin SS. Recall bias in epidemiologic studies. J Clin Epidemiol 1990; 43: 87-91.
• 13 Gordon RJ. Arizona Rural Health Provider Atlas. Rural Health Office, Department of Family and Community Medicine, University of Arizona College of Medicine.. pp. 1-152 1987
• 14 Kennedy P. A Guide to Econometrics. Oxford: Blackwell.; 1992
• 15 Vittinghoff E, Glidden DV, Shiboski SC, McCulloch CE. Regression methods in biostatistics. New York: Springer,; 2005
• 16 Dooley LM, Lindner JR. The handling of nonresponse error. Human Resource Development Quarterly 2003; 14: 99-110.
• 17 Linacre J. Understanding Rasch measurement: Optimizing rating scale category effectiveness. J Appl Meas 2002; 3: 85-106.
• 18 Richards L. Handling qualitative data. California: Sage,; 2005
• 19 Maxson E, Jain S, Kendall M, Mostashari F, Blumenthal D. The Regional Extension Center Program: helping physicians meaningfully use health information technology. Ann Intern Med 2010; 153: 666-670.
• 20 Jamoom E, Beatty P, Bercovitz A. et al. Physician adoption of electronic health record systems: United States, 2011. NCHS data brief, no 98. Hyattsville, MD: National Center for Health Statistics,; 2012
• 21 McDonnell C, Werner K, Wendel L. Electronic Health Record Usability: Vendor Practices and Perspectives. AHRQ Publication No. 09(10)-0091–3-EF. Rockville, MD:: Agency for Healthcare Research and Quality,; 2010
• 22 Johnson WG, Harootunian G, Sama T. The use of electronic medical records and physicians’ attitudes towards a health information exchange. Phoenix, AZ: Arizona State University, Center for Health Information & Research,; 2012
• 23 Aharony L, Strasser S. Patient satisfaction: what we know about and what we still need to explore. Medical Care Review 1993; 50 (01) 49-79.
• 24 Ko HH, Zhang H, Telford JJ, Enns R. Factors influencing patient satisfaction when undergoing endoscopic procedures. Gastrointest Endosc 2009; 69: 883-891.
• 25 Johnson RB, Onwuegbuzie AJ. Mixed methods research: A research paradigm whose time has come. Educ Res 2004; 33: 14-26.