Point-of-Care Mobile Application to Guide Health Care Professionals in Conducting Substance Use Screening and Intervention: A Mixed-Methods User Experience Study

Megan A. O'Grady; Sandeep Kapoor; Evan Gilmer; Charles J. Neighbors; Joseph Conigliaro; Nancy Kwon; Jon Morgenstern

doi:10.1055/s-0039-1684002

ACI Open, Table of Contents

CC BY 4.0 · ACI open 2019; 03(01): e1-e12
DOI: 10.1055/s-0039-1684002

Original Article

Georg Thieme Verlag KG Stuttgart · New York

Point-of-Care Mobile Application to Guide Health Care Professionals in Conducting Substance Use Screening and Intervention: A Mixed-Methods User Experience Study

Megan A. O'Grady

¹Health Services Research, Center on Addiction, New York, New York, United States

,

Sandeep Kapoor

²Northwell Health, New Hyde Park, New York, United States

³Zucker School of Medicine at Hofstra/Northwell, Hempstead, New York, United States

,

Evan Gilmer

¹Health Services Research, Center on Addiction, New York, New York, United States

,

Charles J. Neighbors

¹Health Services Research, Center on Addiction, New York, New York, United States

,

Joseph Conigliaro

²Northwell Health, New Hyde Park, New York, United States

³Zucker School of Medicine at Hofstra/Northwell, Hempstead, New York, United States

,

Nancy Kwon

²Northwell Health, New Hyde Park, New York, United States

³Zucker School of Medicine at Hofstra/Northwell, Hempstead, New York, United States

,

Jon Morgenstern

²Northwell Health, New Hyde Park, New York, United States

³Zucker School of Medicine at Hofstra/Northwell, Hempstead, New York, United States

› Author Affiliations

Abstract

Background Well-documented barriers have limited the widespread, sustained adoption of screening and intervention for substance use problems in health care settings. mHealth applications may address provider-related barriers; however, there is limited research on development and user experience of such applications.

Objective This user experience study examines a provider-focused point-of-care app for substance use screening and intervention in health care settings.

Method This mixed-methods study included think-aloud tasks, task success ratings, semistructured interviews, and usability questionnaires (e.g., System Usability Scale [SUS]) to examine user experience among 12 health coaches who provide substance use services in emergency department and primary care settings.

Results The average rate of successful task completion was 94% and the mean SUS score was 76 out of 100. Qualitative data suggested the app enhanced participants' capability to complete tasks efficiently and effectively. Participants reported being satisfied with the app's features, content, screen layout, and navigation and felt it was easy to learn and could benefit patient interactions. Despite overwhelmingly positive user experience reports, there were some concerns that the app could negatively affect patient interactions due to reductions in eye contact and ability to build rapport.

Conclusion Using the “Fit between Individuals, Task, and Technology” framework to guide interpretation, overall results indicate acceptable user experience and usability for this provider-focused point-of-care mobile app for substance use screening and intervention as well as favorable potential for adoption by health care practitioners. Such mobile health technologies may help to address well-known challenges related to implementing substance use services in health care settings.

Keywords

SBIRT - mHealth - screening - substance use - usability - user experience

Full Text

References

References
1 Naeem F, Gire N, Xiang S. , et al. Reporting and understanding the safety and adverse effect profile of mobile apps for psychosocial interventions: An update. World J Psychiatry 2016; 6 (02) 187-191
2 Ventola CL. Mobile devices and apps for health care professionals: uses and benefits. P&T 2014; 39 (05) 356-364
3 U.S. Department of Health and Human Services (HHS); Office of the Surgeon General. Facing addiction in America: the Surgeon General's report on alcohol, drugs, and health. Washington, DC: HHS; 2016
4 McKnight-Eily LR, Okoro CA, Mejia R. , et al. Screening for excessive alcohol use and brief counseling of adults - 17 states and the District of Columbia, 2014. MMWR Morb Mortal Wkly Rep 2017; 66 (12) 313-319
5 Sacks S, Gotham HJ, Johnson K, Padwa H, Murphy DM, Krom L. Integrating substance use disorder and health care services in an era of health reform: models, interventions, and implementation strategies. Am J Med Res 2016; 3 (01) 75-124
6 Álvarez-Bueno C, Rodríguez-Martín B, García-Ortiz L, Gómez-Marcos MA, Martínez-Vizcaíno V. Effectiveness of brief interventions in primary health care settings to decrease alcohol consumption by adult non-dependent drinkers: a systematic review of systematic reviews. Prev Med 2015; 76 (Suppl): S33-S38
7 Glass JE, Andréasson S, Bradley KA. , et al. Rethinking alcohol interventions in health care: a thematic meeting of the International Network on Brief Interventions for Alcohol & Other Drugs (INEBRIA). Addict Sci Clin Pract 2017; 12 (01) 14
8 SAMHSA. Systems-level implementation of Screening, Brief Intervention, and Referral to Treatment (SBIRT). Technical Assistance Publication (TAP) Series 33. Rockville, MD: Substance Abuse and Mental Health Services Administration; 2013
9 Johnson M, Jackson R, Guillaume L, Meier P, Goyder E. Barriers and facilitators to implementing screening and brief intervention for alcohol misuse: a systematic review of qualitative evidence. J Public Health (Oxf) 2011; 33 (03) 412-421
10 Nilsen P. Brief alcohol intervention--where to from here? Challenges remain for research and practice. Addiction 2010; 105 (06) 954-959
11 Rahm AK, Boggs JM, Martin C. , et al. Facilitators and barriers to implementing Screening, Brief Intervention, and Referral to Treatment (SBIRT) in primary care in integrated health care settings. Subst Abus 2015; 36 (03) 281-288
12 Vendetti J, Gmyrek A, Damon D, Singh M, McRee B, Del Boca F. Screening, brief intervention and referral to treatment (SBIRT): implementation barriers, facilitators and model migration. Addiction 2017; 112 (Suppl. 02) 23-33
13 Harris SK, Knight JR. Putting the screen in screening: technology-based alcohol screening and brief interventions in medical settings. Alcohol Res 2014; 36 (01) 63-79
14 Marsch LA, Borodovsky JT. Technology-based interventions for preventing and treating substance use among youth. Child Adolesc Psychiatr Clin N Am 2016; 25 (04) 755-768
15 Bertholet N, Daeppen JB, McNeely J, Kushnir V, Cunningham JA. Smartphone application for unhealthy alcohol use: A pilot study. Subst Abus 2017; 38 (03) 285-291
16 Blow FC, Walton MA, Bohnert ASB. , et al. A randomized controlled trial of brief interventions to reduce drug use among adults in a low-income urban emergency department: the HealthiER You study. Addiction 2017; 112 (08) 1395-1405
17 Bonar EE, Walton MA, Cunningham RM. , et al. Computer-enhanced interventions for drug use and HIV risk in the emergency room: preliminary results on psychological precursors of behavior change. J Subst Abuse Treat 2014; 46 (01) 5-14
18 Cunningham RM, Chermack ST, Ehrlich PF. , et al. Alcohol interventions among underage drinkers in the ED: a randomized controlled trial. Pediatrics 2015; 136 (04) e783-e793
19 SBIRT [computer program]. Version 1.3. Center on Addiction and Northwell Health; 2015
20 Levesque D, Umanzor C, de Aguiar E. Stage-based mobile intervention for substance use disorders in primary care: development and test of acceptability. JMIR Med Inform 2018; 6 (01) e1
21 Satre DD, Ly K, Wamsley M, Curtis A, Satterfield J. A digital tool to promote alcohol and drug use Screening, Brief Intervention, and Referral to Treatment skill translation: a mobile app development and randomized controlled trial protocol. JMIR Res Protoc 2017; 6 (04) e55
22 Kumar S, Nilsen WJ, Abernethy A. , et al. Mobile health technology evaluation: the mHealth evidence workshop. Am J Prev Med 2013; 45 (02) 228-236
23 Tomlinson M, Rotheram-Borus MJ, Swartz L, Tsai AC. Scaling up mHealth: where is the evidence?. PLoS Med 2013; 10 (02) e1001382
24 Ammenwerth E, Iller C, Mahler C. IT-adoption and the interaction of task, technology and individuals: a fit framework and a case study. BMC Med Inform Decis Mak 2006; 6: 3
25 Honekamp W, Ostermann H. Evaluation of a prototype health information system using the FITT framework. Inform Prim Care 2011; 19 (01) 47-49
26 Noblin A, Shettian M, Cortelyou-Ward K, Schack Dugre J. Exploring physical therapists' perceptions of mobile application usage utilizing the FITT framework. Inform Health Soc Care 2017; 42 (02) 180-193
27 Sheehan B, Lee Y, Rodriguez M, Tiase V, Schnall R. A comparison of usability factors of four mobile devices for accessing healthcare information by adolescents. Appl Clin Inform 2012; 3 (04) 356-366
28 Seffah A, Kececi H, Donyaee M. QUIM: a framework for quantifying usability metrics in software quality models. Paper presented at: Second Asia-Pacific Conference on Quality Software; December 10–11, 2001 ; Hong Kong
29 Nielsen JE. Estimating the number of subjects needed for a thinking aloud test. Int J Hum Comput Stud 1994; 41 (03) 385-397
30 Nielsen JE, Landauer TK. A mathematical model of the finding of usability problems. Paper presented at: INTERCHI'93 Conference on Human Factors in Computing Systems; April 24–29, 1993 ; Amsterdam
31 Press A, DeStio C, McCullagh L, Kapoor S, Morley J, Conigliaro J. ; SBIRT NY-II Team. Usability testing of a national substance use screening tool embedded in electronic health records. JMIR Human Factors 2016; 3 (02) e18
32 Rubin J, Chisnell D. Handbook of Usability Testing: How to Plan, Design, and Conduct Effective Tests. Indianapolis, IN: Wiley; 2008
33 Tullis T, Albert B. Measuring the User Experience: Collecting, Analyzing, and Presenting Usability Metrics. Waltham, MA: Elsevier; 2013
34 Vilardaga R, Rizo J, Kientz JA, McDonell MG, Ries RK, Sobel K. User experience evaluation of a smoking cessation app in people with serious mental illness. Nicotine Tob Res 2016; 18 (05) 1032-1038
35 Wilson V, Neilson CJ. We want it now and we want it easy: usability testing of an online health library for healthcare practitioners. J Can Health Libr Assoc 2014; 32 (02) 51-59
36 Creswell JW, Creswell JD. Research Design: Qualitative, Quantitative, and Mixed Methods Approaches. Los Angeles, CA: Sage Publications; 2017
37 Brinck T, Gergle D, Wood SD. Usability for the Web: Designing Web Sites That Work. San Francisco, CA: Morgan Kaufmann; 2001
38 Jaspers MW, Steen T, van den Bos C, Geenen M. The think aloud method: a guide to user interface design. Int J Med Inform 2004; 73 (11–12): 781-795
39 Hennink MM, Kaiser BN, Marconi VC. Code saturation versus meaning saturation: how many interviews are enough?. Qual Health Res 2017; 27 (04) 591-608
40 Brooke J. SUS: a retrospective. J Usability Stud 2013; 8 (02) 29-40
41 Services USDoHaH. System Usability Scale. Available at: https://www.usability.gov/how-to-and-tools/methods/system-usability-scale.html . Accessed February 12, 2019
42 Lewis JR, Sauro J. The factor structure of the System Usability Scale. In: Kurosu M. ed. Human Computer and Design. HCD 2009. Lecture Notes in Computer Science. Berlin, Heidelberg: Springer; 2009. 5619: 94-103
43 Sauro J. A Practical Guide to the System Usability Scale: Background, Benchmarks & Best Practices. Denver, CO: Measuring Usability LLC; 2011
44 Borsci S, Federici S, Lauriola M. On the dimensionality of the System Usability Scale: a test of alternative measurement models. Cogn Process 2009; 10 (03) 193-197
45 Lewis J. IBM computer usability satisfaction questionnaires: psychometric evaluation and instructions for use. Int J Hum Comput Interact 1995; 7 (01) 57-78
46 Sauro J, Zarolia P. SUPR-Qm: a questionnaire to measure the mobile app user experience. J Usability Stud 2017; 13 (01) 17-37
47 Kortum P, Sorber M. Measuring the usability of mobile applications for phones and tablets. Int J Hum Comput Interact 2015; 31 (08) 518-529
48 Lewis JR. Measuring perceived usability: the CSUQ, SUS, and UMUX. Int J Hum Comput Interact 2018; 34 (12) 1148-1156
49 Bevan NJ. Extending quality in use to provide a framework for usability measurement. Paper presented at: 1st International Conference on Human Centered Design, held as Part of HCI International; 2009 ; San Diego, CA
50 Neal JW, Neal ZP, VanDyke E, Kornbluh M. Expediting the analysis of qualitative data in evaluation: a procedure for the rapid identification of themes from audio recordings (RITA). Am J Eval 2015; 36 (01) 118-132
51 Hsieh HF, Shannon SE. Three approaches to qualitative content analysis. Qual Health Res 2005; 15 (09) 1277-1288
52 Muhr T. ATLAS.ti: The knowledge Workbench: Visual Qualitative Data, Analysis, Management, Model Building: Short User's Manual. Berlin: Scientific Software Development; 1997
53 Erlingsson C, Brysiewicz P. A hands-on guide to doing content analysis. Afr J Emerg Med 2017; 7 (03) 93-99