CC BY 4.0 · ACI open 2019; 03(01): e1-e12
DOI: 10.1055/s-0039-1684002
Original Article
Georg Thieme Verlag KG Stuttgart · New York

Point-of-Care Mobile Application to Guide Health Care Professionals in Conducting Substance Use Screening and Intervention: A Mixed-Methods User Experience Study

Megan A. O'Grady
1   Health Services Research, Center on Addiction, New York, New York, United States
,
Sandeep Kapoor
2   Northwell Health, New Hyde Park, New York, United States
3   Zucker School of Medicine at Hofstra/Northwell, Hempstead, New York, United States
,
Evan Gilmer
1   Health Services Research, Center on Addiction, New York, New York, United States
,
Charles J. Neighbors
1   Health Services Research, Center on Addiction, New York, New York, United States
,
Joseph Conigliaro
2   Northwell Health, New Hyde Park, New York, United States
3   Zucker School of Medicine at Hofstra/Northwell, Hempstead, New York, United States
,
Nancy Kwon
2   Northwell Health, New Hyde Park, New York, United States
3   Zucker School of Medicine at Hofstra/Northwell, Hempstead, New York, United States
,
Jon Morgenstern
2   Northwell Health, New Hyde Park, New York, United States
3   Zucker School of Medicine at Hofstra/Northwell, Hempstead, New York, United States
› Author Affiliations
Funding This work was supported by the Substance Abuse and Mental Health Services Administration (SAMHSA; Grant Number 5U79TI025102). Its contents are solely the responsibility of the authors and do not necessarily represent the official views of SAMHSA.
Further Information

Publication History

05 September 2018

14 January 2019

Publication Date:
27 March 2019 (online)

Abstract

Background Well-documented barriers have limited the widespread, sustained adoption of screening and intervention for substance use problems in health care settings. mHealth applications may address provider-related barriers; however, there is limited research on development and user experience of such applications.

Objective This user experience study examines a provider-focused point-of-care app for substance use screening and intervention in health care settings.

Method This mixed-methods study included think-aloud tasks, task success ratings, semistructured interviews, and usability questionnaires (e.g., System Usability Scale [SUS]) to examine user experience among 12 health coaches who provide substance use services in emergency department and primary care settings.

Results The average rate of successful task completion was 94% and the mean SUS score was 76 out of 100. Qualitative data suggested the app enhanced participants' capability to complete tasks efficiently and effectively. Participants reported being satisfied with the app's features, content, screen layout, and navigation and felt it was easy to learn and could benefit patient interactions. Despite overwhelmingly positive user experience reports, there were some concerns that the app could negatively affect patient interactions due to reductions in eye contact and ability to build rapport.

Conclusion Using the “Fit between Individuals, Task, and Technology” framework to guide interpretation, overall results indicate acceptable user experience and usability for this provider-focused point-of-care mobile app for substance use screening and intervention as well as favorable potential for adoption by health care practitioners. Such mobile health technologies may help to address well-known challenges related to implementing substance use services in health care settings.

Protection of Human and Animal Subjects

This study was conducted in compliance with all human subjects regulations. The study was reviewed and approved by institutional review boards of both the first and second authors.


 
  • References

  • 1 Naeem F, Gire N, Xiang S. , et al. Reporting and understanding the safety and adverse effect profile of mobile apps for psychosocial interventions: An update. World J Psychiatry 2016; 6 (02) 187-191
  • 2 Ventola CL. Mobile devices and apps for health care professionals: uses and benefits. P&T 2014; 39 (05) 356-364
  • 3 U.S. Department of Health and Human Services (HHS); Office of the Surgeon General. Facing addiction in America: the Surgeon General's report on alcohol, drugs, and health. Washington, DC: HHS; 2016
  • 4 McKnight-Eily LR, Okoro CA, Mejia R. , et al. Screening for excessive alcohol use and brief counseling of adults - 17 states and the District of Columbia, 2014. MMWR Morb Mortal Wkly Rep 2017; 66 (12) 313-319
  • 5 Sacks S, Gotham HJ, Johnson K, Padwa H, Murphy DM, Krom L. Integrating substance use disorder and health care services in an era of health reform: models, interventions, and implementation strategies. Am J Med Res 2016; 3 (01) 75-124
  • 6 Álvarez-Bueno C, Rodríguez-Martín B, García-Ortiz L, Gómez-Marcos MA, Martínez-Vizcaíno V. Effectiveness of brief interventions in primary health care settings to decrease alcohol consumption by adult non-dependent drinkers: a systematic review of systematic reviews. Prev Med 2015; 76 (Suppl): S33-S38
  • 7 Glass JE, Andréasson S, Bradley KA. , et al. Rethinking alcohol interventions in health care: a thematic meeting of the International Network on Brief Interventions for Alcohol & Other Drugs (INEBRIA). Addict Sci Clin Pract 2017; 12 (01) 14
  • 8 SAMHSA. Systems-level implementation of Screening, Brief Intervention, and Referral to Treatment (SBIRT). Technical Assistance Publication (TAP) Series 33. Rockville, MD: Substance Abuse and Mental Health Services Administration; 2013
  • 9 Johnson M, Jackson R, Guillaume L, Meier P, Goyder E. Barriers and facilitators to implementing screening and brief intervention for alcohol misuse: a systematic review of qualitative evidence. J Public Health (Oxf) 2011; 33 (03) 412-421
  • 10 Nilsen P. Brief alcohol intervention--where to from here? Challenges remain for research and practice. Addiction 2010; 105 (06) 954-959
  • 11 Rahm AK, Boggs JM, Martin C. , et al. Facilitators and barriers to implementing Screening, Brief Intervention, and Referral to Treatment (SBIRT) in primary care in integrated health care settings. Subst Abus 2015; 36 (03) 281-288
  • 12 Vendetti J, Gmyrek A, Damon D, Singh M, McRee B, Del Boca F. Screening, brief intervention and referral to treatment (SBIRT): implementation barriers, facilitators and model migration. Addiction 2017; 112 (Suppl. 02) 23-33
  • 13 Harris SK, Knight JR. Putting the screen in screening: technology-based alcohol screening and brief interventions in medical settings. Alcohol Res 2014; 36 (01) 63-79
  • 14 Marsch LA, Borodovsky JT. Technology-based interventions for preventing and treating substance use among youth. Child Adolesc Psychiatr Clin N Am 2016; 25 (04) 755-768
  • 15 Bertholet N, Daeppen JB, McNeely J, Kushnir V, Cunningham JA. Smartphone application for unhealthy alcohol use: A pilot study. Subst Abus 2017; 38 (03) 285-291
  • 16 Blow FC, Walton MA, Bohnert ASB. , et al. A randomized controlled trial of brief interventions to reduce drug use among adults in a low-income urban emergency department: the HealthiER You study. Addiction 2017; 112 (08) 1395-1405
  • 17 Bonar EE, Walton MA, Cunningham RM. , et al. Computer-enhanced interventions for drug use and HIV risk in the emergency room: preliminary results on psychological precursors of behavior change. J Subst Abuse Treat 2014; 46 (01) 5-14
  • 18 Cunningham RM, Chermack ST, Ehrlich PF. , et al. Alcohol interventions among underage drinkers in the ED: a randomized controlled trial. Pediatrics 2015; 136 (04) e783-e793
  • 19 SBIRT [computer program]. Version 1.3. Center on Addiction and Northwell Health; 2015
  • 20 Levesque D, Umanzor C, de Aguiar E. Stage-based mobile intervention for substance use disorders in primary care: development and test of acceptability. JMIR Med Inform 2018; 6 (01) e1
  • 21 Satre DD, Ly K, Wamsley M, Curtis A, Satterfield J. A digital tool to promote alcohol and drug use Screening, Brief Intervention, and Referral to Treatment skill translation: a mobile app development and randomized controlled trial protocol. JMIR Res Protoc 2017; 6 (04) e55
  • 22 Kumar S, Nilsen WJ, Abernethy A. , et al. Mobile health technology evaluation: the mHealth evidence workshop. Am J Prev Med 2013; 45 (02) 228-236
  • 23 Tomlinson M, Rotheram-Borus MJ, Swartz L, Tsai AC. Scaling up mHealth: where is the evidence?. PLoS Med 2013; 10 (02) e1001382
  • 24 Ammenwerth E, Iller C, Mahler C. IT-adoption and the interaction of task, technology and individuals: a fit framework and a case study. BMC Med Inform Decis Mak 2006; 6: 3
  • 25 Honekamp W, Ostermann H. Evaluation of a prototype health information system using the FITT framework. Inform Prim Care 2011; 19 (01) 47-49
  • 26 Noblin A, Shettian M, Cortelyou-Ward K, Schack Dugre J. Exploring physical therapists' perceptions of mobile application usage utilizing the FITT framework. Inform Health Soc Care 2017; 42 (02) 180-193
  • 27 Sheehan B, Lee Y, Rodriguez M, Tiase V, Schnall R. A comparison of usability factors of four mobile devices for accessing healthcare information by adolescents. Appl Clin Inform 2012; 3 (04) 356-366
  • 28 Seffah A, Kececi H, Donyaee M. QUIM: a framework for quantifying usability metrics in software quality models. Paper presented at: Second Asia-Pacific Conference on Quality Software; December 10–11, 2001 ; Hong Kong
  • 29 Nielsen JE. Estimating the number of subjects needed for a thinking aloud test. Int J Hum Comput Stud 1994; 41 (03) 385-397
  • 30 Nielsen JE, Landauer TK. A mathematical model of the finding of usability problems. Paper presented at: INTERCHI'93 Conference on Human Factors in Computing Systems; April 24–29, 1993 ; Amsterdam
  • 31 Press A, DeStio C, McCullagh L, Kapoor S, Morley J, Conigliaro J. ; SBIRT NY-II Team. Usability testing of a national substance use screening tool embedded in electronic health records. JMIR Human Factors 2016; 3 (02) e18
  • 32 Rubin J, Chisnell D. Handbook of Usability Testing: How to Plan, Design, and Conduct Effective Tests. Indianapolis, IN: Wiley; 2008
  • 33 Tullis T, Albert B. Measuring the User Experience: Collecting, Analyzing, and Presenting Usability Metrics. Waltham, MA: Elsevier; 2013
  • 34 Vilardaga R, Rizo J, Kientz JA, McDonell MG, Ries RK, Sobel K. User experience evaluation of a smoking cessation app in people with serious mental illness. Nicotine Tob Res 2016; 18 (05) 1032-1038
  • 35 Wilson V, Neilson CJ. We want it now and we want it easy: usability testing of an online health library for healthcare practitioners. J Can Health Libr Assoc 2014; 32 (02) 51-59
  • 36 Creswell JW, Creswell JD. Research Design: Qualitative, Quantitative, and Mixed Methods Approaches. Los Angeles, CA: Sage Publications; 2017
  • 37 Brinck T, Gergle D, Wood SD. Usability for the Web: Designing Web Sites That Work. San Francisco, CA: Morgan Kaufmann; 2001
  • 38 Jaspers MW, Steen T, van den Bos C, Geenen M. The think aloud method: a guide to user interface design. Int J Med Inform 2004; 73 (11–12): 781-795
  • 39 Hennink MM, Kaiser BN, Marconi VC. Code saturation versus meaning saturation: how many interviews are enough?. Qual Health Res 2017; 27 (04) 591-608
  • 40 Brooke J. SUS: a retrospective. J Usability Stud 2013; 8 (02) 29-40
  • 41 Services USDoHaH. System Usability Scale. Available at: https://www.usability.gov/how-to-and-tools/methods/system-usability-scale.html . Accessed February 12, 2019
  • 42 Lewis JR, Sauro J. The factor structure of the System Usability Scale. In: Kurosu M. ed. Human Computer and Design. HCD 2009. Lecture Notes in Computer Science. Berlin, Heidelberg: Springer; 2009. 5619: 94-103
  • 43 Sauro J. A Practical Guide to the System Usability Scale: Background, Benchmarks & Best Practices. Denver, CO: Measuring Usability LLC; 2011
  • 44 Borsci S, Federici S, Lauriola M. On the dimensionality of the System Usability Scale: a test of alternative measurement models. Cogn Process 2009; 10 (03) 193-197
  • 45 Lewis J. IBM computer usability satisfaction questionnaires: psychometric evaluation and instructions for use. Int J Hum Comput Interact 1995; 7 (01) 57-78
  • 46 Sauro J, Zarolia P. SUPR-Qm: a questionnaire to measure the mobile app user experience. J Usability Stud 2017; 13 (01) 17-37
  • 47 Kortum P, Sorber M. Measuring the usability of mobile applications for phones and tablets. Int J Hum Comput Interact 2015; 31 (08) 518-529
  • 48 Lewis JR. Measuring perceived usability: the CSUQ, SUS, and UMUX. Int J Hum Comput Interact 2018; 34 (12) 1148-1156
  • 49 Bevan NJ. Extending quality in use to provide a framework for usability measurement. Paper presented at: 1st International Conference on Human Centered Design, held as Part of HCI International; 2009 ; San Diego, CA
  • 50 Neal JW, Neal ZP, VanDyke E, Kornbluh M. Expediting the analysis of qualitative data in evaluation: a procedure for the rapid identification of themes from audio recordings (RITA). Am J Eval 2015; 36 (01) 118-132
  • 51 Hsieh HF, Shannon SE. Three approaches to qualitative content analysis. Qual Health Res 2005; 15 (09) 1277-1288
  • 52 Muhr T. ATLAS.ti: The knowledge Workbench: Visual Qualitative Data, Analysis, Management, Model Building: Short User's Manual. Berlin: Scientific Software Development; 1997
  • 53 Erlingsson C, Brysiewicz P. A hands-on guide to doing content analysis. Afr J Emerg Med 2017; 7 (03) 93-99