Keywords
digital therapeutics - mobile game - mobile health - AI-medicine - autism spectrum disorder - developmental disorders - behavioral therapy - clinical testing - smartphone - patient population - mental health
Background and Significance
Background and Significance
Autism spectrum disorder (ASD) affects at least 214 million children worldwide, including 1 million children at or under the age of 10 in the United States.[1]
[2] Autism's prevalence in the United States rose from 1 in 149 children in 2000 to 1 in 59 in 2014 (a 2.5× increase), with most recent estimates projecting a further increase.[1]
[3] Children with ASD exhibit restricted and repetitive behaviors, communication, and social skill deficits including difficulties with emotion, recognition, and joint attention.[4]
[5]
[6]
[7]
[8] While multiple studies have demonstrated that the delivery of behavioral therapy as early as possible can effectively treat and even eliminate an ASD diagnosis,[9]
[10]
[11]
[12]
[13] wait times to receive a diagnostic evaluation can,[14] exceed 14 months,[15] and out-of-pocket expenses can reach $80,000 per year[16] leaving one in three autistic children in the United States unable to receive standard care. Geographic isolation and coronavirus disease 2019 (COVID-19) related lockdowns present further limitations in access to consistent and adequate treatment.[17]
[18]
Mobile forms of diagnosis and therapy that can function at home offer a way to affordably meet the demand for early intervention services for children with ASD and related neurodevelopmental disorders. We previously demonstrated that wearable augmented reality glasses such as Superpower Glass,[19]
[20]
[21]
[22]
[23]
[24]
[25] can deliver significant therapeutic benefits to children with autism using a fun exchange between the autistic child and a caregiver through games such as “Capture The Smile” where the child must get the caregiver to express a series of emotions.[26] Other mobile interventions, such as AKL-T01 (EndeavorRx, Akili Interactive Labs, San Francisco, California , United States), the first game-based therapeutic to receive market clearance by the FDA in June 2020,[27] have shown therapeutic effectiveness in attention deficit/hyperactivity disorder (ADHD),[28] anxiety,[29] depression,[30] and other mental health conditions.[31] This growing body of evidence highlights the potential for scalable digital therapies to provide access to care to patient populations in the United States and globally.
Developing engaging and effective mobile game-based digital therapeutics requires several sequential steps from ideation to clinical testing and validation. The first step beyond initial concept and storyboarding is co-design with the stakeholders to ensure that players will be sufficiently engaged in the game and to begin to circumscribe the demographics and clinical characteristics of the users for whom the digital therapy can have its highest impact. The Superpower Glass program started with co-design experiments which examined how children from different demographics and a wide range of autism severity levels interacted with our device.[24]
[32] Likewise, the clinical development of AKL-T01 began with a feasibility study of the effectiveness of a prototype of the intervention in a wide range of children with sensory processing dysfunction with and without ADHD.[33] Following this exploratory stage, to settle on the core functionalities of the digital intervention, researchers must pilot test the intervention's utility and potential in a treatment-only clinical feasibility test. A feasibility test of the wearable augmented reality Superpower Glass system showed significant changes in meaningful clinical end points in 14 autistic children of an average age of 7 years 6 months (SD = 2.51 years).[34] Similarly, AKL-T01 was tested in a study which allowed the researchers to decide on a final clinical target and a set of product characteristics that showed the greatest potential for therapeutic benefit among their target patients.[35] A key next step, often a final step to commercial pathways, is to test the effect of the digital therapeutic in a formal randomized controlled trial that follows an intention-to-treat trial protocol. Both Superpower Glass[26] and AKL-T01[28] were tested in randomized controlled trials and showed significant positive changes on their primary endpoint measures in the treatment group.
We have followed our digital therapeutic translational framework outlined above to test the feasibility of GuessWhat's computer–human interaction design, user experience and engagement in laboratory[36] and home environments[37]
[38]
[39] and to optimize the system to be both fun and therapeutic. Specifically, we prototyped the game with eight families over 6 weeks with a combination of in-person focus group sessions, beta testing, and at-home game play. We then launched the application on the Android and iOS application stores for free download and use and marketed the game's availability through Facebook and other social media outlets to encourage players (irrespective of autism diagnosis or any other characteristic) to try the game. We gathered play metrics and electronic feedback from this virtual collection of players to further refine the game design. We evaluated the data feed to settle on the 90-second game session timeframe for the charades challenges as optimal both for engagement and for data acquisition. These aspects of the human computer interaction design led to the version tested here, which was optimized for engagement and treatment potential. In the present study, we test this version of the game's feasibility as an at-home digital therapeutic for children with autism between the ages of 3 and 12 years (the current ages where we have performed user-testing and engagement design iterations). This study is an important and necessary step toward a formal randomized controlled trial.
Methods
Study Design and Participants
This study was conducted remotely. Participants were recruited through GuessWhat's existing userbase, The Hartwell Foundation's KidsFirst autism research database, ResearchMatch.org, and Facebook advertisements. All participating families met the following criteria: (1) able to read and speak English, (2) possessed or had access to a compatible iOS or Android device with internet access, (3) the parent was 18+ years old, (4) the child was between 3 and 12 years of age and diagnosed with ASD. To safeguard against the potential for self-reporting bias, we required the caregiver to confirm that their child's autism diagnosis came from a formal medical assessment. We asked the caregiver to choose a diagnostic label from a menu of choices including ASD, autistic disorder, pervasive developmental disorder-not otherwise specified, Asperger syndrome, ADHD/ADD, anxiety, speech and language delay. In addition, we required participants to report on the specific type(s) of therapy being administered to their child. We also asked participants to fill out fields associated with the Mobile Autism Risk Assessment,[40] a machine learning tool validated to have high accuracy for autism screening. Finally, our clinical team reviewed all videos shared by participants for exhibition by the autistic child of symptoms and features consistent with an autism diagnosis. All participants provided informed consent online on behalf of their child prior to participation. Ethical approval was obtained from Stanford University's Institutional Review Board prior to all research activities. This approval included a Data Risk Assessment performed by the University Privacy Office that found our system to be compliant with Stanford University, State of California, and Federal privacy regulations.
Mobile Game Platform
GuessWhat is a mobile application available for at-home use through any iOS and Android device ([Fig. 1A]). The game uses a form of mobile charades to create opportunities for social exchange between a child and a playing partner centered around face and eye contact and expanded range of emotion recognition and expressivity for the child. A caregiver holds a smartphone on their forehead while prompts appear on the screen. The child must act out the prompts so that the caregiver will be able to guess what the child is acting ([Fig. 1B, C]). The gameplay integrates key components of applied behavior analysis (ABA),[41] discrete trial training,[42] and pivotal response training[43] ([Table 1]) to engage children in self-initiated and naturalistic imitation[44] and sociodramatic and symbolic play[45] with their social partner (e.g., their parent). In-app daily, weekly, and monthly challenges and reward mechanisms (including audio affirmation, coins, and the unlocking of new game decks and achievement badges) reinforce positive progression through the game and maintain the child's interest through time ([Fig. 1], day f).
Fig. 1 The GuessWhat Mobile User Experience. (A) GuessWhat is a mobile game available for any smartphone device. In a typical game session, (B) the parent holds the smartphone to their forehead and tries to guess what the child is acting in response to the prompt shown on the phone's screen. Upon guessing, the parent tilts the phone to proceed to the next prompt through the end of the 90-second session. (C) After each 90-second game, parent and child can review together, enhancing and strengthening the learning. In-app (D) game modes, (E) unlocking deck and character choices based on coins earned, and (F) activity-based achievement badges reinforce positive progression and ensure optimal child engagement through time.
Table 1
GuessWhat game elements and treatment focus areas. The parent holds the phone on forehead, variable image prompts appear on screen, and child acts what is displayed ([Fig. 1]). If parent can guess the prompt correctly, they tilt the phone downward to earn a coin and move to the next prompt. If unable to guess, the parent tilts the phone upward to skip to the next prompt. The game in its current form comes with over 40 different deck choices that can be used alone or mixed into sessions lasting 90 seconds each. Earned coins enable unlocking of new decks and the purchase of new game characters desired by the child. The mobile application is available on Android and iOS; both versions are linked from guesswhat.stanford.edu
|
GuessWhat game element
|
Treatment focus area
|
|
Charades: This is a primary game mode. Act out images displayed on screen. Vary the acting challenges and try to score higher with each session.
|
Eye contact, joint attention, theory of mind, imitation, expressive and receptive communication, nonverbal social awareness, conceptual (abstract) reasoning; flexible thinking.
|
|
Selecting and mixing decks: Choose from 40 different decks that feature different animals, sports, occupations, environments, and more.
|
Self-initiation, independence, comprehension of and response to multiple cues, reinforcement of variable interests/behaviors.
|
|
Earning coins: Correct guesses by play partner rewarded with coins that allow purchase and unlocking of new game decks and game characters.
|
Social motivation
|
|
Turn taking mode: Child and parent prompted to control the phone and take turns acting out the challenges displayed on the phone screen.
|
Turn taking, sharing, social initiations
|
|
Video playback: Watch yourself perform the challenge.
|
Reinforcement learning and course correction
|
|
Motion challenges: Sports, gestures, objects, chores decks.
|
Gross and fine motor skills
|
|
Sound challenges: sing-along, pledge allegiance, poetry recitals.
|
Speech and language
|
|
Emotion challenges: emojis, human faces decks, cartoon character faces.
|
Facial emotion recognition and expression
|
Each play session begins by the parent selecting one or more of several prompt decks, such as Faces, Dances, or Emojis, which aim at core behavioral targets such as receptive and expressive communication, joint attention, multiple cue response, gross and fine motor skills ([Table 1]).[9]
[41]
[42]
[43] Once a game deck(s) has been selected, the parent presses “Play” and holds the phone to their forehead with the screen facing the child. The child then interprets the prompt displayed on the screen and acts out what they see (e.g., “surprise” for the Faces deck or “the floss” for the Dances deck). The parent must then guess what the child is acting out and confirm if they are correct by communicating via verbal or social cues and tilting the phone up or down. GuessWhat uses the phone's accelerometer to detect a tilt in the forward direction indicating a correct guess or a tilt back indicating “skip” to proceed forward to the next prompt. After 90 seconds, the game session is complete, and the results are presented back to the user as how many prompts were guessed correctly versus skipped. The parent is then given the opportunity to review the answers and the gameplay video with the child, and to save the video to their phone, share it with our laboratory, or delete it altogether. When a user chooses to share their gameplay video, it is stored in a Health Insurance Portability and Accountability Act (HIPAA) compliant encrypted cloud database for further research.
Participant Procedures
All participants were asked to download and start playing GuessWhat at the beginning of the study. Families were instructed to play the game for at least three 90-second sessions per day, 3 days per week, for 4 weeks, for a total of 54 minutes of gameplay over 1 month. Participants were directed to play any of the available game decks, with particular emphasis on Faces and Emojis, which were specifically designed to encourage emotion recognition and social communication. Parent-reported surveys were collected for all participants immediately before and after the 4-week testing period. Participants were sent an Amazon gift card for the completion of study procedures.
Measures
Gameplay metrics and qualitative feedback were collected from all participants who used the GuessWhat application. Gameplay metrics were collected throughout the 4-week intervention period and included game deck selection, number of 90-second game sessions initiated and completed, and total play time per session, per login, and for the entire study period.
A qualitative feedback measure created by our research team assessed the GuessWhat players' user experience at the end of the 4-week period. This measure includes 11 free-text and multiple-choice items capturing participants' feedback and suggestions, technical difficulties they experienced while playing the app, preferred game modes, and likelihood of using the app beyond the study.
All study participants responded to two standardized clinical outcome measures for autism, the Social Responsiveness Scale, 2nd edition[46] (SRS-2) and the Vineland Adaptive Behavior Scales, 2nd edition[47] (VABS-II). Both surveys were collected at the start and end of each family's 1-month testing window.
SRS-2 is a 65-item survey intended to identify the presence and severity of social impairment in children across five social domains. Standardized scores at or above 60 are considered indicative of social impairment.
VABS-II is a 502-item survey which measures children's adaptive functioning on four main domains: communication, daily living skills, socialization, and motor skills. We collected the Socialization Domain (99 questions that measure interpersonal relationships, play and leisure time, and coping skills) and the Receptive and Expressive Communication subscales (74 items) of the Communication Domain using the Comprehensive Parent/Caregiver Form of the survey. Higher scores indicate greater social functioning.
Analysis
Usage metrics were analyzed across each participant from first date of gameplay following pre-test survey data collection through last gameplay session prior to post-test survey data collection. Free-text responses in the qualitative feedback measure recorded during post-test survey data collection were evaluated using content analysis to identify the most common technical difficulties as well as overall satisfaction and engagement with the GuessWhat mobile game. Analysis of the qualitative feedback measure also included the quantification of multiple-choice radio select options measuring how many participants intended to continue playing the app following study completion.
Changes in SRS-2 and VABS-II scores were assessed using means comparisons with two-tailed paired sample t-tests. Values 1.5 times above or below the interquartile range were excluded from the analysis. Families who did not record GuessWhat play attempts but completed all other procedures were analyzed as a separate group for comparison.
Results
We tested the therapeutic feasibility of our mobile game with 72 autistic children (75% male, average age 8 years 2 months, minimum age 3 years 10 months, maximum age 12 years 6 months) ([Fig. 2]). Sixty-two (86%) players were receiving standard autism therapy treatments, with 60 (83%) receiving two types or more during the study ([Table 2]). A comparison control group that did not play the game but completed all other procedures included 19 children (84% male, average age 8 years 5 months, minimum age 3 years 11 months, maximum 12 years 4 months). Eighteen (95%) children in the comparison group received some form of autism-related therapy, with 13 (68%) receiving two types or more ([Table 2], [Supplementary Fig. S1], available in the online version). Prior to the start of our GuessWhat treatment, we did not find significant differences between the two groups in the two baseline measures, the Social Responsiveness Scale (SRS-2) and the Vineland Adaptive Behavior Socialization Standard Score ([Table 2]). We did not find any disagreements between our independent confirmation of the autism diagnosis and the self-reported autism diagnosis in either the treated or the comparison group.
Fig. 2 Study groups. Ninety-one families were assessed for eligibility and completed pre- and post-test surveys between March 2019 and December 2020. Seventy-two families played GuessWhat as instructed. Nineteen families who did not record GuessWhat play attempts but fulfilled all other requirements were analyzed as a separate group for comparison.
Table 2
User demographics and survey scores for the 72 autism families in the GuessWhat treatment group and the 19 matched participants in the comparison control group (autism families who did not play GuessWhat but completed all other procedures)
|
Demographic information, Mean (SD)/Percent (N)iii
|
All users (N = 72)
|
Users with ≥28 sessions (N = 17)
|
Users with 4 wk of consistent usage (N = 13)
|
Comparison group (N = 19)
|
|
Age (years)
|
8.17 (2.25)
|
7.97 (2.58)
|
7.9 (2.13)
|
8.4 (2.89)
|
|
Gender (% male)
|
75% (54)
|
80% (16)
|
85% (11)
|
84% (16)
|
|
Comorbidity
|
Attention deficit disorder
|
14% (10)
|
15% (3)
|
8% (1)
|
21% (4)
|
|
Anxiety disorder
|
6% (4)
|
0% (0)
|
0% (0)
|
21% (4)
|
|
Bipolar disorder
|
1% (1)
|
0% (0)
|
8% (1)
|
0% (0)
|
|
Sensory processing disorder
|
4% (3)
|
0% (0)
|
0% (0)
|
5% (1)
|
|
Race and
ethnicity
|
Caucasian/Euro-American
|
72% (52)
|
65% (13)
|
77% (10)
|
68% (13)
|
|
African American
|
7% (5)
|
10% (2)
|
0% (0)
|
26% (5)
|
|
East Asian/Asian American
|
10% (7)
|
10% (2)
|
8% (1)
|
0% (0)
|
|
South Asian/Indian American
|
4% (3)
|
10% (2)
|
0% (0)
|
5% (1)
|
|
Middle Eastern/Arab American
|
1% (1)
|
0% (0)
|
0% (0)
|
0% (0)
|
|
Pacific Islander
|
0% (0)
|
0% (0)
|
0% (0)
|
11% (2)
|
|
Native American/Alaskan Native
|
1% (1)
|
0% (0)
|
0% (0)
|
5% (1)
|
|
Hispanic/Latino/Spanish Origin
|
22% (16)
|
20% (4)
|
8% (1)
|
21% (4)
|
|
Unknown/Not listed
|
6% (4)
|
5% (1)
|
77% (10)
|
0% (0)
|
|
Concurrent therapy enrollment
|
Applied behavior analysis (ABA)
|
47% (34)
|
50% (10)
|
62% (8)
|
58% (11)
|
|
Social skills therapy
|
35% (25)
|
40% (8)
|
31% (4)
|
37% (7)
|
|
Special education classes
|
44% (32)
|
30% (6)
|
38% (5)
|
53% (10)
|
|
Speech-language pathology
|
74% (53)
|
70% (14)
|
69% (9)
|
47% (9)
|
|
Occupational therapy
|
58% (42)
|
65% (13)
|
62% (8)
|
68% (13)
|
|
No therapy
|
14% (10)
|
15% (3)
|
8% (1)
|
5% (1)
|
|
1 type of therapy only
|
2.8% (2)
|
5% (1)
|
0% (0)
|
26% (5)
|
|
2+ types of therapy
|
83% (60)
|
80% (16)
|
0% (0)
|
68% (13)
|
|
Clinical evaluations
|
|
|
Social Responsiveness Scale (SRS-2), intake score
|
Social awareness
|
75.24 (9.85)
|
74.29 (9.42)
|
76.54 (6.71)
|
79.71 (10.44)
|
|
Social cognition
|
77.46 (8.36)
|
76.47 (9.07)
|
77.77 (6.1)
|
76.22 (11.63)
|
|
Social communication
|
78.5 (9.51)
|
78.53 (11.23)
|
80.08 (4.05)
|
79.24 (9.29)
|
|
Social motivation
|
69.67 (11.14)
|
68.12 (11.94)
|
67.62 (8.71)
|
70.4 (5.74)
|
|
Restricted and repetitive behavior
|
79.04 (9.77)
|
78.24 (9.97)
|
82.31 (7.03)
|
77 (11.06)
|
|
Total*
|
80 (8.43)
|
78.82 (10.56)
|
81 (4.56)
|
80.82 (8.18)
|
|
Vineland adaptive behavior scales-II (VABS-II), intake score
|
Receptive communication
|
9.55 (3.31)
|
9.76 (3.88)
|
8.85 (1.63)
|
8.93 (2.43)
|
|
Expressive communication
|
9.86 (4.04)
|
10.94 (5.02)
|
9 (1.28)
|
8.22 (4.05)
|
|
Socialization relationships
|
8.99 (3.69)
|
9.56 (3.79)
|
8.54 (2.02)
|
7.78 (3.25)
|
|
Socialization play
|
8.55 (3.18)
|
10.18 (5.87)
|
8.77 (2.3)
|
7.67 (4.56)
|
|
Socialization adaptive
|
10.82 (3.21)
|
12 (3.82)
|
10.23 (2.13)
|
9.65 (3.14)
|
|
Socialization standard Total*
|
68.94 (15.53)
|
73.44 (20.85)
|
68.08 (6.75)
|
65.17 (20.52)
|
Note: The baseline total scores (designated with a) for the two primary measures did not differ significantly between the treatment and control groups, regardless of usage patterns.
All players reported high engagement and satisfaction with the GuessWhat mobile application. Sixty-seven percent of families expressed intention to continue playing beyond the study period. Eleven families reported experiencing technical difficulties, five of whom were able to resolve the issues during the testing period. The most common issues experienced by these users were trouble logging in (n = 5), poor internet connection (n = 3), or the application freezing or crashing (n = 2). Thirteen families reported difficulties such as confusion about how to play the game (n = 6) or prompts that were too difficult (n = 4). Seven families expressed difficulty interpreting the game instructions.
GuessWhat users played an average of 17.68 (SD = 18.04) sessions over 5.14 (SD = 4.63) days during the 4-week testing period, 49% of the recommended use. Participants selected the Emojis deck in 16% of the game sessions, Faces in 12%, Animals in 13%, followed by Objects (8%), Jobs (7%), Sports (7%), and all others (37%) ([Fig. 3]). Proper gameplay was verified through videos recorded during game play and that were shared with our team ([Table 3]).
Fig. 3
GuessWhat game decks. Users played a total of 1,965 game decks over the course of the study. The Emojis deck was chosen in 16% of all initiated game sessions, followed by Animals (13%), Faces (12%), Objects (8%), Jobs (7%), and Sports (7%). Cumulatively, all other decks were chosen in 37% of game sessions.
Table 3
GuessWhat usage statistics. Gameplay statistics were recorded over 1 mo for all 72 players in the treatment group
|
Game start
|
Game end
|
Game share
|
Points
|
|
Game
|
Num.
|
Avg.
|
Num.
|
Avg.
|
Num.
|
Avg.
|
Num.
|
Avg.
|
|
Emojis
|
316 (16%)
|
4.33 (SD = 5.98)
|
251 (15%)
|
3.44 (SD = 4.68)
|
208 (14%)
|
2.85 (SD = 4.1)
|
2,220 (25%)
|
30.41 (SD = 50.44)
|
|
Faces
|
233 (12%)
|
3.19 (SD = 4.34)
|
201 (12%)
|
2.75 (SD = 4.11)
|
172 (12%)
|
2.36 (SD = 3.82)
|
1,950 (22%)
|
26.71 (SD = 50.51)
|
|
Animals
|
250 (13%)
|
3.42 (SD = 7.15)
|
223 (13%)
|
3.05 (SD = 6.57)
|
202 (14%)
|
2.77 (SD = 6.55)
|
1,818 (21%)
|
24.90 (SD = 52.49)
|
|
Objects
|
155 (8%)
|
2.12 (SD = 4.09)
|
136 (8%)
|
1.86 (SD = 3.52)
|
111 (8%)
|
1.52 (SD = 3.34)
|
884 (10%)
|
12.11 (SD = 27.52)
|
|
Jobs
|
136 (7%)
|
1.86 (SD = 5.05)
|
124 (7%)
|
1.70 (SD = 4.54)
|
103 (7%)
|
1.41 (SD = 4.32)
|
720 (8%)
|
9.86 (SD = 27.37)
|
|
Sports
|
134 (7%)
|
1.84 (SD = 5.07)
|
113 (7%)
|
1.55 (SD = 4.58)
|
93 (6%)
|
1.27 (SD = 4.43)
|
746 (8%)
|
10.22 (SD = 25.88)
|
|
Other
|
740 (38%)
|
10.15 (SD = 24.81)
|
665(39%)
|
9.11 (SD = 21.95)
|
591 (40%)
|
8.10 (SD = 21.55)
|
488 (6%)
|
6.68 (SD = 95.66)
|
|
Тotal decks
|
1,964
|
26.92 (SD = 47.08)
|
1713
|
23.47 (SD = 42.46)
|
1,480
|
20.27 (SD = 41.32)
|
8,826
|
120.90 (SD = 158.86)
|
|
Total sessions
|
1,291
|
17.68 (SD = 18.04)
|
1105
|
15.14 (SD = 15.81)
|
959
|
13.14 (SD = 15.32)
|
183.93 (N/A)
|
Users demonstrated an average of 3.97-point improvement in SRS-2 total T-score (SD = 4.92, p < 0.001) as well as an improvement of 5.27 points on the VABS-II socialization standard score (SD = 9.29, p = 0.002). The game players who played more than 28 sessions (n = 17) showed a 3.82-point (SD = 5.70, p = 0.01) positive change on the SRS-2 Total T-score and a 6.21-point (SD = 8.13, p = 0.11) improvement on the VABS-II Socialization Standard score. The game players who played consistently over all 4 weeks (n = 13) showed a 2.85-point (SD = 4.56, p = 0.03) positive change on the SRS-2 total T-score and 5.46-point (SD = 6.75, p = 0.58) improvement on the VABS-II socialization standard score. The 19 families who did not play GuessWhat but completed all other procedures showed no significant change in scores recorded by either survey. Results of the primary means comparisons are presented in [Table 4] (subsection scores available in [Supplementary Table S1], available in the online version).
Table 4
Change in parent-reported survey scores by user group. Results include 72 families who played GuessWhat and a comparison group of 19 matched participants who did not play GuessWhat but completed all other procedures. Reductions in SRS-2 and increases in VABS-II scores are indicative of improvement and vice versa
|
Measure
|
Mean change from pre-test to post-test (SD)
|
|
All treated players
|
p-Value
|
Users treated with ≥28 sessions
|
p-Value
|
Users with 4 wk. treatment consistency
|
p-Value
|
Comparison group
|
p-Value
|
|
SRS-2 total T-score
|
−3.97 (SD = 4.92)
|
<0.001[c]
|
−3.82 (SD = 5.70)
|
0.01[a]
|
−2.85 (SD = 4.56)
|
0.03[a]
|
−4.71 (SD = 7.92)
|
0.10
|
|
VABS-II socialization standard score
|
5.27 (SD = 9.29)
|
0.002[b]
|
6.21 (SD = 8.13)
|
0.11
|
5.46 (SD = 6.75)
|
0.58
|
−1.61 (SD = 12.37)
|
0.62
|
a p <0.05.
b
p <0.01.
c
p <0.001 between cohorts from a two-tailed paired sample t-test.
Discussion
Our study examined the potential for the mobile game GuessWhat to provide a therapeutic effect for autistic children and families who play the game for a period of 1 month. The game fosters a prosocial exchange between the child and playing partner in short 90-second sessions designed for social skills development including eye contact and emotion recognition. Out of the 91 families assessed for eligibility who completed pre- and post-test surveys, 72 (79%) successfully played one or more GuessWhat game sessions. The 19 (21%) families who did not play the game at all were assigned to a separate comparison group. The 72 GuessWhat users exhibited significant improvements in SRS-2 and VABS-II scores, while the untreated comparison group showed no significant differences in either measure over the same 4-week timeframe.
Average improvements in SRS-2 total scores were in line with minimal clinically important differences (MCIDs) required for a shift from severe to moderate, moderate to mild, or mild autism severity to within normal limits.[48] Average improvements in VABS-II Socialization domain scores exceeded the MCID of 3.7 points.[49] Interestingly, these therapeutic effects were evident even with adherence to dose recommendations at approximately 50% of the recommended game play frequency. GuessWhat users averaged 5.14 gameplay days over the 4-week testing period even though they were asked to play a minimum of 12 days (three sessions, 3 days a week over 4 weeks). Potential reasons for the lower adherence include technical difficulties experienced by some families, decreased child engagement with the app over time due to boredom, and/or decreased motivation by the parent to continue using the intervention. While these lower usage patterns demonstrate the need for an enhanced user interface including reminder and reward systems, the observed gains in socialization provide support for the digital game's potential as a useful form of autism therapy. The results also support previous game therapy research showing that parental involvement, a major design focus for our system, positively impacts the therapeutic effect.[50]
Players who played more than 28 GuessWhat sessions and/or played during each week of the intervention period showed larger improvements on VABS-II Socialization Standard scores and on VABS-II Receptive and Expressive Communication scores suggesting that improvements may be greater with higher compliance to the recommended gameplay ([Table 4]). This dose-dependency in therapeutic gains also supports the likelihood that the observed effects were likely not due to a placebo effect. Further tests are needed to confirm the therapeutic effectiveness, to determine optimal levels of gameplay, to design the most effective means of ensuring adherence to it, and to show the potential for sustained gains after therapy periods conclude.
Limitations
This research demonstrates how a new type of mobile game-based therapy may be used from the convenience of people's homes for improving social acuity in children with autism. One limitation of the study was the use of nonblinded, parent-administered intervention and outcome measures, which risks biased reporting. It can be expected that potential biases in parent reporting would manifest as stronger on the SRS-2 scale than on VABS-II, since the latter asks more quantitative questions and therefore leaves less opportunity for subjective judgment. Accordingly, the lack of discrepancies between SRS-2 and VABS-II results for GuessWhat players ([Table 4], [Supplementary Table S1], available in the online version) demonstrates that the socialization gains observed following a month of gameplay were not primarily attributable to biases in parental reporting. Instead, the game's therapeutic effect is likely due to the increased frequency of (1) structured social engagement between parent and child, and (2) practice of facial and emotion recognition skills during the charades-style interaction.
A second limitation of the present study was potential self-selection bias. It may be the case that the 19 families who did not attempt to play GuessWhat but completed all other study procedures had lower motivation to participate in the study. A lack of motivation in this comparison group could have therefore resulted in reduced gains measured by either SRS-2 or VABS-II parent-reported surveys. Nevertheless, the inconsistent changes between the two outcome measures (SRS-2 scores improved while VABS-II worsened) and the large standard deviation observed in the comparison group ([Table 4], [Supplementary Table S1] [available in the online version]) indicate that reduced gains due to lack of motivation did not sufficiently account for the differences observed between GuessWhat players and non-players. A larger comparison group will be needed to confirm these results.
A third limitation was that the demographics of our study sample were predominantly male and Caucasian ([Table 2]). Recruitment of balanced populations is challenged by the 4:1 male:female prevalence of autism[1] and underdiagnosis and undertreatment among non-Caucasian autism families.[2] Simple correlations between participant demographics and usage data or outcome measures yielded no clear conclusions about the best responders. We will use GuessWhat's advantages as a digital intervention to develop new game decks that will appeal to participants of any gender or ethnicity and to recruit a more balanced cohort among a wider range of participants from all over the United States and the world in future studies.
Finally, the enrollment of 32 GuessWhat treatment and 18 of the 19 control participants during COVID-19-related school closures may have introduced additional confounding factors such as loss of regular therapy, childcare and school services, and others. Changes in therapy and lifestyle during the intervention period or at a post-test remain a topic for future investigation.
Conclusion
This study demonstrates that GuessWhat can improve socialization in children with autism in an engaging and accessible manner. The results support the potential of game-based mobile systems to augment the standard approaches to autism therapy, ideally increasing the continuity of behavioral therapy and preventing loss in services where access to care is limited.
Clinical Relevance Statement
Clinical Relevance Statement
The results of this study indicate that game-based digital therapeutics may be an effective to deliver behavioral therapy for children with autism spectrum disorder. This outcome is particularly relevant in the context of the current coronavirus disease pandemic and restrictions to in-person clinical services. Mobile tools such as GuessWhat present a viable way for bridging gaps in continuity of care and ensuring that all families can access effective and engaging behavioral therapy from the comfort of their home.
Multiple Choice Questions
Multiple Choice Questions
-
What is the first step (after prototyping) in the design of a novel digital therapeutic?
Correct Answer: The correct answer is option c. The design of novel digital therapeutic begins with a series of co-design experiments done with the target patient population intended to optimize engagement, to begin to understand the dose–response relationship, to refine hypotheses regarding the expected clinical treatment effect, and to generally understand the ways in which the stakeholders will interact with a prototype of the product.
-
When did FDA begin clearing game-based digital therapeutics for market use?
-
1990s.
-
2000s.
-
2010s.
-
2020s.
Correct Answer: The correct answer is option d. The Food and Drug Administration cleared the first game-based digital intervention for market use in the United States in June 2020.
