Sleep apps: current limitations and challenges

Sleep app ownership is increasing exponentially, due to their accessibility and ease-of-use. However, there are several concerns regarding the use of sleep apps. Few sleep apps demonstrate empirical evidence to support their claims, and if they do, this evidence can be based on significant methodological limitations. In addition, there are data privacy concerns with regards to sleep apps, which share sensitive user data with business and marketing partners, unbeknownst to their users. Moreover, sleep apps may increase engagement with healthcare professionals, which may place additional strain on under-pressure sleep services. This would be compounded by the fact that some sleep apps produce many false positives, and clinicians would need more time to analyze the data provided by these apps. In the future, sleep apps must undergo rigorous validation studies and grant more autonomy to their users over how their data is shared.

Mobile Applications - Validation Study - Smartphone - Polysomnography - Sleep

INTRODUCTION

A sleep app is defined as a specialised programme downloaded onto medical devices, marketed with the assertion that it performs sleep monitoring or sleep-related interventions[1]. Sleep app ownership is increasing exponentially, due to their affordability, ease-of-use and the ubiquitous nature of smartphones; there will be an estimated 6 billion smartphones in use by the end of 2020[2]. Sleep apps are used to screen for various conditions, including obstructive sleep apnoea, insomnia and periodic limb movement disorder[3]-[5]. While sleep apps have the potential to raise awareness of sleep conditions and promote healthy sleep habits, there are several concerns regarding their use.

This article aims to outline the current limitations of sleep apps and challenges which need to be overcome before sleep apps can be safely integrated into clinical practice. In particular, many sleep apps are not supported by high-quality evidence. Moreover, the sensitive data collected by sleep apps may be breached and exploited by the apps’ developers. Finally, sleep apps may place added strain on sleep services.

Lack of evidence-based medicine in sleep apps

Many sleep apps are not supported with evidence-based medicine. 32.9% of sleep apps available in the Google Play Store demonstrated empirical evidence to support their claims, and 15.8% were developed with the input of clinicians[6]. To our knowledge, there are no similar studies evaluating sleep apps in other app stores (such as Apple’s App Store). Furthermore, a recent systematic review of sleep apps which detect sleep parameters showed that only 3 out of 73 apps had undergone validation studies using the gold standard: polysomnography[7]. In these validation studies, each subject underwent polysomnography while simultaneously using the sleep app. All 3 validation studies demonstrated weak correlation between polysomnography and the sleep apps, including the detection of sleep stages ([Table 1])[8]-[10]. This is particularly relevant as all 3 apps claim to have a “smart alarm” that wakes up users during non-REM sleep. Moreover, several “smart alarms” assume that sleep cycles are 90 minutes long. This is opposed by a landmark study, which showed that sleep cycle durations vary from 41.3 to 97.6 minutes[11]. There is even conflicting evidence regarding the relevance of sleep stage awakening on the duration and severity of sleep inertia[12], which undermines the basis of “smart alarms”. The lack of rigorous evidence in sleep apps is perpetuated by their frequent classification as “entertainment” products, with fine print absolving them of medical responsibility[13]. More validation studies of sleep app use in healthy adults are needed, as most of the current evidence is based on children with suspected or confirmed sleep disorders[8]-[9].

Table 1

Results of validations studies of sleep apps which claim to detect sleep parameters. All comparisons are made with polysomnography[8]-[10].

Sleep App	Study Population	Exclusion Criteria	Study Findings
Sleep Cycle[8]	- n=25 (22 suspected OSA, 3 healthy volunteers); - Age (years) = 8.0 ± 3.6; - % male = 56.	- Complex genetic or craniofacial disorders.	- No correlation with polysomnography in the measurement of total sleep time (CCC 0.22, p=0.36); - No correlation in the measurement of sleep latency (CCC 0.05, p=0.16); - No correlation in the detection of sleep cycle stages (data not provided).
MotionX 24/7[9]	- n=78 (all suspected OSA); - Age (years) = 8.4 ± 4.0; - % male = 65.	- Conditions affecting motor control or limb movement.	- Over-estimated total sleep time by 106 minutes (p<0.0001); - Over-estimated sleep efficiency by 17% (p<0.0001); - Over-estimated sleep period time by 16 minutes (p<0.0001).
Sleep Time[10]	- n=20 (all healthy volunteers); - Age (years) = 39.5 ± 12.4; - % male = 60.	- Diagnosed sleep disorder.	- No correlation with polysomnography in the measurement of sleep efficiency (p=0.59) or sleep latency (p=0.09); - Under-estimated light sleep by 27.9% (p<0.0001); - Over-estimated deep sleep by 11.1% (p<0.0001).

Age expressed as mean ± standard deviation; OSA: Obstructive sleep apnoea; CCC: Concordance correlation co-efficient.

Furthermore, sleep apps may rely on evidence with significant methodological limitations. For example, the “Pzizz” app, which is promoted on the UK’s National Health Service website, is supported by a single clinical trial[14]. 16 participants used “Pzizz” during 20-minute naps every day for 2 weeks. This resulted in increased well-being survey scores at the end of the study period. However, the study lacked a control group and so it is unclear whether the act of daytime napping was the cause of the increased well-being scores. Furthermore, the well-being survey was created using “Pzizz promotional material”[14], rather than a validated survey. In addition, most studies of sleep apps used to detect obstructive sleep apnoea are undertaken in a controlled environment, rather than at the users’ homes, which is where these apps will be primarily used[15]. This reduces the external validity of these studies, and may explain why some sleep apps have weak diagnostic sensitivity for obstructive sleep apnoea (as they are unable to detect snoring in a noisier, less controlled environment)[16]. Thus, it is important to critically appraise any evidence supporting sleep apps.

Data privacy concerns

There are data privacy concerns with regards to sleep apps. The privacy notice of Sleep Number states that user data “may be shared” with business and marketing partners for the purpose of “research, analysis or administering surveys”[17]. Another app (Sleep Cycle) collects data about sleep habits worldwide, with its privacy policy stating that users consent to give the app access to their location if they use the app[18]. However, one can question the ethicality of placing this information in a privacy policy, since 91% of app users accept legal terms and conditions without reading them[19]. Therefore, it is unclear whether the users of sleep apps are aware that sensitive data is shared.

These examples of data sharing act as a microcosm of the data privacy concerns surrounding health apps. A systematic review of 24 health apps found that user data was shared with 46 “third parties” and 216 “fourth parties”[20]. Another review of 20 popular health apps showed that most violated the EU’s General Data Protection Regulation, so that de-anonymized personal details (such as locations) could conceivably be shared with third parties[21]. Sleep app data is vulnerable to these breaches, as this data is not protected by the privacy laws that apply to data collected in sleep clinics[22]. Sharing of de-anonymized health app data opposes the beliefs of health app users: 67% of users state that anonymity is “very” or “extremely” important[23].

These data breaches are particularly worrying with regards to sleep apps, as they collect highly sensitive information, such as sleep habits. One data privacy expert has even stated that third parties may be able to “interpret raw data” collected by sleep apps to identify when users engage in sexual activity[22]. This data may be used to develop targeted advertising, whereby online advertisements are tailored to an individual based on data provided by their sleep app[24]. This is already common practice when mobile health data is shared with third parties: 66% of these third parties collected the data for advertising and analytics[20]. For example, an expert in internet law has hypothesized that the data provided by a user’s sleep app may suggest that the user engages frequently in sexual intercourse, leading to marketing companies sending multiple online advertisements for contraceptives[25]. Other people using the same Internet device may see these online advertisements, and so this may lead to intrusions into a sleep app user’s privacy. Furthermore, online hackers are increasingly able to de-anonymize data shared by health apps; the names and fitness records of military personnel working in a high-security nuclear base were accessed using data provided by the fitness app Strava[26]. Similar data breaches are possible with sleep apps. For instance, some sleep apps store unencrypted audio files of users’ sleep sounds. Malicious apps are able to access these sleep recordings if they have read storage permission and can then send these recordings to external servers if they have internet permission[27].

Impact on clinical practice

Sleep apps will likely increase engagement with healthcare professionals[28], but this may have several unintended consequences. In the UK, the increase in sleep clinic referrals will put more strain on sleep services, which have already seen the number of sleep tests double from 2007-08 to 2016-17[29]. This added strain may be unnecessary since some sleep apps produce many false positives, such as an app that detects snoring[30]. Furthermore, clinicians may require additional training and time to analyze data from sleep apps, as this data can be presented in many different ways (such as values, graphs or a combination of both)[1]. This is because there are no clear standards in the way that sleep app data should be presented[31]. Also, clinicians may struggle to interpret the “sleep scores” provided by sleep apps, since the algorithms used to formulate these scores are often not disclosed, and neither are the normal ranges[32]. This issue is compounded by the fact that few sleep apps allow users to export their data for further analysis, and so clinicians may not be able to access the raw data that contributes to the “sleep scores”[7]. Moreover, it will be difficult to integrate sleep app data into patient health records, as many countries lack fully-digitized patient records; for example, only 12% of hospital trusts in the UK are fully-digitized[33]. Finally, sleep apps may even damage the doctor-patient relationship, as they could provide conflicting opinions to users’ doctors[34].

Recommendations

While sleep apps now appear to be a mainstay of sleep self-management, more work is needed for them to be used with confidence. Sleep apps must undergo rigorous validation studies to ensure that their claims are evidence-based. These validation studies should compare the sleep data generated by the app with sleep data from a relevant gold-standard (such as polysomnography). The sharing of user data with third parties must be fully disclosed in a more transparent manner than fine print in privacy notices, and users should retain autonomy over how their data is shared. Finally, the developers of sleep apps should listen to feedback from clinicians, such as suggestions on how to improve usability, to facilitate the smooth transition of these apps into clinical practice.

Conflict of Interests

The authors have no conflict of interests to declare.

• REFERENCES

• 1 Khosla S, Deak MC, Gault D, Goldstein CA, Hwang D, Kwon Y, et al. Consumer sleep technology: an American Academy of Sleep Medicine Position Statement. J Clin Sleep Med. 2018;14(5):877-80.
• 2 IHS Markit. More than six billion smartphones by 2020, IHS Markit Says [Internet]. London: IHS Markit; 2017; [access in 2020 Apr 22]. Available from: https://news.ihsmarkit.com/press-release/technology/ more-six-billion-smartphones-2020-ihs-markit-says
• 3 Sands SA, Owens RL. Does my bed partner have OSA? There’s an app for that!. J Clin Sleep Med. 2014;10(1):79-80.
• 4 Bhopi R, Nagy D, Erichsen D. Can a novel smartphone application detect periodic limb movements?. Stud Health Technol Informat. 2012;182:36-42.
• 5 Khan MN, Nock R, Gooneratne NS. Mobile devices and insomnia: understanding risks and benefits. Curr Sleep Med Rep. 2015 Dec;1(4):226-31.
• 6 Lee-Tobin PA, Ogeil RP, Savic M, Lubman DI. Rate my sleep: examining the information, function, and basis in empirical evidence within sleep applications for mobile devices. J Clin Sleep Med. 2017 Nov;13(11):1349-54.
• 7 Choi YK, Demiris G, Lin SY, Iribarren SJ, Landis CA, Thompson HJ, et al. Smartphone applications to support sleep self-management: review and evaluation. J Clin Sleep Med. 2018 Oct;14(10):1783-90.
• 8 Patel P, Kim JY, Brooks LJ. Accuracy of a smartphone application in estimating sleep in children. Sleep Breath. 2017 May;21(2):505-11.
• 9 Toon E, Davey MJ, Hollis SL, Nixon GM, Horne RSC, Biggs SN. Comparison of commercial wrist- based and smartphone accelerometers, actigraphy, and PSG in a clinical cohort of children and adolescents. J Clin Sleep Med. 2016;12(3):343-50.
• 10 Bhat S, Ferraris A, Gupta D, Mozafarian M, DeBari VA, Gushway- Henry N, et al. Is there a clinical role for smartphone sleep apps? Comparison of sleep cycle detection by a smartphone application to polysomnography. J Clin Sleep Med. 2015 Jul;11(7):709-15.
• 11 Feinberg I, Floyd TC. Systematic trends across the night in human sleep cycles. Psychophysiology. 1979;16(3):283-91.
• 12 Jewett ME, Wyatt JK, Cecco AR, Khalsa SB, Dijk DJ, Czeisler CA. Time course of sleep inertia dissipation in human performance and alertness. J Sleep Res. 1999 Mar;8(1): 1-8.
• 13 Ko PR, Kientz JA, Choe EK, Kay M, Landis CA, Watson NF. Consumer sleep technologies: a review of the landscape. J Clin Sleep Med. 2015;11(12):1455-61.
• 14 French L, Newman E, Hayward S, Polashek D, Gary M. A napping soundtrack can enhance well- being more than traditional relaxation soundtracks. Cognitive Technol. 2008;13(1):27-33.
• 15 Penzel T, Schöbel C, Fietze I. New technology to assess sleep apnea: wearables, smartphones, and accessories. F1000Research. 2018 Mar;7:413.
• 16 Nakano H, Hirayama K, Sadamitsu Y, Toshimitsu A, Fujita H, Shin S, et al. Monitoring sound to quantify snoring and sleep apnea severity using a smartphone: proof of concept. J Clin Sleep Med. 2014;10(1):73-8.
• 17 Sleep Number. Sleep number privacy policy [Internet]. Mineápolis, US: Sleep Number; 2000-2020; [access in 2020 Apr 25]. Available from: https://www.sleepnumber.com/legal-notices/privacy-policy
• 18 Robbins R, Affouf M, Seixas A, Beaugris L, Avirappattu G, Jean-Louis G. Fouryear trends in sleep duration and quality: a longitudinal study using data from a commercially available sleep tracker. J Med Internet Res. 2020;22(2):e14735.
• 19 Deloitte. 2017 Global Mobile Consumer Survey: US Edition [Internet]. New York: Deloitte; 2017; [access in 2020 Jun 7]. Available from: https://www2. deloitte.com/tr/en/pages/technology-media-and- telecommunications/ articles/global-mobile-consumer-survey-us-edition.html
• 20 Grundy Q, Chiu K, Held F, Continella A, Bero L, Holz R. Data sharing practices of medicines related apps and the mobile ecosystem: traffic, content, and network analysis. BMJ. 2019;364:I920.
• 21 Papageorgiou A, Strigkos M, Politou E, Alepis E, Solanas A, Patsakis C. Security and privacy analysis of mobile health applications: the alarming state of practice. IEEE Access. 2018;6:9390-403.
• 22 Appleby J, Kaiser Health News. A wake-up call on smart beds and sleep apps that collect your data. Time [Internet]. 2019 May; [cited 2020 Apr 28]. Available from: https://khn.org/news/a-wake-up-call-on-datacollecting- smart-beds- and-sleep-apps/
• 23 Bietz MJ, Bloss CS, Calvert S, Godino JG, Gregory J, Claffey MP, et al. Opportunities and challenges in the use of personal health data for health research. J Am Med Inform Assoc. 2016 Apr;23(e1):e42-8.
• 24 UK Government. Centre for Data Ethics and Innovation. CDEI Review of online targeting: final report and recommendations [Internet]. London, UK: Open Government Licence v3.0; 2020; [access in 2020 Jun 8]. Available from: https://www.gov.uk/government/ publications/cdei-review-of-online-targeting
• 25 INFOSEC (US). Privacy risks of sleep-tracking devices [Internet]. New York, US: INFOSEC Institute; 2014; [access in 2020 Jun 8]. Available from: https://resources.infosecinstitute.com/privacy-risks-sleep-tracking-devices/
• 26 WIRED (UK). Strava’s data lets anyone see the names (and heart rates) of people exercising on military bases [Internet]. London, UK: WIRED; 2018 Jan; [access in 2020 Apr 28]. Available from: https://www.wired.co.uk/article/strava-military-bases-area-51-mapafghanistan- gchq-military
• 27 He D, Naveed M, Gunter CA, Nahrstedt K. Security Concerns in Android mHealth Apps. AMIA Annu Symp Proc. 2014 Nov;2014:645-54.
• 28 Watson NF, Lawlor C, Raymann, Roy JEM, Choi K. Will consumer sleep technologies change the way we practice sleep medicine?. J Clin Sleep Med. 2019;15(1):159-61.
• 29 NHS (UK). Diagnostic waiting times and activity data. Leeds: NHS; 2017.
• 30 Shin H, Cho J. Unconstrained snoring detection using a smartphone during ordinary sleep. BioMed Eng OnLine. 2014;13:116.
• 31 Kim Y, Lee B, Choe EK. Investigating data accessibility of personal health apps. J Am Med Inform Assoc. 2019;26(5):412-9.
• 32 Khosla S, Deak MC, Gault D, Goldstein CA, Hwang D, Kwon Y, et al. Consumer sleep technologies: how to balance the promises of new technology with evidence-based medicine and clinical guidelines. J Clin Sleep Med. 2019;15(1):163-5.
• 33 Spadafora A. Only one in ten NHS Trusts are fully digitized. TECHRADAR PRO [Internet]. 2019; [cited 2020 Apr 26]. Available from: https://www. techradar.com/uk/news/only-one-in-ten-nhs-trusts-are-fully-digitized
• 34 Kelly JM, Strecker RE, Bianchi MT. Recent developments in home sleep-monitoring devices. ISRN Neurology. 2012;2012:768794.

Publication History

Received: 04 May 2020

Accepted: 14 July 2020

Article published online:
30 November 2023