A Proposal for a Robust Validated Weighted General Data Protection Regulation-Based Scale to Assess the Quality of Privacy Policies of Mobile Health Applications: An eDelphi Study

Jaime Benjumea; Jorge Ropero; Enrique Dorronzoro-Zubiete; Octavio Rivera-Romero; Alejandro Carrasco

doi:10.1055/a-2155-2021

Methods of Information in Medicine, Table of Contents

CC BY 4.0 · Methods Inf Med 2023; 62(05/06): 154-164
DOI: 10.1055/a-2155-2021

Original Article for a Focus Theme

A Proposal for a Robust Validated Weighted General Data Protection Regulation-Based Scale to Assess the Quality of Privacy Policies of Mobile Health Applications: An eDelphi Study

Jaime Benjumea

¹Department of Electronic Technology, Universidad de Sevilla, Sevilla, Spain

,

Jorge Ropero

¹Department of Electronic Technology, Universidad de Sevilla, Sevilla, Spain

,

Enrique Dorronzoro-Zubiete

¹Department of Electronic Technology, Universidad de Sevilla, Sevilla, Spain

,

Octavio Rivera-Romero

¹Department of Electronic Technology, Universidad de Sevilla, Sevilla, Spain

,

Alejandro Carrasco

¹Department of Electronic Technology, Universidad de Sevilla, Sevilla, Spain

› Author Affiliations

Abstract

Background Health care services are undergoing a digital transformation in which the Participatory Health Informatics field has a key role. Within this field, studies aimed to assess the quality of digital tools, including mHealth apps, are conducted. Privacy is one dimension of the quality of an mHealth app. Privacy consists of several components, including organizational, technical, and legal safeguards. Within legal safeguards, giving transparent information to the users on how their data are handled is crucial. This information is usually disclosed to users through the privacy policy document. Assessing the quality of a privacy policy is a complex task and several scales supporting this process have been proposed in the literature. However, these scales are heterogeneous and even not very objective. In our previous study, we proposed a checklist of items guiding the assessment of the quality of an mHealth app privacy policy, based on the General Data Protection Regulation.

Objective To refine the robustness of our General Data Protection Regulation-based privacy scale to assess the quality of an mHealth app privacy policy, to identify new items, and to assign weights for every item in the scale.

Methods A two-round modified eDelphi study was conducted involving a privacy expert panel.

Results After the Delphi process, all the items in the scale were considered “important” or “very important” (4 and 5 in a 5-point Likert scale, respectively) by most of the experts. One of the original items was suggested to be reworded, while eight tentative items were suggested. Only two of them were finally added after Round 2. Eleven of the 16 items in the scale were considered “very important” (weight of 1), while the other 5 were considered “important” (weight of 0.5).

Conclusion The Benjumea privacy scale is a new robust tool to assess the quality of an mHealth app privacy policy, providing a deeper and complementary analysis to other scales. Also, this robust scale provides a guideline for the development of high-quality privacy policies of mHealth apps.

Keywords

participatory health informatics - mHealth apps - privacy policies - assessment scale - GDPR

Full Text

References

References
1 Swan M. Health 2050: the realization of personalized medicine through crowdsourcing, the quantified self, and the participatory biocitizen. J Pers Med 2012; 2 (03) 93-118
2 Denecke K, Gabarron E, Petersen C, Merolli M. Defining participatory health informatics - a scoping review. Inform Health Soc Care 2021; 46 (03) 234-243
3 Caulfield BM, Donnelly SC. What is Connected Health and why will it change your practice?. QJM 2013; 106 (08) 703-707
4 Whittaker R. Issues in mHealth: findings from key informant interviews. J Med Internet Res 2012; 14 (05) e129
5 Riley WT, Rivera DE, Atienza AA, Nilsen W, Allison SM, Mermelstein R. Health behavior models in the age of mobile interventions: are our theories up to the task?. Transl Behav Med 2011; 1 (01) 53-71
6 Iribarren SJ, Cato K, Falzon L, Stone PW. What is the economic evidence for mHealth? A systematic review of economic evaluations of mHealth solutions. PLoS One 2017; 12 (02) e0170581
7 Ghani Z, Jarl J, Sanmartin Berglund J, Andersson M, Anderberg P. The cost-effectiveness of mobile health (mHealth) interventions for older adults: systematic review. Int J Environ Res Public Health 2020; 17 (15) 5290
8 Rinaldi G, Hijazi A, Haghparast-Bidgoli H. Cost and cost-effectiveness of mHealth interventions for the prevention and control of type 2 diabetes mellitus: a systematic review. Diabetes Res Clin Pract 2020; 162: 108084
9 de Batlle J, Massip M, Vargiu E. et al; CONNECARE-Lleida Group. Implementing mobile health-enabled integrated care for complex chronic patients: intervention effectiveness and cost-effectiveness study. JMIR Mhealth Uhealth 2021; 9 (01) e22135
10 Giunti G, Guisado Fernández E, Dorronzoro Zubiete E, Rivera Romero O. Supply and demand in mhealth apps for persons with multiple sclerosis: systematic search in app stores and scoping literature review. JMIR Mhealth Uhealth 2018; 6 (05) e10512
11 Martín Payo R, Harris J, Armes J. Prescribing fitness apps for people with cancer: a preliminary assessment of content and quality of commercially available apps. J Cancer Surviv 2019; 13 (03) 397-405
12 Byambasuren O, Sanders S, Beller E, Glasziou P. Prescribable mHealth apps identified from an overview of systematic reviews. NPJ Digit Med 2018; 1: 12
13 ISO/TS 82304–2:2021. Health Software. Part 2: Health and Wellness apps – Quality and reliability. Accessed August 22, 2023 at: https://www.iso.org/standard/78182.html
14 Ribeiro N, Moreira L, Barros A, Almeida AM, Santos-Silva F. Guidelines for a cancer prevention smartphone application: a mixed-methods study. Int J Med Inform 2016; 94: 134-142
15 Robertson MC, Tsai E, Lyons EJ. et al. Mobile health physical activity intervention preferences in cancer survivors: a qualitative study. JMIR Mhealth Uhealth 2017; 5 (01) e3
16 Monteiro-Guerra F, Signorelli GR, Rivera-Romero O, Dorronzoro-Zubiete E, Caulfield B. Breast cancer survivors' perspectives on motivational and personalization strategies in mobile app-based physical activity coaching interventions: qualitative study. JMIR Mhealth Uhealth 2020; 8 (09) e18867
17 Giunti G, Kool J, Rivera Romero O, Dorronzoro Zubiete E. Exploring the specific needs of persons with multiple sclerosis for mHealth solutions for physical activity: mixed-methods study. JMIR Mhealth Uhealth 2018; 6 (02) e37
18 Benjumea J, Ropero J, Rivera-Romero O, Dorronzoro-Zubiete E, Carrasco A. Privacy assessment in mobile health apps: scoping review. JMIR Mhealth Uhealth 2020; 8 (07) e18868
19 Minen MT, Stieglitz EJ, Sciortino R, Torous J. Privacy issues in smartphone applications: an analysis of headache/migraine applications. Headache 2018; 58 (07) 1014-1027
20 Leigh S, Ouyang J, Mimnagh C. Effective? Engaging? Secure? Applying the ORCHA-24 framework to evaluate apps for chronic insomnia disorder. Evid Based Ment Health 2017; 20 (04) e20
21 O'Loughlin K, Neary M, Adkins EC, Schueller SM. Reviewing the data security and privacy policies of mobile apps for depression. Internet Interv 2018; 15: 110-115
22 Knorr K, Aspinall D, Wolters M. On the privacy, security and safety of blood pressure and diabetes apps. In: Proceedings of the IFIP (International Federation for Information Processing) International Conference on ICT (Information and Communications Technology) Systems Security and Privacy Protection. Cham, Switzerland: Springer; 2015 May 9. Presented at: IFIP (International Federation for Information Processing) International Conference on ICT (Information and Communications Technology) Systems Security and Privacy Protection; May 26–28, 2015; Hamburg, Germany; 571–584
23 Benjumea J, Ropero J, Rivera-Romero O, Dorronzoro-Zubiete E, Carrasco A. Assessment of the fairness of privacy policies of mobile health apps: scale development and evaluation in cancer apps. JMIR Mhealth Uhealth 2020; 8 (07) e17134
24 Papageorgiou A, Strigkos M, Politou E, Alepis E, Solanas A, Patsakis C. Security and privacy analysis of mobile health applications: the alarming state of practice. IEEE Access 2018; 6: 9390-9403
25 Huckvale K, Torous J, Larsen ME. Assessment of the data sharing and privacy practices of smartphone apps for depression and smoking cessation. JAMA Netw Open 2019; 2 (04) e192542
26 Sunyaev A, Dehling T, Taylor PL, Mandl KD. Availability and quality of mobile health app privacy policies. J Am Med Inform Assoc 2015; 22 (e1): e28-e33
27 Scott K, Richards D, Adhikari R. A review and comparative analysis of security risks and safety measures of mobile health apps. AJIS Australas J Inf Syst 2015; 19: 1-18
28 Zapata B, Hernández Niñirola A, Fernández-Alemán JL, Toval A. Assessing the privacy policies in mobile personal health records. In: Proceedings of the 36th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC 2014). New York, NY: IEEE; 2014; Presented at: 36th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC 2014); August 26–30, 2014; Chicago, IL; 4956–4959
29 Bachiri M, Idri A, Fernández-Alemán JL, Toval A. Evaluating the privacy policies of mobile personal health records for pregnancy monitoring. J Med Syst 2018; 42 (08) 144
30 Hutton L, Price BA, Kelly R. et al. Assessing the privacy of mHealth apps for self-tracking: heuristic evaluation approach. JMIR Mhealth Uhealth 2018; 6 (10) e185
31 Baumel A, Faber K, Mathur N, Kane JM, Muench F. Enlight: a comprehensive quality and therapeutic potential evaluation tool for mobile and web-based eHealth interventions. J Med Internet Res 2017; 19 (03) e82
32 Bondaronek P, Alkhaldi G, Slee A, Hamilton FL, Murray E. Quality of publicly available physical activity apps: review and content analysis. JMIR Mhealth Uhealth 2018; 6 (03) e53
33 Ni Z, Wang Y, Qian Y. Privacy policy compliance of chronic disease management apps in China: scale development and content evaluation. JMIR Mhealth Uhealth 2021; 9 (01) e23409
34 Robillard JM, Feng TL, Sporn AB. et al. Availability, readability, and content of privacy policies and terms of agreements of mental health apps. Internet Interv 2019; 17: 100243
35 Powell AC, Singh P, Torous J. The complexity of mental health app privacy policies: a potential barrier to privacy. JMIR Mhealth Uhealth 2018; 6 (07) e158
36 Adhikari AR, Richards D, Scott K. Security and privacy issues related to the use of mobile health apps. Paper presented at: Proceedings of the 25th Australasian Conference on Information Systems; 2014; Auckland, New Zealand; 8–10
37 Vaughan-Graham J, Cott C. Defining a Bobath clinical framework - a modified e-Delphi study. Physiother Theory Pract 2016; 32 (08) 612-627
38 Robustillo Cortés MdeL, Cantudo Cuenca MR, Morillo Verdugo R, Calvo Cidoncha E. High quantity but limited quality in healthcare applications intended for HIV-infected patients. Telemed J E Health 2014; 20 (08) 729-735
39 Administrative Simplification HIPAA. 45 CFR Parts 160, 162, and 164 (unofficial version, as amended through March 26, 2013). Accessed October 2021 at: http://www.hhs.gov/sites/default/files/ocr/privacy/hipaa/administrative/combined/hipaa-simplification-201303.pdf
40 Savage M, Savage LC. Doctors routinely share health data electronically under HIPAA, and sharing with patients and patients' third-party health apps is consistent: interoperability and privacy analysis. J Med Internet Res 2020; 22 (09) e19818
41 FTC Warns Health Apps and Connected Device Companies to Comply With Health Breach Notification Rule. FTC Press Releases. Accessed October 2021 at: https://www.ftc.gov/news-events/press-releases/2021/09/ftc-warns-health-apps-connected-device-companies-comply-health
42 European Parliament and Council. Regulation (EU) 2016/679 of the European Parliament and of the Council of 27 April 2016 on the protection of natural persons with regard to the processing of personal data and on the free movement of such data, and repealing Directive 95/46/EC (General Data Protection Regulation), EUR-Lex. Accessed October 2021 at: https://eur-lex.europa.eu/eli/reg/2016/679/2016-05-04 . Last consulted: October 2021
43 Mense MA, Urbauer P, Sauermann S, Wahl H. Simulation environment for testing security and privacy of mobile health apps. Paper presented at: Modeling and Simulation in Medicine Symposium; 2016; Pasadena, CA; 3–6
44 Dalkey N, Helmer O. An experimental application of the DELPHI method to the use of experts. Manage Sci 1963; 9 (03) 458-467
45 Linstone HA, Turoff M. The Delphi Method: Techniques and Applications. London: Addison-Wesley; 1975
46 Fischer RG. The Delphi method: a description, review and criticism. J Acad Librariansh 1978; 4: 64-70
47 Jünger S, Payne SA, Brine J, Radbruch L, Brearley SG. Guidance on Conducting and REporting DElphi Studies (CREDES) in palliative care: recommendations based on a methodological systematic review. Palliat Med 2017; 31 (08) 684-706
48 McKenna HP. The Delphi technique: a worthwhile research approach for nursing?. J Adv Nurs 1994; 19 (06) 1221-1225
49 Von der Gracht HA. Consensus measurement in Delphi studies: review and implications for future quality assurance. Technol Forecast Soc Change 2012; 79 (08) 1525-1536
50 Schmidt RC. Managing Delphi surveys using non-parametric statistical techniques. Decis Sci 1997; 28: 763-774
51 Golenko X, Pager S, Holden L. A thematic analysis of the role of the organisation in building allied health research capacity: a senior managers' perspective. BMC Health Serv Res 2012; 12 (276) 276
52 Robles N, Puigdomènech Puig E, Gómez-Calderón C. et al. Evaluation criteria for weight management apps: validation using a modified Delphi process. JMIR Mhealth Uhealth 2020; 8 (07) e16899
53 Studnek JR, Lerner EB, Shah MI. et al. Consensus-based criterion standard for the identification of pediatric patients who need emergency medical services transport to a hospital with higher-level pediatric resources. Acad Emerg Med 2018; 25 (12) 1409-1414
54 Universidad de Sevilla. Record of processing activities for “Evaluación de Políticas de Privacidad: Estudio no experimental con expertos y usuarios” (in Spanish). Accessed August 22, 2023 at: https://sic.us.es/sites/default/files/pd/rateval.pol_.priv_.pdf
55 Birko S, Dove ES, Özdemir V. Evaluation of nine consensus indices in Delphi foresight research and their dependency on Delphi survey characteristics: a simulation study and debate on Delphi design and interpretation. PLoS One 2015; 10 (08) e0135162
56 Brüggemann BT, Hansen J, Dehling T, Sunyaev A. Privacy technologies and policy. In: Schiffner S, Serna J. eds. An Information Privacy Risk Index for mHealth Apps. Cham: Springer; 2016
57 Benjumea J, Ropero J, Dorronzoro-Zubiete E, Rivera-Romero O, Carrasco A. A proposal for a robust validated weighted general data protection regulation-based scale to assess the fairness of privacy policies of mobile health applications: a Delphi study. 21 June 2022, PREPRINT (Version 1) available at Research Square

Supplementary Material

Supplementary Material