Keywords
Adversarial attacks - machine learning - medical images - privacy
1 Introduction
In recent years, data produced by medical systems has grown exponentially. The processing and knowledge extraction from these data contributed to the development of the so-called big data [[1]]. Medical systems produce complex data from sensors, imaging, or genomics data, among others. Medical complex data are essential for decision-making and generating new knowledge. Large amounts of medical images are collected to support physicians in the diagnosis process and to help identify disease patterns. Decision-making strategies are based on classical machine learning (ML) or deep learning (DL). Physicians can integrate ML techniques to analyze and assist decision-making, considering the recommendations of the models to enhance the diagnosis precision [[1], [2]].
Although ML can improve physicians' decision-making, ML methods applied to health systems can also suffer attacks [[2], [3], [4]]. Attacks on ML methods correspond to a study field called adversarial attacks (AA), which builds methods to train and test models in adversarial environments [[3], [4]]. ML methods are susceptible to attacks, such as poisoning the training data (data poisoning), bypassing the test data (evasion attack), invalidating the model, and exploiting backdoors [[2]]. For instance, [Figure 1] illustrates an example of AA on optical coherence tomography (OCT) images using the projected gradient descent (PGD) [[5]] attack.
Fig. 1 Example of adversarial attack using projected gradient descent (PGD) on optical coherence tomography (OCT) image.
Unfortunately, DL models for health systems are vulnerable to AA and suffer from privacy risks. According to [[4]], systems that handle sensitive health data need to be designed to consider privacy risks. Concerning privacy, many governments have defined regulations to formalize organizations' data handling since growing data leakage resulted in decreasing systems confidence. The European Union proposed the general data protection regulation (GDPR) to establish rules and rights to manage sensitive data. Furthermore, in 1996 the United States proposed a regulation to handle medical data called the health insurance portability and accountability act (HIPAA). We explain all acronyms in [Table 1].
Table 1 Explanations of acronyms.
This paper investigates security and privacy in ML for health, considering three perspectives: the most common attacks, suitable defenses, and privacy-preserving strategies. Finally, we highlight in this survey the following main contributions: (i) current taxonomies for security and privacy in ML for health systems; (ii) trends in attacks, defenses, and privacy-preserving strategies during the last years (2018-2022); (iii) challenges from developing attacks, defenses to detect and mitigate attacks, as well as to employ privacy methods in ML; (iv) tools and databases most applied to run experiments in security and privacy in ML for health; and (v) a summary of most relevant studies that cover strategies for attacks, defense, and privacy.
2 Background
This section addresses essential concepts about security and privacy in ML. They motivate studying security and privacy in the health environment when handling sensitive information.
2.1 Security in Machine Learning
ML methods are susceptible to adversarial attacks (AA). AA can exploit vulnerabilities in ML models and data [[2], [4]]. Adversarial example (AE) is formally defined in [Equation 1], which minimizes the distance between AE and the original example using the Euclidean distance. [Equation 1] represents an AE as , the original example as , the noise level as , the class label as , and the loss function of the ML algorithm . An AE aims to induce a visual perception like the original example, fooling the ML model during the test or training phases. The AA seeks to maximize the loss of the ML algorithm, mainly used for DL methods [[1], [3]]. According to [[3], [4]], the security in ML for health can involve attacks and defense methods.
AA for health cover features such as capabilities, system violations, knowledge, perturbation metrics, and classification or segmentation tasks [[3], [4]]. The objective of the attack can be poisoning or evasion. Poisoning attacks affect the training set, and evasion attacks affect the test set. System violations define which features of the system the attacker attempts to bypass. System violations target integrity, availability, and privacy. Attacker knowledge defines the permission level. The permissions are: (i) limited (black-box), which only explores the interface to access the model and test it; partial (gray-box), which explores a specific part of the system, such as the parameters; open (white-box) that targets several structures of the models, such as the hyperparameters and database. The perturbation metrics are used to craft examples and generate AE. Usually, these metrics are based on distances such as , , and . Examples of AA are: the fast gradient sign method (FGSM) [[6]], projected gradient descent (PGD) [[5]], One Pixel [[7]], jacobian saliency map attack (JSMA) [[8]], DeepFool [[9]], carlini & wagner (C&W) [[10]], and universal adversarial perturbations (UAPs) [[11]]. Besides, attacks can be against pre-processing algorithms, such as Image Scaling [[12], [13]]. Finally, the defenses to mitigate attacks are adversarial training [[5]], feature squeezing [[14]], defensive distillation [[15]], and generative adversarial network (GAN)-based (e.g., Magnet) [[16]].
2.2 Privacy in Machine Learning
Organizations have been concerned about privacy due to the growing data leakage and establishing of privacy regulations, such as GDPR [[17]
[18]
[19]]. Privacy violations are increasing and require mitigation. ML models can suffer data leakage, resulting in privacy disasters for organizations. According to [[18],[19]], challenges to privacy in ML include developing robust defenses to mitigate attacks, such as membership inference or re-identification. Threat models, attacks, defenses, and features categorize privacy in ML. Threat models can be Linkage [[20]], Inference [[21]], and Re-identification [[22]]. Attacks are Feature Estimation [[23]], Membership Inference [[24]], Model Memorization [[25]], Extraction [[26]], and DeepFakes [[27]]. Attack features are knowledge of the attacker and attack objective. The attacker's knowledge are black-box, gray-box, and white-box. Moreover, the attack objective targets models or training data [[19]].
Privacy-preserving strategies are obfuscation, cryptography, and aggregation [[19]]. Obfuscation methods hide sensitive variables using perturbations that seek to mitigate privacy risks, such as differential privacy (DP) [[28]] and GAN-based ones. Cryptographic methods use algorithms to hide user identities, using homomorphic encryption (HE) [[29]] and multiparty computation (MPC) [[30]]. These methods encrypt sensitive information, enabling complex operations on the encrypted data [[19]]. The aggregation methods work on collaborative training, including federated learning (FL) [[31]]. FL creates clean models and sends them to the organizations that handle sensitive data. These organizations train models on sensitive data without making it public and send the trained model to a server that aggregates the models on a general model [[17]
[18]
[19]].
3 Materials and Methods
We applied a methodology of software engineering proposed by [[32]] to conduct this research on security and privacy in ML for health. We investigate papers from 2018 to 2022. This section describes the method applied to search and select the relevant papers. We carried out the methodology encompassing the six steps, as follows: (i) define the research questions; (ii) select the databases; (iii) select the proper keywords; (iv) define the search string; (v) define inclusion and exclusion criteria; (vi) perform data extraction. The main purpose of this research is to identify strategies and issues of security and privacy in ML for health.
We define our research question to guide this work. First, we did an initial search to raise relevant papers and authors from the literature based on papers [[2]] and [[4]]. Afterward, we did a manual search to analyze papers that cited [[2]] and [[4]]. Also, we selected papers by analyzing abstracts and titles to collect important topics of security and privacy in ML for health. Thus, we collected candidate papers and analyzed their discussions, including or excluding papers if following the main topic (security and privacy in ML). Finally, referring to the papers collected, we defined research questions that guided the selection of the set of studies: (i) what state-of-art attack the study applied? (ii) has it employed defense to mitigate the attack? (iii) which features of defense contribute to mitigate the attacks? (iv) has it applied privacy-preserving ML techniques? (v) what metrics were applied to quantify attacks and defenses in machine learning for health?
The databases selected were the most used ones in computer science for health research, following the study of [[33]], such as ACM Digital Library, IEEE Explore, PubMed, Web@Science, and ScienceDirect. The percentage of papers found in each database are: EI Compendex (25.64%), ACM Digital Library (24.44%), IEEE Explore (1.13%), PubMed (3.98%), Web@Science (2.03%), and ScienceDirect (42.78%). Based on the research questions and topics, we selected the keywords most commonly used in the candidate papers initially collected. We used the Mendeley platform[1] to identify common keywords. The keywords selected were adversarial machine learning, privacy, security, deep learning, medical systems, medical image, and healthcare systems.
The search string was drawn to cover variants of topics related to deep learning, machine learning, adversarial attacks, privacy, and medical systems. We identified relevant topics dependent on the manual search and fine-tuned terms based on papers [[2]] and [[4]], as well as the most cited authors. Our search string was defined in the review process based on the initial search conducted by the following steps:
-
Manual search based on papers cited and keywords extracted from [[2]].
-
We select the most used databases for Computer Science, such as ACM digital library, IEEE Explore, IE Compedex, Web@Science, PubMed, and Science Direct. These databases are collected according to [[32]] and validated on papers [[2]
[3]
[4], [18], [19]] that are reviews related to security or privacy in ML.
-
We extracted keywords from papers [[2]
[3]
[4], [18], [19]] and fine-tuned keywords using the Mendeley platform that stored papers from manual searches. The keywords are adversarial machine learning, privacy, security, deep learning, medical, medical image, and healthcare.
-
Having to define keywords, we composed the search string, placed in the box as follows:
(„deep learning“ OR „machine learning“ OR „artificial intelligence“) AND („medical“ OR „healthcare“) AND („medical image“ OR „medical imaging“) AND („adversarial attacks“ OR „adversarial perturbations“ OR defenses or privacy)
Therefore, after searching papers in the database, we refined the relevant papers, and we applied a selection criteria to include or exclude primary studies. The inclusion criteria are:
-
The study addresses any topic about adversarial attacks or defenses of machine learning in the medical field;
-
The study addresses any topic about privacy concerns in machine learning applied to the medical field;
-
The study includes strategies of attack or defense in machine learning applied to the medical field;
-
The paper is applied to complex data, such as medical images;
-
The study is a research paper;
-
The study is available;
-
The study was written in English.
Also, we defined the following exclusion criteria:
-
The study is not related to machine learning security or privacy in the medical field;
-
The study does not discuss strategies or problems of adversarial attacks, defenses, or privacy applied to the medical field;
-
The paper is not aimed to complex data;
-
The study is gray literature, i.e., tutorials, electronic books, pre-print, tutorials, or technical reports;
-
The paper is not available;
-
The study is not written in English.
Finally, we collected the papers based on the search string and stored them in the Mendeley platform. These papers are fine-tuned by removing duplicates, considering their title and abstract, and analyzing the full papers. The data extraction comprehends relevant information from studies, such as title, authors, interest topics, strategies applied, and challenges. To complete the data extraction, we defined two taxonomies for security and privacy in ML for health, respectively. [Figure 3] describes the security taxonomy inspired in [[3]]. We built the taxonomy of security following this specification: (i) we select the main topic of adversarial attack in health systems; (ii) we specify and group aspects analyzed in the literature, such as features, category, defenses, and health task; (iii) we classify strategies following [[2]] that defines which features are systems' violations, the goal, and knowledge, as well as the categories following attacks method based on gradient, optimization, and preprocessing; (iv) finally, we select strategies and papers collected from the literature that address this strategy. [Figure 4] shows the taxonomy of privacy inspired in [[19]] that collects the following aspects from literature: (i) the main topic; (ii) the group of aspects analyzed, such as threat model, attacks, defenses, and features; (iii) we classify strategies following features and defenses, for instance, according to [[19]] defenses are obfuscation, cryptography, and aggregation; (iv) the strategies selected correspond to papers collected from the literature that address these strategies for health task.
4 Result
This section presents our findings about security and privacy in ML for health, based on the selected literature works from 2018 to 2022. [Figure 2] shows a pipeline to collect the papers. We followed three stages: (i) search strategy, (ii) selection and exclusion criteria, and (iii) data extraction.
Fig. 2 Pipeline of the literature review. This review collects relevant papers from the literature from 2018 to 2022, including security and privacy in machine learning for health. The research issues focus on adversarial attacks, defenses, and privacy concerns.
Based on [Figure 2], we describe the following stages. Stage 1, the search strategy, defined the initial search following a manual search to collect primary studies, such as relevant surveys and reviews. Such reviews are [[2]] and [[4]] used to guide research questions outlined in Section 3. Also, based on the manual search, we selected databases regarding computer science and medicine related to [[2]] and [[4]]. The main topics collected from the manual search assisted in defining the keywords: adversarial machine learning, privacy, security, deep learning, medical systems, medical image, and healthcare systems. We built a search string based on keywords and constraining them for the period between 2018 to 2022, English language, and if it is a research paper. Stage 1 returned 1,330 primary studies that will be fed to stage 2. Stage 2 filters studies following the selection and exclusion criteria (see Section 3). We removed 246 duplicated papers, reducing from 1,330 to 1,084 papers. Thereafter, we filtered papers by title and abstract, removing 880 from 1,084 to 204 papers. Finally, we filtered from 204 to 40 papers by analyzing the full text. Stage 3 does the data extraction, considering the main topics about adversarial attacks, defenses, and privacy results. In summary, we identify titles, authors, challenges, and strategies applied to the main topics posed.
4.1 Overview
This section presents an overview of results that summarize the main strategies and the taxonomy proposed. Our findings comprehend 40 papers related to 3 domains: attacks with 17 papers (42.50%), defenses with 14 papers (35.00%), and privacy with 9 papers (22.50%). The main topics of the papers are strategies to attack DL classifiers tested on medical images, techniques to identify or mitigate attacks, and strategies to privacy-preserving medical images with sensitive attributes. In the literature, most attacks applied in DL for healthcare are FGSM (23.53%) [[4]], PGD (11.76%) [[5]], GAN-based (17.65%) [[34]], and UAPs (11.76%) [[11]]. Furthermore, we found that the most employed defenses are frequency domain (13.33% of the papers), GANs (26.67% of the papers), and adversarial training (20.00% of the papers) to mitigate or identify AE.
We proposed two taxonomies to summarize the main strategies found and to classify the papers collected. We were inspired by [[3]] and [[19]] to build our taxonomies and extend them to DL for healthcare. [Figure 3] presents a taxonomy of security in ML for health, regarding the attack category, attacker knowledge, defense features, and defense category. Attacks are classified into categories: Gradient-based, Optimization, and pre-processing. Other significant aspects of attacks are the features that classify a system violation, the objective, and the knowledge. Defensive methods are organized as pre-processing, identification with out-of-distribution (OOD) and GANs, mitigation with frequency domain and adversarial training, as well as the Surrogate model with GANs. Our taxonomy classifies the papers as targeting the attack strategy or defense strategy.
Fig. 3 Taxonomy of security in machine learning for health. The figure shows the definition of adversarial attacks (bottom part) and defensive methods (upper part).
Our results show that the most employed strategies for privacy-preserving in ML are: FL [[31]] with 44.44% of the papers, DP [[28]] with 22.22% of the papers, HE [[29]] with 11.11% of the papers, and MPC [[30]] with 11.11% of the papers (see Section 2). Moreover, DL models could be attacked for feature estimation [[23]], membership [[24]], model memorization [[25]], and extraction [[26]]. The privacy attacks are modeled by the threat model following linkage [[20]], inference [[21]], and re-identification [[22]]. These results are summarized in our taxonomy of privacy in ML for health that define relevant topics and papers addressing privacy-preserving strategies. [Figure 4] illustrates our taxonomy, drawing papers that applied the strategy presented in green squares. The threat models are state-of-art papers, such as [[20]
[21]
[22]]. In addition, attacks are outlined in papers [[23]
[24]
[25]
[26]
[27]]. For health, the defensive methods most employed are DP [[28], [68], [70]], GAN-based [[65], [71]], MPC [[67]], HE [[72]], and FL [[66], [67], [69], [72]]. Also, we list privacy features in ML, such as permission level and attack objective.
Fig. 4 Taxonomy of privacy in machine learning for health. The figure shows the definition of mitigation methods (center part), privacy attacks (left part), and features of attacks (right part).
In terms of medical datasets, papers in the literature are usually collected from public (e.g., Kaggle[2]) or private datasets, selecting different categories of medical images, such as X-ray, Computed Tomography (CT), Magnetic Resonance Imaging (MRI), Dermoscopy, Fundoscopy, and OCT. Most images analyzed correspond to brain, chest, skin, and eye, as well as COVID-19 images. [Figure 5](a) shows the most employed medical datasets, including X-ray, CT, MRI, Dermoscopy, Fundoscopy, and EHRs. These datasets are exploited to generate attacks or to build defenses and privacy-preserving strategies. X-ray images are widely addressed to outline attack and defense strategies. Note that the papers collected from the literature are detailed in [Tables 2], [3], and [4].
Fig. 5 literature review results related to most used medical datasets, attacks, defenses, and privacy-preserving strategies.
Table 2 Summary of paper about attacks in machine learning for health.
Table 3 Summary of paper about defenses in machine learning for health.
Table 4 Summary of paper about privacy in machine learning for health.
The papers collected from the literature described target databases, attack methods (see [Table 2]), defense methods (see [Table 3]), and privacy-preserving strategies (see [Table 4]). The next section describes the highlighted strategies applied to the attack, defense, and privacy-preserving ML models in health.
4.2 Highlighted Strategies of Security in Machine Learning for Health
Security strategies in ML for health applications must be aware of attacks and defenses for ML models. We summarized the literature collected from attacks in [Table 2] and defenses in [Table 3].
Papers have applied attacks such as FGSM, PGD, One Pixel, and UAPs. Furthermore, the authors propose strategies to attack the segmentation or classification task. Such papers [[38], [46], [48]] investigated attacks to fool the segmentation task using UNet[3] to generate perturbed masks. In the classification task, papers [[35]] and [[41]
[42]
[43]
[44]] employed the FGSM attack, [[35], [41], [44]] the PGD attack, [[39], [40]] the UAP attack, [[37]] the One Pixel attack, and [[46], [48], [60]] GANs-based attack. As far as DeepFake attacks are concerned, which generate fake data, e.g., inserting a malign tumor into a medical image that is supposed to be benign. These papers collect medical databases, including diverse categories. [Figure 5](b) illustrates the categories of the most employed data by studies collected from the literature. The authors mostly applied Chest X-rays images due to the COVID-19 pandemic. Also, they have employed images of Dermoscopy, Fundoscopy, and OCT, as well as EHRs. Finally, MRI and CT images are applied less frequently than X-rays.
Regarding defenses in adversarial environments for health, papers explore strategies based on pre-processing, identification, mitigation, and surrogate models. Trending strategies are identifying attacks with GANs [[51], [52]] and OOD [[57]]. In addition, [[53], [55], [59]
[60]
[61]
[62]
[63]
[64]] develop strategies to mitigate AA using adversarial training and applying GANs. GANs methods could generate synthetic AE to teach models the features of an AE during adversarial training. The work in [[49]] aims to identify anomalies in the frequency domain and mitigate AA in medical images. Besides, we found that the surrogate models can reduce attack effects, and GANs strategies tend to be most applied in this context, such as [[54]], [[59]
[60]
[61]
[62]
[63]]. [Figure 5](c) summarizes the number of studies that proposed defensive strategies, corresponding attacks, and defenses. To evaluate defenses, we raised attacks FGSM, PGD, One Pixel, C&W, segmentation, and GANs. FGSM and segmentation attacks are most used to test the strengths of defenses, such as identification, GANs, surrogate model, frequency domain, and adversarial training. Furthermore, papers mostly build defenses based on surrogate models with GANs.
4.3 Highlighted Strategies of Privacy in Machine Learning for Health
We analyzed trends in privacy in ML for health (detailed in [Table 4]), collecting privacy-preserving strategies, such as FL, DP, HE, and MPC, as well as other defenses against re-identification attacks. Note that [[68], [70]] and [[64], [71]] present obfuscation with DP and GANs, respectively. Other strategies are the cryptographic one with MPC in [[67]] and HE in [[72]]. Besides, the most employed strategy is the FL addressed by [[66], [67], [69], [72]] to protect privacy in datasets of X-rays images, Electronic Health Records (EHR), and Electrocardiograms (EGG). The works in [[66], [67], [68], [72]] carry out strategies to protect DL models trained on X-ray images. Another important issue is tackled in [[65]], which investigates protections against re-identification in Picture Archiving and Communication Systems (PACS), and [[70]] that mitigates leakage in EHRs data. On the other hand, [[27]] generates DeepFake images in EGGs.
We observed that privacy-preserving strategies are evaluated over attacks against privacy. [Figure 5](d) shows re-identification, inference, and DeepFakes attacks against FL, DP, HE, and MPC. FL is the most applied privacy-preserving strategy, which privately trains minimal models to share sensitive data. Our findings corroborate that the inference attack is the most applied attempt to infer sensitive attributes from a dataset. Re-identification attacks are exploited in the health context as well. DP and HE play a role in mitigating re-identification attacks in datasets of images and EHR. In summary, the papers related to AA, defenses, and privacy concerns contribute to improving the discussion of security and privacy in ML for health.
4.4 Tools
Tools are established in the literature to produce attack, defense, and privacy-preserving strategies. AA can be generated using Python libraries SecML[4], Adversarial Robustness Toolbox[5] (ART), and TorchAttacks[6]. ART is practical because it implements tools to generate adversarial attacks and defenses, including attacks and defenses for privacy. SecML and TorchAttacks run AA, such as FGSM, PGD, One Pixel, and others. SecML works on TensorFlow models and TorchAttacks on Pytorch models. Moreover, the most used tools to build privacy-preserving strategies are TensorFlow or PyTorch for FL, PyDP for DP from DeepMind, Microsoft SEAL for HE, and MPyC. Finally, ART can run defenses against AA, such as Adversarial Training and Defensive Distillation.
5 Discussion
This section examines trends and challenges related to attacks, defenses in adversarial environments, and privacy concerns for health.
5.1 Trends
We presented the tendencies and directions of AA regarding defenses and privacy concerns in ML for health. [Figure 6] presents a timeline of the primary studies that introduced trends of AA, including defenses and privacy-preserving. Likewise, we define a timeline from 2018 to 2022 that includes the main trends. Directions in AA for health address PGD and FGSM attacks used to generate efficient AE and to distort DL models [[35]]. DL models ResNet50, VGG19, VGG16, and InceptionV3 are widely employed to classify medical images. Besides, attacks target to corrupt these models and reduce their performance. The trends for analyzing attacks started in 2018, exploring attacks vulnerabilities to corrupt NasNet and Inception trained on medical images. In 2019, the papers exploited the attacks FGSM, PGD, segmentation, and GAN-based, as well as attacks to generate DeepFakes on medical images. In 2020, papers employed attacks to build DeepFakes, and run the One Pixel attack. The attack trends in 2021 were UAPs and DeepFakes generators. In addition, trends in 2022 address UAPs, FGSM, and other strategies, such as attacks based on watermarks. Finally, the directions to develop new attacks in medical images in the next years follow the DeepFakes generator and UAPs.
Fig. 6 Timeline of papers collected from the literature between 2018 and 2022. Each paper covers the privacy, attack, or defense domain.
Defenses against AA for health systems need to improve the model's robustness. [Figure 6] illustrates the timeline regarding the trends of attacks, defenses, and privacy-preserving strategies. Our review did not find defenses against AA for the health environment from 2018 to 2019. Trends for defenses in 2020 focused on mitigating attacks using GANs, adversarial training, and detecting corrupted models. GANs strategies improve the robustness of the discriminator model to identify AE and the generator to reconstruct examples without perturbations [[51], [54]]. In 2021, defenses focused on identifying attacks employing GANs and OOD strategies. Directions in 2022 were towards of creating novel strategies, such as Stochastic Coordinate Descent [[55]], perturbation denoising [[56], [60], [61]], and enhanced adversarial training [[61], [65]]. In health environments, we need to improve model defenses at the system level due to the sensitivity of the data handled. According to our extracted data, we should develop novel defenses based on GANs and propose new strategies for health systems.
Privacy-preserving trends have led to strategies to mitigate the leakage of sensitive-health data in ML for health. In 2018, according to our review, papers mainly addressed methods to mitigate re-identification attacks in PACS and integrate ML methods into medical images. Tendencies in 2019 were protecting mini-batches of DL models and EHRs by employing DP to obfuscate the original content. In 2021, papers committed FL to share medical images, DP to protect sensitive attributes, and GANs for generating synthetic attributes based on sensitive attributes. Directions in 2022 tend to generate synthetic and sensitive data to hide the original content and combine privacy strategies to enhance FL, such as HE. Moreover, literature on health privacy tends to combine FL, DP, or MPC [[67]]. When handling unstructured data, such as images, privacy protection methods are needed to improve their protection.
5.2 Challenges
The scenario of AA in health systems has challenges, such as building powerful defenses to the AA, which focus on poisoning and evasion, bypassing fine-tuning methods, transferability of attacks, and attacks on real-world databases. Poisoning and evasion attacks aim to explore vulnerabilities in DL, applying UAPs, AutoAttack [[73]], and GANs. DL models tend to use fine-tuning strategies. When an attack affects DL models, a challenge is to develop a method to bypass the fine-tuning strategy aimed at enhancing misclassification. Attack transferability is a relevant feature that can handle and indicate whether an attack is transferable to other domains. A challenge is treating transferability when building an attack to make it more generalizable. Developing attacks in real-world environments is arduous because the attack may have another behavior that needs fixing in the modeling phase.
Defensive methods are necessary and pose an arduous task in protecting ML for health. We collected the main challenges to creating defensive strategies using proactive and reactive approaches for applying GANs, equilibrating privacy and defense, and calibrating models. Proactive defenses identify attacks before the attack happens, and reactive defenses work after the attack happens, aimed at mitigating the ill effects. Likewise, GANs are methods for building robust defenses because they can simulate attack scenarios and generate synthetic data to emulate malicious ones. Equilibrating privacy and defenses are challenging because defenses can show more information than they should. Based on privacy concerns, papers achieve a calibrated approach as an alternative to improve the model security, because it represents a more robust approach.
We observe that privacy-preserving strategies are challenging to develop Federated Learning (FL) privacy, equilibrating privacy and accuracy scores, as well as setting the privacy budget, protecting privacy in medical images, and combining privacy methods. However, in FL, the aggregate server can suffer attacks, and its security should be improved. Besides, the privacy strategy can reduce the DL model's performance. Privacy in unstructured data is challenging because the methods proposed, such as DP, work better with tabular data. Then, we must explore the method of privacy-preserving that works in medical images. The combination of privacy techniques should be a robust strategy to improve other methods, such as combining Federated Learning (FL) with Differential Privacy (DP) or FL with MultiParty Computation (MPC). Another challenge to combine these techniques is to find a suitable method to improve the privacy budget while keeping the accuracy level.
Finally, we highlight that the development of novel attacks, defenses, and privacy strategies have room for improvement. Each technique can contribute to another, such as exploring vulnerabilities to produce attacks leads to building novel defenses. Defensive methods can improve the robustness of DL models. Nevertheless, it can result in privacy issues. Thus, the defense method will be modeled based on gaps in defenses. In turn, privacy strategies are concerned with the performance of models because high-budget privacy levels can result in poor model accuracy.
6 Conclusion
We presented a survey on recent works from the literature and discussed health-related strategies and challenges regarding security and privacy in ML for health systems. We classified the papers into three domains: security, defenses against adversarial attacks (AA), and privacy concerns. The AA strategies cover gradient and optimization attacks, as well as defenses inspired by GANs to make adaptive strategies and generate synthetic Adversarial Examples (AE).
Regarding privacy, the strategies frequently applied are based on FL. However, each strategy comprehends issues, such as attacks that bypass fine-tuning, defenses that work reactively and proactively, and privacy based on methods for unstructured data. In summary, we highlight that security and privacy for health systems remain a strong trend for the next years. According to [[17]], developing ML models on sensitive data should always consider their risk and vulnerability.
