Securing Machine Learning Algorithms

DECEMBER 2021 0 SECURING MACHINE LEARNING ALGORITHMS December 2021

ABOUT ENISA

The European Union Agency for Cybersecurity, ENISA, is the Union’s agency dedicated to achieving a high common level of cybersecurity across Europe. Established in 2004 and strengthened by the EU Cybersecurity Act, the European Union Agency for Cybersecurity contributes to EU cyber policy, enhances the trustworthiness of ICT products, services and processes with cybersecurity certification schemes, cooperates with Member States and EU bodies, and helps Europe prepare for the cyber challenges of tomorrow. Through knowledge sharing, capacity building and awareness raising, the Agency works together with its key stakeholders to strengthen trust in the connected economy, to boost resilience of the Union’s infrastructure, and, ultimately, to keep Europe’s society and citizens digitally secure. More information about ENISA and its work can be found here: www.enisa.europa.eu. CONTACT For contacting the authors please use info@enisa.europa.eu For media enquiries about this paper, please use press@enisa.europa.eu EDITORS Apostolos Malatras, Ioannis Agrafiotis, Monika Adamczyk, ENISA ACKNOWLEDGEMENTS We would like to thank the Members and Observers of the ENISA ad hoc Working Group on Artificial Intelligence for their valuable input and feedback. LEGAL NOTICE Notice must be taken that this publication represents the views and interpretations of ENISA, unless stated otherwise. This publication should not be construed to be a legal action of ENISA or the ENISA bodies unless adopted pursuant to the Regulation (EU) No 2019/881. This publication does not necessarily represent state-of the-art and ENISA may update it from time to time. Third-party sources are quoted as appropriate. ENISA is not responsible for the content of the external sources including external websites referenced in this publication. This publication is intended for information purposes only. It must be accessible free of charge. Neither ENISA nor any person acting on its behalf is responsible for the use that might be made of the information contained in this publication. COPYRIGHT NOTICE © European Union Agency for Cybersecurity (ENISA), 2021 Reproduction is authorised provided the source is acknowledged. Copyright for the image on the cover: © Shutterstock For any use or reproduction of photos or other material that is not under the ENISA copyright, permission must be sought directly from the copyright holders. ISBN: 978-92-9204-543-2 – DOI: 10.2824/874249 - Catalogue Nr.: TP-06-21-153-EN-N 1 SECURING MACHINE LEARNING ALGORITHMS December 2021

TABLE OF CONTENTS

EXECUTIVE SUMMARY 3 1. INTRODUCTION 4 1.1 OBJECTIVES 4 1.2 METHODOLOGY 4 1.3 TARGET AUDIENCE 5 1.4 STRUCTURE 6 2. MACHINE LEARNING ALGORITHMS TAXONOMY 7 2.1 MAIN DOMAIN AND DATA TYPES 8 2.2 LEARNING PARADIGMS 9 2.3 NAVIGATING THE TAXONOMY 10 2.4 EXPLAINABILITY AND ACCURACY 10 2.5 AN OVERVIEW OF AN END-TO-END MACHINE LEARNING LIFECYCLE 11 3. ML THREATS AND VULNERABILITIES 13 3.1 IDENTIFICATION OF THREATS 13 3.2 VULNERABILITIES MAPPED TO THREATS 16 4. SECURITY CONTROLS 18 4.1 SECURITY CONTROLS RESULTS 18 5. CONCLUSION 26 A ANNEX: TAXONOMY OF ALGORITHMS 28 B ANNEX: MAPPING SECURITY CONTROLS TO THREATS 34 C ANNEX: IMPLEMENTING SECURITY CONTROLS 38 D ANNEX: REFERENCES 43 2 SECURING MACHINE LEARNING ALGORITHMS December 2021

EXECUTIVE SUMMARY

The vast developments in digital technology influence every aspect of our daily lives. Emerging technologies, such as Artificial Intelligence (AI), which are in the epicentre of the digital evolution, have accelerated the digital transformation contributing in social and economic prosperity. However, the application of emerging technologies and AI in particular, entails perils that need to be addressed if we are to ensure a secure and trustworthy environment. In this report, we focus on the most essential element of an AI system, which are machine learning algorithms. We review related technological developments and security practices to identify emerging threats, highlight gaps in security controls and recommend pathways to enhance cybersecurity posture in machine learning systems. Based on a systematic review of relevant literature on machine learning, we provide a taxonomy This report for machine learning algorithms, highlighting core functionalities and critical stages. The provides a taxonomy sheds light on main data types used by algorithms, the type of training these taxonomy for algorithms entail (supervised, unsupervised) and how output is shared with users. Particular machine learning emphasis is given to the explainability and accuracy of these algorithms. Next, the report presents a detailed analysis of threats targeting machine learning systems. Identified threats algorithms, a include inter alia, data poisoning, adversarial attacks and data exfiltration. All threats are detailed analysis associated to particular functionalities of the taxonomy that they exploit, through detailed tables. of threats and Finally, we examine mainstream security controls described in widely adopted standards, such security controls as ISO 27001 and NIST Cybersecurity framework, to understand how these controls can in widely adopted effectively detect, deter and mitigate harms from the identified threats. To perform our analysis, we map all the controls to the core functionalities of machine learning systems that they protect standards and to the vulnerabilities that threats exploit in these systems. Our analysis indicates that the conventional security controls, albeit very effective for information systems, need to be complemented by security controls tailored to machine learning functionalities. To identify these machine-learning controls, we conduct a systematic review of relevant literature, where academia and research institutes propose ways to avoid and mitigate threats targeting machine learning algorithms. Our report provides an extensive list of security controls that are applicable only for machine learning systems, such as “include adversarial examples to training datasets”. For all controls, we map the core functionality of machine learning algorithms that they intend to protect to the vulnerabilities that threats exploit. Our findings indicate that there is no unique strategy in applying a specific set of security controls to protect machine learning algorithms. The overall cybersecurity posture of organisations who use machine learning algorithms can be enhanced by carefully choosing controls designed for these algorithms. As these controls are not validated in depth, nor standardised in how they should be implemented, further research should focus on creating benchmarks for their effectiveness. We further identified cases where the deployment of security controls may lead to trade-offs between security and performance. Therefore, the context in which controls are applied is crucial and next steps should focus on considering specific use cases and conducting targeted risk assessments to better understand these tradeoffs. Finally, given the complexity of securing machine learning systems, governments and related institutions have new responsibilities in raising awareness regarding the impact of threats on machine learning. It is important to educate data scientists on the perils of threats and on the design of security controls before machine learning algorithms are used in organisations’ environments. By engaging experts in machine learning in cybersecurity issues, we may create the opportunity to design innovative security solutions and mitigate the emerging threats on machine learning systems. 3 SECURING MACHINE LEARNING ALGORITHMS December 2021

1. INTRODUCTION

Artificial Intelligence (AI) has grown significantly in recent years and driven by computational advancements has found wide applicability. By providing new opportunities to solve decisionmaking problems intelligently and automatically, AI is being applied to more and more use cases in a growing number of sectors. The benefits of AI are significant and undeniable. However, the development of AI is also accompanied by new threats and challenges, which relevant professionals will have to face. In 2020, ENISA published a threat landscape report on AI . This report, published with the support of the Ad-Hoc Working Group on Artificial Intelligence Cybersecurity , presents the Agency’s active mapping of the AI cybersecurity ecosystem and its threat landscape. This threat landscape not only lays the foundation for upcoming cybersecurity policy initiatives and technical guidelines, but also stresses relevant challenges. Machine learning (ML), which can be defined as the ability for machines to learn from data to solve a task without being explicitly programmed to do so, is currently the most developed and promising subfield of AI for industrial and government infrastructures. It is also the most commonly used subfield of AI in our daily lives. ML algorithms and their specificities, such as the fact that they need large amount of data to learn, make them the subject of very specific cyber threats that project teams must consider. The aim of this study is to help project teams identify the specific threats that can target ML algorithms, associated vulnerabilities, and security controls for addressing these vulnerabilities. Building on the ENISA AI threat landscape mapping, this study focuses on cybersecurity threats specific to ML algorithms. Furthermore, vulnerabilities related to the aforementioned threats and importantly security controls and mitigation measures are proposed. The adopted description of AI is a deliberate simplification of the state of the art regarding that vast and complex discipline with the intent of not precisely or comprehensively define it but rather pragmatically contextualise the specific technique of machine learning. 1.1 OBJECTIVES The objectives of this publication are: • To produce a taxonomy of ML techniques and core functionalities to establish a logical link between threats and security controls. • To identify the threats targeting ML techniques and the vulnerabilities of ML algorithms, as well as the relevant security controls and how these are currently being used in the field to ensure minimisation of security risks. • To propose recommendations on future steps to enhance cybersecurity in systems that rely on ML techniques. 1.2 METHODOLOGY To produce this report, the work was divided into three stages. At the core of the methodology was an extensive literature review (full list of references may be found in Annex D). The aim 4 SECURING MACHINE LEARNING ALGORITHMS December 2021 was to consult documents that are more specific to ML algorithms in general in order to build the taxonomy, and to consult documents more specific to security to identify threats, vulnerabilities, and security controls. At the end of the systematic review, more than 200 different documents (of which a hundred are related to security) on various algorithms of ML had been collected and analysed. First, we introduced a high-level ML taxonomy. To understand the vulnerabilities of different ML algorithms, how they can be threatened and protected, it is crucial to have an overview of their core functionalities and lifecycle. To do so, a first version of the desk research on ML-focussed sources was compiled and the ML lifecycle presented in ENISA's work on AI cybersecurity challenges was consulted . We then analysed and synthesised all references to produce a first draft of the taxonomy. The draft was submitted and interviews were held with the ENISA Ad-Hoc Working Group on Artificial Intelligence Cybersecurity. After considering their feedback, the ML taxonomy and lifecycle were validated. The second step was to identify the cybersecurity threats that could target ML algorithms and potential vulnerabilities. For this task, the threat landscape from ENISA’s report on AI cybersecurity challenges was the starting point, which was then enriched through desk research with sources related to the security of ML algorithms. Additionally, the expertise of the ENISA Ad-Hoc Working Group on Artificial Intelligence Cybersecurity was sought. This work allowed us to select threats and identify associated vulnerabilities. Subsequently, they were linked to the previously established ML taxonomy. The last step of this work was the identification of the security controls addressing the vulnerabilities. To do this, we utilised the desk research and enriched it with the most relevant standard security controls from ISO 27001/2 and the NIST 800-53 framework. The output was reviewed with the experts of the ENISA Ad-Hoc Working Group on Artificial Intelligence Cybersecurity. This work allowed us to identify security controls that were then linked to the ML taxonomy. It is important to note that we opted to enrich the ML-targeted security controls with more conventional ones to highlight that applications using ML must also comply with more classic controls in order to be sufficiently protected. Considering measures that are specific to ML would only give a partial picture of the security work needed on these applications. 1.3 TARGET AUDIENCE The target audience of this report can be divided into the following categories: • Public/governmental sector (EU institutions and agencies, Member States’ regulatory bodies, supervisory authorities in the field of data protection, military and intelligence agencies, law enforcement community, international organisations, and national cybersecurity authorities): to help them with their risk analysis, identify threats and understand how to secure ML algorithms. • Industry (including Small and Medium Enterprises (SMEs)) that makes use of AI solutions and/or is engaged in cybersecurity, including operators of essential services: to help them with their risk analysis, identify threats and understand how to secure ML algorithms. • AI technical community, AI cybersecurity experts and AI experts (designers, developers, ML experts, data scientists, etc.) with an interest in developing secure solutions and in integrating security and privacy by design in their solutions. • Cybersecurity community: to identify threats and security controls that can apply to ML algorithms. 5 SECURING MACHINE LEARNING ALGORITHMS December 2021 • Academia and research community: to obtain knowledge on the topic of securing ML algorithms and identify existing work in the field. • Standardisation bodies: to help identify key aspects to consider regarding securing ML algorithms. 1.4 STRUCTURE The report aims to help the target audience to identify the cyber threats to consider and the security controls to deploy in order to secure their ML applications. Accordingly, the report is structure into three sections: • ML algorithms taxonomy: first, a taxonomy to describe the main characteristics of the algorithms is defined. The different ML algorithms are categorised based on their core functionalities (e.g., the learning paradigm) and the lifecycle of a ML algorithm is defined. • Identification of relevant threats and vulnerabilities: secondly, a list of the cybersecurity threats and associated vulnerabilities to consider for ML algorithms is defined. Threats are mapped to the taxonomy to highlight the link between them, the core functionalities, and the lifecycle of the ML algorithms. • Security controls: thirdly, a list of security controls for addressing the previously considered vulnerabilities is given. They are also mapped to the ML taxonomy. This report focuses on threats that target ML algorithms and on the associated security controls. It is important to note that this publication examines security controls that are specific to ML algorithms as well as standard security controls that are also applicable to ML algorithms and systems making use of them. To use this publication effectively, it is important to note that: • As is the case for any application, when using ML, one must also consider traditional security standards (e.g. ISO 27001/2, NIST 800-53), because ML applications are subject not only to AI/ML specific threats but also to general nature cybersecurity threats. • The context of the application (e.g. manipulated data, business case, deployment) must be considered to correctly assess the risks and prioritise deployment of the security controls accordingly. 6 SECURING MACHINE LEARNING ALGORITHMS December 2021

2. MACHINE LEARNING ALGORITHMS TAXONOMY

One of the objectives of this work was to devise a (non-exhaustive) taxonomy, to support the process of identifying which specific threats can target ML algorithms, their associated vulnerabilities, and security controls for addressing these vulnerabilities. An important disclaimer needs to be made concerning this taxonomy, namely that it is not meant to be complete or exhaustive when it comes to ML, instead it aims to support the security analysis of ML algorithms in this report. Based on the desk research and interviews with experts of the ENISA AI Working group, we identified 40 of the most commonly used ML algorithms. A taxonomy was built based on the analysis of these algorithms. In particular, it was noted that ML algorithms were driven mainly by the learning paradigms and the problem they address (main domain). These aspects were therefore chosen to form the key taxonomy dimensions, as seen in Figure 1. It should be noted that Annex A provides a complete listing of the 40 algorithms and their mapping to the features of the taxonomy, whereas the Figure serves for illustration purposes. Figure 1: Machine Learning Algorithm taxonomy 7 SECURING MACHINE LEARNING ALGORITHMS December 2021 There is a strong correlation between the domain of application (the problem being addressed) and the data type which is being worked on, as well as between data environments and learning paradigm. Thus, further dimensions of the taxonomy were introduced accordingly. 2.1 MAIN DOMAIN AND DATA TYPES Different algorithms are used in different domains of ML. Therefore, the algorithms have been categorised according to the main domains represented. Three main domains were (nonexhaustively) selected, namely Computer Vision, NLP (Natural Language Processing) & Speech Processing (understanding and generating speech), and Classic Data Science. The inputs that are given to a ML algorithm are data and therefore, the algorithms can be categorised based on the types of data that is fed into them. In most cases, specific types of data are used in certain domains of ML. Indeed, all the algorithms used in computer vision are fed with images and videos, in the same way that all algorithms used in Natural Language Processing are fed with text . In Table 1, the main domains and the type of data used in each of them are listed. Table 1: Main domains and data types Main Data type Definition domain Visual representation of a matrix of pixels constituted of 1 channel for black and Image white images, 3 elements (RGB) for coloured images or 4 elements (RGBA) for Computer coloured images with opacity. Vision A succession of images (frames), sometimes grouped with a time series (a Video sound). NLP & Text A succession of characters (e.g. a tweet, a text field). Speech processing Time series A series of data points (e.g. numerical) indexed in time order. Data organised in a predefined model of array with one specific column for each feature (e.g. textual, numerical data, date). To be more accurate, structured data refer to organised data that can be found in a relational data base for example Classic (that may contain textual columns as mentioned). Data Structured Data Science Quantitative data can be distinguished from qualitative data. Quantitative data corresponds to the numerical data that can supports some arithmetic operations whereas qualitative data is usually used as categorical data to classify data according to their similarities. Certain domains such as NLP and Computer Vision have been separated from Classic Data Science. The purpose of this separation was to make a distinction between algorithms that may be used specifically or predominantly for each domain.

Audio data are also used for speech recognition. For the purposes of this report, we consider only text for the NLP for the taxonomy. considering that this will not create differences for the work on threats. For the purposes of this report, time series belong to the two main domains: Classic Data Science and Speech processing. By restraining Time series to Classic Data Science and Speech processing, we aspired to emphasise the specific approaches that are used for this domain like ARIMA and Hidden Markov Model. Furthermore, we include audio data under time series and made the choice to separate video from time series.

8 SECURING MACHINE LEARNING ALGORITHMS December 2021 2.2 LEARNING PARADIGMS Learning paradigm in ML relates to how a machine learns when data is fed to it. For example, all the classification and regression algorithms use labelled data, meaning that they are doing only supervised learning. Indeed, supervised learning, by definition, is the learning of labelled data, which can be either numerical (in this case, the learning paradigm is regression), or categorical (the learning paradigm is classification). An example of classification can be differentiating a cat from a dog in a picture, and an example of regression can be predicting the price of a house. On the other hand, a clustering algorithm uses unlabeled data, which is an unsupervised type of learning. Therefore, one can conclude that each learning paradigm is a specific case of one data environment. In addition to the data types fed into the algorithms, we also focused on three learning paradigms, namely supervised learning, unsupervised learning, and reinforcement learning: • Supervised learning learns a function that maps an input to an output based on example input-output pairs. It infers a function from labelled training data consisting of a set of training examples. • Unsupervised learning learns patterns from unlabelled data. It discovers hidden patterns or data groupings without the need for human intervention. • Reinforcement learning enables an agent to learn in an interactive environment by trial and error using feedback from its own actions and experiences. Table 2: Learning paradigms with typical subtypes. Learning paradigm Subtypes Definition Classification is the process of predicting the Classification class of given data points. (Is the picture a cat or a dog?) Supervised learning Regression models are used to predict a Regression continuous value. (Predict the price of a house based on its features). Clustering is the task of dividing a set of data points into several groups such that data points Clustering in the same groups are more similar each other than from the data points of the other groups. Unsupervised learning Dimensionality reduction refers to techniques Dimensionality reduction for reducing the number of input variables in training data. Rewarding is an area of ML concerned with how intelligent agents ought to take actions in an Reinforcement learning Rewarding environment to maximise the notion of cumulative reward, learning by using feedback from their experiences. Each of these learning paradigms have different security-related properties which may lead to attacks and therefore, it is relevant to represent this information in the taxonomy of ML algorithms, from which security controls will be mapped. For instance, the most common learning paradigm is classification and thus, it has many more examples of vulnerabilities due to its popularity. 9 SECURING MACHINE LEARNING ALGORITHMS December 2021 2.3 NAVIGATING THE TAXONOMY Each algorithm is placed in its corresponding cell of the taxonomy grid, according to its learning paradigm, data type and main domain. For instance, Recurrent Neural Networks (RNN), which are a type of neural network helpful in modelling sequenced data, are used for regression in supervised learning, so they must be mapped in the first column. Moreover, the data fed into them can be text, time series, images, or videos so the RNN box covers all the corresponding lines in the taxonomy. However, some of the widely used and mentioned algorithms are based on common elementary components, or are extensions of the same principle, and can therefore form families or clusters of algorithms on this taxonomy grid. Hence, we map those specific algorithms in groups by using nested boxes, as it allows for the representation of a wide variety of algorithms, while showing that some have relationships with one another. To continue with the previous example, a more recent version of RNN is LSTM (Long-Short Term Memory), which differs from RNN based on its optimisation techniques, making it faster to learn and more precise. Since LSTM is a specific extension of RNN, the LSTM box was nested in the RNN box in the taxonomy: this indicates that the two algorithms are part of the same family. 2.4 EXPLAINABILITY AND ACCURACY An important aspect of security of AI is that of explainability. Understanding the algorithms and making them explainable makes them more accessible to as many people as possible. It also helps to increase the trustworthiness of AI and support forensics and analysis of decisions. Following inputs from the desk research exercise and from the research on attacks targeting ML models, we additionally included two important parameters in the taxonomy: • Explainability: For the purposes of this study, algorithms are deemed to be "explainable" if the decision it makes can be understood by a human. That is to say, decisions can be understood by a human such as a developer or an auditor and then explained to an end-user, for example. To be fully explainable, an algorithm must be: o Globally explainable: a user can identify the features’ importance for the trained model. o Locally explainable: a user can explain why the algorithm gives a specific output (prediction) to a specific input data (features’ values). • Accuracy (probability score): Some algorithms provide, in addition to a predictive output, the probability of this prediction which can be interpreted as an “accuracy level”. If an algorithm doing classification predicts that a picture of a cat is indeed a picture of a cat at 95% accuracy, one can say that the algorithm has a “high accuracy classification”. Otherwise, if the prediction was at 55% accuracy, one could say that the algorithm has a “low accuracy classification”. It is important to note that we focused on the algorithms’ explainability because this work is important for other parts of the publication. For example, in one identified security control, it is highlighted that it is necessary to ensure that ML projects comply with regulatory constraints such as the GDPR, which describes some explainability requirements . 10 SECURING MACHINE LEARNING ALGORITHMS December 2021 2.5 AN OVERVIEW OF END-TO-END MACHINE LEARNING LIFECYCLE An ML system lifecycle includes several interdependent phases ranging from its design and development (including sub-phases such as requirement analysis, data collection, training, testing, integration), installation, deployment, operation, maintenance, and disposal. It defines the phases that an organisation should follow to take advantage of AI and of ML models in particular to derive practical business value. The latter can be represented as the architecture illustrated in Figure 22 : Figure 2: Typical AI lifecycle (from the ENISA AI Threat Landscape) Building on the AI lifecycle, we describe in Figure 3 an overview of a typical ML lifecycle with a complete overview of the principal steps.

suitable safeguards, which should include specific information to the data subject and the right to obtain human intervention, to express his or her point of view, to obtain an explanation of the decision reached after such assessment and to challenge the decision.” 9 https://www.enisa.europa.eu/publications/artificial-intelligence-cybersecurity-challenges 1 Please use footnotes for providing additional or explanatory information and/or relevant links. References should be listed in a dedicated section. Use only the function References/Insert Footnote

11 SECURING MACHINE LEARNING ALGORITHMS December 2021 The aim of the ML algorithm taxonomy is to focus not only on the functionalities of the algorithms but also on the ML models’ workflow represented by the lifecycle. This lifecycle summarises the principle steps to produce an ML model. It is important to note that several steps could have been added, such as data creation and data analysis (for instance, to analyse if there are some personal data or biases). However, to simplify the lifecycle, some steps have been condensed. Thus, for example, data cleaning has been included. Regarding data creation, it was considered as being external to the ML lifecycle. 12 SECURING MACHINE LEARNING ALGORITHMS December 2021

3. ML THREATS AND VULNERABILITIES

3.1 IDENTIFICATION OF THREATS Based on the methodology described in the Introduction and using a combination of desktop research and experts’ interviews, we identified a list of six high-level threats and seven sub-threats that were then mapped to the taxonomy. It is important to note that: • Threats against supporting infrastructures are not analysed in this publication. • All threats relate to the previous ENISA publication on the AI Threat Landscape; accordingly, they have been mapped to AI assets (environments, tools, data, etc.) The table following summarises Machine Learning threats and includes: • Threats and sub-threats definitions. • Whether they are specific to ML algorithms or not. • At which stage of the life cycle defined in the first section the threat is likely to occur. 13 SECURING MACHINE LEARNING ALGORITHMS December 2021 Table 3: Threats and sub-threats Stage of the lifecycle

g g g

Threats | sub- g n in

in in n n t

Definition n s ig in ti o n

threats n s s s ti n e g a e e ra e a n tio c d T T is tio m ri c l l l l a l y o e Cle ro e e e e e lo it a ll a a p d d d tim d lu d n o o o o a o o re v Dat Co Dat Dat P M M M Op M E M Dep M

A type of attack in which the attacker works on the ML algorithm's inputs to find small perturbations leading to large modification of its outputs (e.g. decision errors). It is as if the attacker created an optical illusion for the Evasion x algorithm. Such modified inputs are often called adversarial examples. Example: the projection of images on a house could lead the algorithm of an autonomous car to take the decision to suddenly make it brake. In some cases, the attacker has access to Use of adversarial information (model, model parameters, etc.) that examples crafted can allow him to directly build adversarial in white or grey x examples. One example is to directly use the box conditions model's gradient to find the best perturbation to (e.g. FGSM…) add to the input data to evade the model. A type of attack in which the attacker explores a model by providing a series of carefully crafted inputs and observing outputs. These attacks can be previous steps to more harmful types, evasion or poisoning for example. It is as if the attacker made the model talk to Oracle then better compromise it or to obtain x information about it (e.g. model extraction) or its training data (e.g. membership inferences attacks and Inversion attacks). Example: an attacker studies the set of inputoutput pairs and uses the results to retrieve training data. A type of attack in which the attacker altered data or model to modify the ML algorithm's behavior in a chosen direction (e.g. to sabotage its results, to insert a backdoor). It is as if the attacker conditioned the algorithm according to Poisoning x x x x x x x x its motivations. Such attacks are also called causative attacks. Example: massively indicating to an image recognition algorithm that images of dogs are indeed cats to lead it to interpret it this way. An attack in which the attacker corrupts the labels of training data. Label modification x x x x This sub-threat is specific to Supervised Learning. This threat refers to the possibility of leakage of all or partial information about the model. Model or data Example: the outputs of a ML algorithm are so x x x x x x x x x x disclosure verbose that they give information about its configuration (or leakage of sensitive data)

We have chosen to separate the oracle attacks from this threat to address the specifics of both threats and give them both a fair representation. However, Oracle-type attacks may be considered as a ML specific sub-threat of model or data disclosure.

14 SECURING MACHINE LEARNING ALGORITHMS December 2021 Stage of the lifecycle

g g g

Threats | sub- g n in

in in n n t

Definition n s ig in ti o n g

threats n a s s s ti n e e e ra e a n tio c d T T tio m ri l l l is l a l y o c Cle ro e e e e e it a e a a p lu lo ll d d d tim d a d n re o o o o v o o Dat Co Dat Dat P M M M Op M E M Dep M

This threat refers to a leak of data manipulated by ML algorithms. This data leakage can be explained by an inadequate access control, a handling error of the project team or simply because sometimes the entity that owns the Data disclosure x x x x x x x x x x model and the entity that owns the data are distinct. To train the model, it is often necessary for the data to be accessed by the model provider. This involve sharing the data and thus share sensitive data with a third party. This threat refers to a leak of the internals (i.e. parameter values) of the ML model. This model Model disclosure leakage could occur because of human error or x x x x x x x contraction with a third party with a too low security level. This threat refers to the compromise of a component or developing tool of the ML Compromise of application. ML application x x x x x x x x x x Example: compromise of one of the open-source components libraries used by the developers to implement the ML algorithm. This threat refers to ML application failure (e.g. denial of service due to bad input, unavailability due to a handling error). Example: the service level of the support Failure or infrastructure of the ML application hosted by a malfunction of ML third party is too low compared to the business x x application needs, the application is regularly unavailable. Note that this threat does not consider failure of business use cases (for example, the algorithm fails because it is not accurate enough to handle all real-life situations it is exposed to). The different stakeholders of the model can make mistakes that result in a failure or Human error malfunction of ML application. For example, due x x x x x x x x x x to lack of documentation, they may use the application in use-cases not initially foreseen. ML algorithms usually consider input data in a defined format to make their predictions. Thus, a denial of service could be caused by input data Denial of service whose format is inappropriate. It may also due to inconsistent happen that a malicious user of the model x data or a sponge constructs an input data (a sponge example) example specifically designed to increase the computation time of the model and thus potentially cause a denial of service. Cybersecurity This threat refers to the possibility that a project incident not team may not report security incidents to reported to x x x x x x x x x x dedicated teams while a policy of mandatory incident response incident reporting has been defined. teams 15 SECURING MACHINE LEARNING ALGORITHMS December 2021 3.2 VULNERABILITIES MAPPED TO THREATS To identify the security controls, we determined vulnerabilities associated with the threats described in the previous section. It is important to note that the same vulnerabilities may be found behind one or more threats (e.g. the “Poor access management” vulnerability). The table below lists vulnerabilities of ML algorithms and maps them to the aforementioned threats. Table 4: Threats and associated vulnerabilities Threats | sub-threats Vulnerabilities Lack of detection of abnormal inputs Poor consideration of evasion attacks in the model design implementation Poor consideration of evasion attacks in the model design implementation Evasion Lack of training based on adversarial attacks Using a widely known model allowing the attacker to study it Inputs totally controlled by the attacker which allows for input-output-pairs Use of adversarial examples crafted in Too much information available on the model white or grey box conditions (e.g. FGSM…) Too much information about the model given in its outputs Poor access rights management The model allows private information to be retrieved Too much information about the model given in its outputs Too much information available on the model Oracle Lack of consideration of attacks to which ML applications could be exposed to Lack of security process to maintain a good security level of the components of the ML application Weak access protection mechanisms for ML model components Model easy to poison Lack of data for increasing robustness to poisoning Poor access rights management Poor data management Undefined indicators of proper functioning, making complex compromise identification Poisoning Lack of consideration of attacks to which ML applications could be exposed to Use of uncontrolled data Use of unsafe data or models (e.g. with transfer learning) Lack of control for poisoning No detection of poisoned samples in the training dataset Weak access protection mechanisms for ML model components Label modification Use of unreliable sources to label data Poor access rights management Model or data disclosure Existence of unidentified disclosure scenarios Weak access protection mechanisms for ML model components 16 SECURING MACHINE LEARNING ALGORITHMS December 2021 Threats | sub-threats Vulnerabilities Lack of security process to maintain a good security level of the components of the ML application Unprotected sensitive data on test environments Too much information about the model given in its outputs Data disclosure The model can allow private information to be retrieved Disclosure of sensitive data for ML algorithm training Too much information available on the model Model disclosure Too much information about the model given in its outputs Poor access rights management Too much information available on the model Existence of several vulnerabilities because the ML application was not included into process for integrating security into projects Use of vulnerable components (among the whole supply chain) Too much information about the model given in its outputs Existence of unidentified compromise scenarios Compromise of ML application Undefined indicators of proper functioning, making complex compromise identification components Bad practices due to a lack of cybersecurity awareness Lack of security process to maintain a good security level of the components of the ML application Weak access protection mechanisms for ML model components Existence of several vulnerabilities because ML specificities are not integrated to existing policies Existence of several vulnerabilities because ML application do not comply with security policies Contract with a low security third party Existing biases in the ML model or in the data ML application not integrated in the cyber-resilience strategy Existence of unidentified failure scenarios Undefined indicators of proper functioning, making complex malfunction identification Failure or malfunction of ML Lack of explainability and traceability of decisions taken application Lack of security process to maintain a good security level of the components of the ML application Existence of several vulnerabilities because ML specificities are not integrated in existing policies Contract with a low security third party Application not compliant with applicable regulations Poor access rights management Human error Lack of documentation on the ML application Denial of service due to inconsistent Use of uncontrolled data data or a sponge example Cybersecurity incident not reported to Lack of cybersecurity awareness incident response teams 17 SECURING MACHINE LEARNING ALGORITHMS December 2021

4. SECURITY CONTROLS

4.1 SECURITY CONTROLS RESULTS Having identified a set of threats that can target vulnerabilities in applications which use ML algorithms, it is possible to identify which security controls can be put in place to mitigate them. To do this, we commenced with the vulnerabilities identified in the previous Chapter and came up with a list of 37 security controls that were then mapped to the taxonomy. Table 5 summarises security controls for ML algorithms and lists: • Security controls definitions. • At which stage of the lifecycle the security controls can be applied. For ease of reading, they were divided into three categories: • “Organisational and Policy” are more traditional security controls, either organisational or linked to security policies. • “Technical” are more classic technical security controls. • “Specific to ML” are security controls that are specific to applications using ML. In Annex 5.C, a set of operational implementation examples are listed for each of the security controls. This includes: • For security controls not specific to ML algorithms: examples from the ISO 27001/2 family of standards or NIST 800-53 framework that should be considered when implementing the security control. • For security controls specific to ML: examples of techniques found in the current literature. All sources are referenced and may be found in Annex 5.D. The overall mapping of threats, vulnerabilities and security controls is available in Annex 5.B. 18 SECURING MACHINE LEARNING ALGORITHMS December 2021 Table 5: Security controls Stages of the lifecycle Security controls Definition ti c n ti in n n a y ORGANISATIONAL Define access rights management using a RBAC (Role Based Access Control) model respecting the least privileged principle. This should cover all components of the ML model (e.g. host infrastructures) and allow Apply a RBAC model, for the protection of resources such as the respecting the least privileged model (e.g. its configuration, its code) and x x x x x x x x x x principle the data it used (e.g. training data). It is notable that the roles to be included also concern the end user. For example: the end user who can submit inputs to the model should not be able to have access to its configuration. As for all projects, documentation must be produced for AI to preserve knowledge on the choices made during the project phase, the application architecture, its configuration, its maintenance, how to maintain its Apply documentation effectiveness over time and the assumptions x x x x x x x x x x requirements to AI projects made about the model use. This documentation should also include the changes that will be applied, including to the documentation throughout the algorithm's life cycle. As all applications, those using ML can be subject to regulations and laws (e.g., depending on collected data). Such Assess the regulations and assessment must be done as soon as laws the ML application must possible during the project phase, and x x x x x x x x x x comply with should be regularly updated thereafter as regulations are rapidly evolving (e.g., an AI Act has been proposed at the European level). As all applications, those using ML must comply with data security requirements to ensure the overall lifecycle of the data they Ensure ML applications use will be secured (e.g. description of data comply with data security x x x x x x x x x x lifecycle and associated controls, data requirements classification, protection of data at rest and in transit, use of appropriate cryptographic means, data quality controls). As all applications, those using ML must comply with defined policies regarding identity management (e.g. ensure all users are integrated in the departure process), Ensure ML applications authentication (e.g. passwords complexity, comply with identity use of Multi-Factors Authentication (MFA), x x x x x x x x x x management, authentication, access restriction) and access control (e.g. and access control policies RBAC model, connection context). Underlying security requirements must be applied to all ML application components (e.g. model configuration, host infrastructures, training data). 19 SECURING MACHINE LEARNING ALGORITHMS December 2021 Stages of the lifecycle Security controls Definition o g e a o g g ti my As all applications, those using ML must Ensure ML applications comply with protection policies (e.g. comply with protection hardening, anti-malware policy) and be x x x x x x x x x x policies and are integrated to integrated to security operations processes security operations processes (e.g. vulnerability management, backups). Ensure ML applications As all applications, those using ML must x x x x x x x x x x comply with security policies comply with existing security policies. As all applications, those using ML must be integrated in global processes for detection and incident response. This implies Include ML applications into collecting the appropriate logs, configuring detection and response to x x x x x x x x x X 15 relevant detection use cases to detect security incident processes attacks on the application, and giving the keys to incident response team for efficient response. As all applications, those using ML must be integrated to global processes for asset Include ML applications in management to ensure their assets are x x x x x x x x x X asset management processes inventoried, their owners are identified, their information classified. As any application, ML ones must be Integrate ML applications into integrated in the overall cyber-resilience the overall cyber-resilience strategy, to ensure their architecture and x x x x x x x x x x strategy operational processes (e.g. backups) take into account cybersecurity scenario. Specific ML security attention points should Integrate ML specificities to be integrated in existing security policies and x x x x x x x x x x existing security policies guidelines to ensure they are taken into consideration. TECHNICAL Some model designs are more commonly used or shared than others and, especially in the ML field; it can be included in their lifecycle to widely share them (e.g. open source sharing). These aspects must be considered in the global application risk Assess the exposure level of analysis. For example, two elements can be x x x x x x x x x x the model used distinguished: - Do not reuse models taken directly from the internet without checking them. - Use models for which the threats are clearly identified and for which security controls exist.

Please note that ML components with false positives might have adverse effect.

20 SECURING MACHINE LEARNING ALGORITHMS December 2021 Stages of the lifecycle Security controls Definition o g e a o g g ti my During the lifecycle of an ML algorithm, several components (such as software, programming libraries or even other models) are used to complete the project. Security Check the vulnerabilities of checks have to be carried out to ensure that the components used so that these components offer an adequate level of x x x x x x x x x x they have an appropriate security. Moreover, some mechanisms need security level to be used to prevent tampering with the components used. For example: if an open-source library is to be used, code reviews or check for public vulnerabilities on it can be done. A risk analysis of the overall application should be conducted to take into account the specificities of its context, including: - The attacker’s motivations - The sensitivity of the data handled (e.g. medical or personal and thus subject to regulatory constraints, strategic for the Conduct a risk analysis of the company and should thus be highly x x x x x x x x x x ML application protected) - The application hosting (e.g. through third parties services, cloud or on premise environments) - The model architecture (e.g. its exposition, learning methods) - The ML application lifecycle (e.g., model sharing Data must be checked to ensure they will suit the model and limit the ingestion of malicious data: - Evaluate the trust level of the sources to check it's appropriate in the context of the application Control all data used by the - Protect their integrity along the whole data x x x x x x x x x x ML model supply chain - Their format and consistence are verified - Their content is checked for anomalies, automatically or manually (e.g. selective human control) - In the case of labeled data, the issuer of the label is trusted. ML is a field in which the use of open-source elements is widespread (e.g., data for training, including labeled ones, models). The trust level of the different sources used Ensure reliable sources should be assessed to prevent using compromise ones. x x are used For example: the project wants to use labeled images from a public library. Are the contributors sufficiently trusted to have confidence in the contained images or the quality of their labelling? Removing suspicious samples from the Use methods to clean the training and testing dataset can help prevent training dataset from poisoning attacks. Some methods exist to x x x suspicious samples identify those that could cause strange behavior of the algorithm. 21 SECURING MACHINE LEARNING ALGORITHMS December 2021 Stages of the lifecycle Security controls Definition o g e a o g g ti my Define dashboards of key indicators integrating security indicators (peaks of Define and monitor indicators change in model behavior etc.) to follow-up for proper functioning of the x the proper functioning of the model model regarding the business case, in particular to allow rapid identification of anomalies. Test environments must also be secured according to the sensitivity of the information Ensure appropriate protection they contain. Special care must be paid to is deployed for test the data used in these environments, to x x x x x x x x x x environments ensure their protection (e.g., same protection measures as for production if not desensitiser). As all applications, those using ML must Ensure ML applications comply with third parties’ security comply with third parties’ x x x x x x x x x x requirements if their context involves security requirements suppliers. As any project, ML projects must comply to process for integrating security into projects, including the followings: - Risk analysis on the whole application - Check of the integration of cybersecurity best practices regarding architecture, secure development. - Check that the application will be Ensure ML projects follow the integrated in existing operational security global process for integrating processes: monitoring and response, patch x x x x x x x x x x security into projects management, access management, cyberresilience. - Check of the production of adequate documentation to ensure the sustainability of the application (e.g., technical architecture, hardening, exploitation, configuration and installation documents) - Security checks before going to production (e.g. security audit, pen tests) SPECIFIC ML Include adversarial examples to the Add some adversarial algorithm's training to enable it to be more examples to the training resilient to such attacks. Depending on the x x x dataset application domain and ambient conditions, such training could be done continuously. Adding a step to modify the model's inputs (e.g. data randomisation which consists in adding random noise to each piece of data), can improve the robustness of the model to Apply modifications on attacks. Such steps can make it more 17 x x inputs difficult for an attacker to understand the functioning of the algorithm and thus to manipulate it and reduce the impacts of an attack. This security control can be applied during training or model deployment stages.

This security control is often referred to as “Robust adversarial training” in the literature. One important thing to keep in mind is that such modifications should not overly impact model performance on benign inputs.

22 SECURING MACHINE LEARNING ALGORITHMS December 2021 Stages of the lifecycle Security controls Definition o g e a o g g ti my The ML models should be explainable, even if it means simplifying them, to enable a good understanding of their functioning and decision factors. It can also be a regulatory requirement (e.g. GDPR). However, once again, security Build explainable models x x interferes with the explainability property of the model (easier-to-understand decisions can be easier-to-build adversarial examples). It is therefore a trade-off between the need for explainability and security. Some model designs can be more robust than others against attacks. For instance, ensemble methods like bagging can mitigate Choose and define a more the impact of poisoning (during the training x resilient model design phase). Another example is defensive distillation, which may allow deep neural networks to better deal with evasion attacks. Using a set of training data expansion techniques (e.g. data augmentation) addresses the lack of data and improves the robustness of the model to poisoning attacks by diluting their impact. It is notable, Enlarge the training dataset however, that this security control more x x specifically addresses poisoning attacks that aim to reduce the performance of the model than those that seek to establish a backdoor. Moreover, one needs to ensure the reliability of the sources used to augment the dataset. The introduction of bias in ML algorithms will not be detailed because it is not the topic of the publication. Ensure that models are However, some techniques can be used to x x x x x x unbiased mitigate bias: verify the training dataset is representative enough regarding the business case, check the relevance of the attributes used to make decisions etc. Differential privacy (DP) is a strong, mathematical definition of privacy in the context of statistical and ML analysis. According to this mathematical definition, DP is a criterion of privacy protection, which many tools for analysing sensitive personal Ensure that models respect information have been devised to satisfy. It differential privacy to a is noticeable that this security control can x x x x x x x x sufficient degree greatly reduce the performance of the model. It is therefore important to estimate the need for data or model protection. Example: Differential privacy makes it possible for technology companies to collect and share aggregate information about user habits, while maintaining the privacy of individual users. Ensure that the model is sufficiently resilient Ensure that the model is against the environment in which it will sufficiently resilient to the operate. This includes, for instance, ensure x x x x x x x x x X environment in which it will that learning process and data are operate. representative enough of the real conditions in which the model will evolve. 23 SECURING MACHINE LEARNING ALGORITHMS December 2021 Stages of the lifecycle Security controls Definition o g e a o g g ti my ML is a rapidly evolving field, especially regarding its cybersecurity. Regular Implement processes to checking of new attacks and defenses must maintain security levels of ML be integrated into the processes for x x x x x x x x x X components over time maintaining security level applications. The security level should thus be regularly assessed too. Input-based detection tools can be of interest to identify whether a given input has Implement tools to detect if a been modified by an attacker or not. data point is an adversarial One example, in the case of Deep Neural x x x example or not Networks (DNNs), is to add a neural subnetwork to an architecture trained to detect adversarial examples. ML considerations should be added to awareness programs for concerned stakeholders and they must all receive cybersecurity awareness training: Integrate ML specificities to - Global cybersecurity awareness training awareness strategy and including best practices to prevent attackers x x x x x x x x x x ensure all ML stakeholders compromising the ML application. are receiving it - Manipulation of potentially sensitive data or data subject to regulatory restrictions. - Configurations to prevent applications being vulnerable - ML-specific attack awareness Before moving the model to production and then on a regular basis, the model should be evaluated to ensure it has not been poisoned. This differs from the security control “Use methods to clean the training Integrate poisoning control dataset from suspicious samples”. Indeed, after the "model evaluation" x here, it’s the model itself that is evaluated. phase For example: deep learning classification algorithms can be checked for poisoning using the STRIP technique. The principle is to disturb the inputs and observe the randomness of the predictions. This defense consists of limiting the information about the model when it is not necessary. More precisely, it aims at taking the necessary actions in order to reduce the information available on the model such as information on the training data set or any other information that could be used by an Reduce the available attacker (e.g., not publishing the model in x x x x x x x x x x information about the model open source). Of course, there is a trade-off between security and the fact that stakeholders (e.g., users, ML teams) sometimes want open source models. However, it remains notable that in many cases, research has shown that minimal information is sufficient to mount attacks.

See https://arxiv.org/pdf/1902.06531.pdf. It is notable that STRIP (STRong Intentional Perturbatio) may have a huge runtime overhead and may be infeasible for large dataset.

24 SECURING MACHINE LEARNING ALGORITHMS December 2021 Stages of the lifecycle Security controls Definition o g e a o g g ti my Controlling the information (like its verbosity) provided by the model by applying basic cybersecurity hygiene rules is a way of limiting the techniques that an attacker can use to build adversarial examples. One of the basic rules of hygiene, for example, is to reduce the information of the output determined by the model to the Reduce the information given 19 maximum, or by profile making the request. x by the model For example: considering a classification application, it would consist of communicating only the predicted class to the users of solution, not the associated probability. However, it remains notable that in many cases, research has shown that minimal information is sufficient to mount attacks. Federated learning is a set of training techniques that trains a model on several decentraliser servers containing local data Use federated learning to samples, without exchanging their data x x minimize risk of data breaches samples. This avoids the need to transfer the data and/or entrust it to an untrusted third party and thus helps to preserve the privacy of the data. The transferability property can be used to force adversarial examples from a substitution model to evade another. The ease of transferring an adversarial example Use less easily transferable 20 from a model to another depends on the x models family of algorithms. One possible defense is thus to choose an algorithm family that is less sensitive to the transferability of adversarial examples.

It is important to keep in mind that, in case of attacks like evasion or oracle, this security control can help. However, in some cases, it may possible to bypass the security control by using more queries. Some evasion attacks are based on the following principle: train a model with data like the target model used and generate adversarial examples from this model. Then, present these adversarial examples to the target model to perform an evasion attack. Whether or not to transfer an adversarial example generated by one model to another depends on their respective design as shown in the reference 215.

25 SECURING MACHINE LEARNING ALGORITHMS December 2021

5. CONCLUSION

Machine Learning algorithms are at the core of modern AI systems and applications. However, There is no silver they are faced with a series of threats and vulnerabilities. In this report we have identified bullet for ML-multiple security controls that can be applied to ML applications to address the threats they specific attacks, face. Some of the security controls are specific to ML algorithms, but others are standard technical and organisational cybersecurity controls to mitigate general attacks. It is important to but mitigation apply both types of controls because AI systems, in addition to ML specific vulnerabilities, there measures can exist also general type of vulnerabilities, which may also be exploited by adversaries. still raise the bar for attackers. Mitigation controls for ML-specific attacks outlined in the report should in general be deployed Thus, more during the entire lifecycle of the ML system. This includes measures for assuring the data quality and protecting its integrity, making the ML algorithms more robust and controlling access attention should to both the model and the data to ensure their privacy. The report also emphasizes the need for be given to the explainability of decisions, and the importance of detecting bias that can be present or security controls injected in a model by an attacker, which can then lead to unethical uses of AI. to enable comparability An important point highlighted in the report is that the identified security measures can be applied to all algorithms. Nevertheless, their operational implementations (see Annex C) may be and increase specific to certain types of algorithms. For example, for the security control “Choose and define resilience. a more resilient model design”, the defensive distillation implementation is specific to neural networks. It is also notable that with the prevalence of research papers on supervised learning, there are more examples of operational implementations for this type of algorithms. This report addresses an emerging subject. Thus, it remains very important to keep an active watch on threats and security controls in the field of ML in order to understand the latest innovations both from a technical point of view, or with a view to comply with standards provided by ISO, IEEE and ETSI . When looking ahead and given the complexity of the issue of securing ML, companies and governments have new responsibilities. For instance, it is increasingly important to raise cybersecurity awareness within companies, especially regarding the security of ML systems. For some populations, particularly data science teams, cybersecurity has not been at the forefront for many years. Moreover, by including data science actors in these actions, they are also given the opportunity to think of innovative solutions to mitigate the various threats. Thus, to this end, training and education programs should be organised regularly and the vulnerabilities of ML should be demonstrated using concrete examples. Finally, the context in which security controls are applied is crucial and specific use cases should be considered when conducting targeted risk assessments. All mitigations used should be proportional to the application-specific threat level and consider specific conditions of the environment that may either favor or hamper attacks. Moreover, defenders should be aware of the following points: 1) There is no silver bullet for mitigating ML-specific attacks. Some security controls may be bypassed by adaptive attackers. However, applied mitigations can still raise the bar for attackers. 26 SECURING MACHINE LEARNING ALGORITHMS December 2021 2) ML-specific mitigation controls are not generally evaluated in a standardised way even if it is a current and important issue to enable comparability. More research should be devoted to standardised benchmarks for comparing ML-specific mitigations on a level playing field. These benchmarks should also be enforced to ensure that the methods used in practice are the ones that perform best. 3) Deploying security controls often leads to a trade-off between security and performance and this is a topic of particular importance that should be further pursued by the research and cybersecurity communities. 27 SECURING MACHINE LEARNING ALGORITHMS December 2021

A ANNEX: TAXONOMY OF ALGORITHMS

Algorithm Main Data Data Learning Accuracy Definition Explainability Refs Name domain type environments Paradigm Provided

AdaBoost uses multiple iterations to generate a single composite strong learner by iteratively adding weak learners. During each Classic phase of training, a new Structured Supervised Classification, Globally AdaBoost Data 38 weak learner is added to the data learning Regression Explainable Science ensemble, and a weighting vector is adjusted to focus on examples that were misclassified in previous rounds.

Adam optimisation is an extension to Stochastic gradient decent and can be used in place of classical Classic Adam stochastic gradient descent Structured Data / Optimisation 24 optimisation to update network weights data Science more efficiently, thanks to two methods: adaptative learning rate and momentum

Agglomerative clustering is a "bottom-up" approach of hierarchical clustering. Each Classic Agglomerative Structured Unsupervised observation starts in its own Data Clustering 32 clustering data Learning cluster, and pairs of clusters Science are merged as one moves up the hierarchy.

Given a time series Xt, the ARMA/ARIMA model is a tool to understand and predict the future values of Classic ARMA/ARIMA Supervised this series. The model is Data Time series Regression Fully Explainable 136 model learning composed of two parts: an Science autoregressive part (AR) and a moving average part (MA)

Bidirectional Encoder Representations from Transformers (BERT) is a NLP & Transformer-based ML Supervised BERT Speech Text Classification Not Explainable Yes 5 technique for natural learning processing language processing (NLP) pre-training developed by Google.

16, 22, A Convolutional Neural 36, 43, Network is a deep learning 49, 50, algorithm which can take in 56, 58, Computer Convolutional an input, assign importance Image, 59; 64, Vision, NLP Supervised Neural (learnable weights and video, text, Classification Not Explainable Yes 67, 68, & Speech learning Network biases) to various time series 69, 70, processing aspects/objects in the data 82, 89, and be able to differentiate 103, one from the other. 124, 161

28 SECURING MACHINE LEARNING ALGORITHMS December 2021 Algorithm Main Data Data Learning Accuracy Definition Explainability Refs Name domain type environments Paradigm Provided

DBSCAN - Density-Based Spatial Clustering of Applications with Noise is a density-based clustering non-parametric algorithm: given a set of points in some space, it groups 26, Computer Unsupervised DBSCAN together points that are Image Clustering 129, Vision Learning closely packed together 142 (points with many nearby neighbours), marking as outliers points that lie alone in low-density regions (whose nearest neighbours are too far away).

A decision tree is a graph that uses a branching Classic method to illustrate every Structured Supervised Classification, 40, 42, Decision tree Data Fully Explainable possible output for a specific data learning Regression 120, Science input in order to break down complex problems.

Deep Q-learning works as Q-learning algorithm at the difference that it uses a Classic Deep Q- neural network to Reinforcement Data Time series Rewarding Yes 65, 85 learning approximate the Q-value learning Science function to manage big amount of states and actions.

EfficientNet is a Convolutional Neural Network based on depth wise convolutions, which Computer Supervised EfficientNet makes it lighter than other Image Classification Not Explainable Yes 4 Vision learning CNNs. It also allows to scale the model with a unique lever: the compound coefficient.

The factorial correspondence analysis (CFA) is a statistical method of data analysis which allows the analysis and

Factor

prioritisation of the Classic analysis of Structured Unsupervised Dimension information contained in a Data corresponde data Learning Reduction rectangular table of data Science nces and which is today particularly used to study the link between two variables (qualitative or categorical).

A GAN is a generative model where two networks are placed in competition. The first model is the generator, it generates a sample (e.g. an image), Computer Image, Unsupervised GAN while its opponent, the 53, 135 Vision Video Learning discriminator, tries to detect whether a sample is real or whether it is the result of the generator. Both improve on the performance of the other.

29 SECURING MACHINE LEARNING ALGORITHMS December 2021 Algorithm Main Data Data Learning Accuracy Definition Explainability Refs Name domain type environments Paradigm Provided

A Gaussian mixture model

is a probabilistic model that Computer Text, time assumes all the data points Vision, NLP series, Unsupervised GMM are generated from a Clustering 31, 131 & Speech Image, Learning mixture of a finite number of processing video, Gaussian distributions with

unknown parameters.

Generative Pre-trained

Transformer 3 (GPT-3) is an NLP & autoregressive language Supervised GPT-3 Speech Text Classification Not Explainable Yes 6 model that uses deep learning processing learning to produce human-

like text.

Gradient boosting is a Gradient technique that optimises a Classic 3, 51, Structured Supervised Classification, Globally boosting decision tree by combining Data 54, 55, data learning Regression Explainable machine weak models to improve Science 140

model prediction.

Gradient descent is a first-

order iterative optimisation

algorithm for finding a local

minimum of a differentiable

function. The idea is to take Classic Gradient repeated steps in the Structured Data / Optimisation 17 descent opposite direction of the data Science gradient (or approximate

gradient) of the function at

the current point, because

this is the direction of

steepest descent.

Graph neural networks

(GNNs) are deep learning-

based methods that operate Computer

Graph neural

on graph domain. Graphs Vision, Supervised Regression, networks Image 20 are a kind of data structure Speech learning classification

(GNNs)

which models a set of processing

objects (nodes) and their

relationships (edges)

Hierarchical clustering is a

method of cluster analysis

which seeks to build a Classic Hierarchical hierarchy of clusters. The Structured Unsupervised Data Clustering 32 clustering result is a tree-based data Learning Science representation of the

objects, named a

dendrogram.

Hidden Markov Model is a

statistical Markov model in Structured Hidden Structured which the system being data, NLP & Reinforcement Markov data, time Rewarding Yes 29 modelled is assumed to be Speech learning Model (HMM) series, text a Markov process with processing

unobservable hidden states.

ICA is a special case of

blind source separation. A

Independent common example Classic Structured Unsupervised Dimension component application is the "cocktail Data 2 data Learning Reduction analysis party problem" of listening in Science

on one person's speech in a

noisy room.

30 SECURING MACHINE LEARNING ALGORITHMS December 2021 Algorithm Main Data Data Learning Accuracy Definition Explainability Refs Name domain type environments Paradigm Provided

The isolation forest returns the anomaly score of each sample. It isolates observations by randomly Classic Isolation selecting a feature, and then Structured Unsupervised Anomaly 157, Data forest randomly selecting a split data learning detection 161 Science value between the maximum and minimum values of the selected feature.

K-means clustering is a method of vector quantification that aims to partition n observations into Classic k clusters in which each Structured Unsupervised K-means Data Clustering 129 observation belongs to the data Learning Science cluster with the nearest mean (cluster centres or cluster centroid), serving as a prototype of the cluster.

K-Nearest Neighbour is a simple algorithm that stores all the available cases and classifies the new data or Classic K-Nearest Structured Supervised case based on a similarity Data Classification Fully Explainable Yes 21, 40, Neighbour data learning measure. It is mostly used Science to classify a data point based on how its neighbours are classified.

Linear regression attempts to model the relationship between two or more variables by fitting a linear equation to observed data. One variable is considered to be an explanatory Classic Linear Structured Supervised 2, 117, variable, and the other is Data Regression Fully Explainable regression data learning 221 considered to be a Science dependent variable. For example, a modeller might want to relate the weights of individuals to their heights using a linear regression model.

Logistic regression is used to classify data by modelling the probability of a certain class or event existing such as pass/fail, win/lose, Classic Logistic Structured Supervised 2, 120, alive/dead or healthy/sick. Data Classification Fully Explainable Yes regression data learning 177 This can be extended to Science model several classes of events such as determining whether an image contains a cat, dog, lion, etc.

Long short-term memory (LSTM) is an artificial recurrent neural network (RNN) architecture used in 22, 44, the field of deep learning. NLP & 45, 46, Unlike standard feedforward Speech Text, 47, 50, Supervised LSTM neural networks, LSTM has processing, image, Regression Not Explainable 84, 85, learning feedback connections. It computer video 131, cannot only process single vision 158, data points (such as 161 images), but also entire sequences of data (such as speech or video).

31 SECURING MACHINE LEARNING ALGORITHMS December 2021 Algorithm Main Data Data Learning Accuracy Definition Explainability Refs Name domain type environments Paradigm Provided

Mean shift is a non-

parametric feature-space

analysis technique for Computer Image, Unsupervised Mean shift Clustering 27 locating the maxima of a Vision video Learning

density function, a so-called

mode-seeking algorithm

Computer MobileNets are based on a Vision, streamlined architecture that Image, Classic uses depth-wise separable video, text, Data Unsupervised MobileNet convolutions instead of time series, Clustering Yes 4 Science, learning convolutions, in order to structured NLP & build light wFeight deep data Speech neural networks. processing

A Monte Carlo algorithm is a

randomised algorithm Classic Monte Carlo Structured Reinforcement whose output may be Data Rewarding 30, 105 algorithm data learning incorrect with a certain Science

(typically small) probability.

A Multimodal Parallel

Network helps to manage Computer

Multimodal

audio-visual event Vision, Supervised Parallel Video Classification 18 localisation by processing Speech learning Network both audio and visual processing

signals at the same time.

Naive Bayes classifiers are

a family of simple 39, 40, probabilistic classifiers Classic Naive Bayes Structured Supervised 89, based on applying Bayes' Data Classification Fully Explainable Yes classifiers data learning 120, theorem with strong (naïve) Science 210 independence assumptions

between the features.

A family of policy gradient

methods for Reinforcement

Proximal Learning that alternate Classic Structured Reinforcement Policy between sampling data and Data data, time Rewarding Yes 137 learning Optimisation optimising a surrogate Science series

objective function using

stochastic gradient ascent.

The main idea of principal

component analysis (PCA)

is to reduce the

dimensionality of a data set Principal Classic consisting of many variables Structured Unsupervised Dimension Component Data 2 correlated with each other, data Learning Reduction Analysis Science either heavily or lightly,

while retaining the variation

present in the dataset, up to

the maximum extent.

Q-learning is a model-free

reinforcement learning Classic Structured algorithm to learn the value Reinforcement Q-learning Data data, time Rewarding Yes 28 of an action in a particular learning Science series state. It does not require a

model of the environment.

Random forests are an

ensemble learning method

that operates by

constructing a multitude of

decision trees at training Classic 51, Random Structured Supervised Classification, Globally time and outputting the Data 136, forests data learning Regression Explainable class that is the mode of the Science 140

classes (classification) or

mean/average prediction

(regression) of the individual

trees.

32 SECURING MACHINE LEARNING ALGORITHMS December 2021 Algorithm Main Data Data Learning Accuracy Definition Explainability Refs Name domain type environments Paradigm Provided

A recurrent neural network

(RNN) is a class of artificial 14, 17,

neural networks where Computer Time 44, 45,

Recurrent

connections between nodes Vision, NLP series, text, Supervised 46, 47, neural Regression Not Explainable form a directed graph along & Speech image, learning 49, 50,

network

a temporal sequence. This processing video 52, 89,

allows it to exhibit temporal 13

dynamic behaviour.

A residual neural network

(ResNet) is an artificial

neural network (ANN) that

builds on constructs known Computer Supervised ResNet from pyramidal cells in the Image Classification Not Explainable Yes 4, 7, 37 Vision learning cerebral cortex by utilising

skip connections, or

shortcuts to jump over some

layers.

Spatial Spatial Temporal Graph

Convolutional Networks is a

Temporal

convolutional neural Computer Supervised Graph Video Classification 25 network that automatically Vision learning

Convolutiona

learns both the spatial and l Networks temporal patterns from data.

Stochastic gradient descent

is an iterative method for

optimising an objective

function with suitable

smoothness properties. It

can be regarded as a Stochastic stochastic approximation of Classic Structured gradient gradient descent Data / Optimisation 17, 24 data descent optimisation, since it Science

replaces the actual gradient

(calculated from the entire

data set) by an estimate

thereof (calculated from a

randomly selected subset of

the data).

42, 47,

51, 67, SVM are linear classifiers 69, 87, which are based on the 89, 92, margin maximisation 98, principle. They accomplish Support Classic 106, the classification task by Structured Supervised vector Data Classification Fully Explainable Yes 120, constructing, in a higher data learning machine Science 136, dimensional space, the 139, hyperplane that optimally 142, separates data into two 152, categories. 177,

185

Wavenet is a deep neural

network for generating raw

audio waveforms. The

model is fully probabilistic NLP & 44, Unsupervised WaveNet and autoregressive, with the Speech Time series NLP task 45,131, learning predictive distribution for processing 132

each audio sample

conditioned on all previous

ones

XGBoost is an extension to

gradient boosted decision

trees (GBM) and specially Classic Structured Supervised Classification, Globally XGBoost designed to improve speed Data 3 data learning Regression Explainable and performance by using Science

regularisation methods to

fight overfitting.

33 SECURING MACHINE LEARNING ALGORITHMS December 2021

B ANNEX: MAPPING SECURITY CONTROLS TO THREATS

Threats | sub- Threats Vulnerabilities Security Controls threats references Implement tools to detect if a data point is an adversarial example or not Lack of detection of abnormal inputs Include ML applications in detection and response to security incident processes 13, 34, 37, 48, Poor consideration of evasion attacks Choose and define a more resilient model 49, 51, 53, 56, 59, 60, 62, 65, in the model design implementation design 66, 67, 73, 80, Lack of consideration of attacks to Integrate ML specificities to awareness 81, 82, 83, 84, which ML applications could be strategy and ensure all ML stakeholders are 90, 95, 97, exposed receiving it 100, 107, 109, 110, 121, 125, Evasion Lack of training based on adversarial Add some adversarial examples to the training 139, 144, 154, attacks dataset 155, 162, 163, 169, 170, 175, Lack of security process to maintain a good 181, 183, 185, security level of the components of the ML 199, 200, 201, Use a widely known model allowing application 202, 204, 205, the attacker to study it 206, 207, 209, Use less easily transferable models 211, 213, 215 Assess the exposure level of the model used Inputs totally controlled by the attacker Apply modifications to inputs which allows for input-output-pairs Use of Too much information available on the Reduce the available information about the 34, 35, 48, 51, adversarial model model 56, 59, 60, 62, examples 65, 80, 81, 82, crafted in white 100, 109, 110, or grey box Too much information about the model 125, 139, 144, Reduce the information given about the model conditions (e.g. given in its outputs 154, 170, 204, FGSM…) 209 Apply a RBAC model, respecting the least Poor access rights management privileged principle The model allows private information Ensure that models respect differential privacy to be retrieved Too much information about the model Reduce the information given about the model given in its outputs Too much information available on the Reduce the available information about the model model 121, 145, 146, 152, 170, 177, Oracle Lack of consideration of attacks to Integrate ML specificities to awareness 194, 203, 204, which ML applications could be strategy and ensure all ML stakeholders are 208, 214 exposed to receiving it Lack of security process to maintain a Implement processes to maintain security good security level of the components levels of ML components over time of the ML application Weak access protection mechanisms Ensure ML applications comply with identity for ML model components management, authentication, and process control policies 34 SECURING MACHINE LEARNING ALGORITHMS December 2021 Threats | sub- Threats Vulnerabilities Security Controls threats references Choose and define a more resilient model design Model easy to poison Implement processes to maintain security levels of ML components over time Assess the exposure level of the model used Lack of data for increasing robustness Enlarge the training dataset to poisoning Apply a RBAC model, respecting the least Poor access rights management privileged principle Ensure ML applications comply with data Poor data management security requirements 74, 77, 79, 99, 114, 115, 116, Undefined indicators of proper 117, 118, 121, Define and monitor indicators for proper functioning, making complex 126, 140, 142, functioning of the model Poisoning compromise identification 143, 162, 167, 170, 171, 172, Lack of consideration of attacks to Integrate ML specificities to awareness 173, 189, 196, which ML applications could be strategy and ensure all ML stakeholders are 197, 198, 199, exposed to receiving it 204, 210 Use of uncontrolled data Control all data used by the ML model Use of unsafe data or models (e.g with Ensure reliable sources are used transfer learning) Integrate poisoning control after the "model Lack of control for poisoning evaluation" phase No detection of poisoned samples in Use methods to clean the training dataset from the training dataset suspicious samples Ensure ML applications comply with identity Weak access protection mechanisms management, authentication, and access for ML model components control policies Label Use of unreliable source to label data Ensure reliable sources are used 125, 140, 204 modification Apply a RBAC model, respecting the least Poor access rights management privileged principle Existence of unidentified disclosure Conduct a risk analysis of the ML application scenarios Ensure ML applications comply with identity Weak access protection mechanisms Model or data management, authentication, and access for ML model components disclosure control policies Lack of security process to maintain a Implement processes to maintain security good security level of the components levels of ML components over time of the ML application Unprotected sensitive data on test Ensure appropriate protection are deployed for environments test environments as well 121, 194, 221, Integrate ML specificities to awareness Too much information about the model 222 strategy and ensure all ML stakeholders are given in its outputs receiving it The model can allow private Data Ensure that models respect differential privacy information to be retrieved disclosure The model can allow private Reduce the information given by the model information to be retrieved Disclosure of sensitive data for ML Use federated learning to minimise the risk of algorithm training data breaches Too much information available on the Reduce the available information about the Model model model disclosure Too much information about the model Reduce the information given by the model given in its outputs 35 SECURING MACHINE LEARNING ALGORITHMS December 2021 Threats | sub- Threats Vulnerabilities Security Controls threats references Apply a RBAC model, respecting the least Poor access rights management privileged principle Too much information available on the Reduce the available information about the model model Existence of several vulnerabilities because the ML application was not Ensure ML projects follow the global process integrated into process for integrating for integrating security into projects security into projects Check the vulnerabilities of the components Use of vulnerable components (among used so that they have an appropriate security the whole supply chain) level Too much information about the model Reduce the information given by the model given in its outputs Existence of unidentified compromise Conduct a risk analysis of the ML application scenarios Undefined indicators of proper Define and monitor indicators for proper functioning, making complex functioning of the model compromise identification Compromise of ML Integrate ML specificities to awareness 121, 164, 183, application Bad practices due to a lack of strategy and ensure all ML stakeholders are 189 components cybersecurity awareness receiving it Implement processes to maintain security Lack of security process to maintain a levels of ML components over time good security level of the components Ensure ML applications comply with protection of the ML application policies and are integrated to security operations processes Ensure ML applications comply with identity Weak access protection mechanisms management, authentication, and access for ML model components control policies Existence of several vulnerabilities Integrate ML specificities to existing security because ML specificities are not policies integrated to existing policies Ensure ML applications comply with security Existence of several vulnerabilities policies because ML application do not comply with security policies Include ML applications into asset management processes Ensure ML applications comply with third Contract with a low security third party parties’ security requirements Existing biases in the ML model or in Ensure that models are unbiased the data Lack of consideration of real-life Ensure that the model is sufficiently resilient to conditions in training the model the environment in which it will operate. ML application not integrated in the Integrate ML applications into the overall cyber-resilience strategy cyber-resilience strategy Existence of unidentified failure Failure or Conduct a risk analysis of the ML application scenarios 121, 164, 183, malfunction of ML 189, 191 application Undefined indicators of proper Define and monitor indicators for proper functioning, making complex functioning of the model malfunction identification Lack of explainability and traceability Build explainable models of decisions taken Lack of security process to maintain a Implement processes to maintain security good security level of the components levels of ML components over time of the ML application 36 SECURING MACHINE LEARNING ALGORITHMS December 2021 Threats | sub- Threats Vulnerabilities Security Controls threats references Existence of several vulnerabilities Ensure ML projects follow the global process because ML specificities are not for integrating security into projects integrated to existing policies Ensure ML applications comply with third Contract with a low security third party parties’ security requirements Application not compliant with Assess the regulations and laws the ML applicable regulations application must comply with Apply a RBAC model, respecting the least Poor access rights management privilege principle Apply documentation requirements to AI Human error Lack of documentation on the ML projects application Include ML applications into asset management processes Denial of service due to inconsistent Use of uncontrolled data Control all data used by the ML model data or a sponge example Cybersecurity incident not Integrate ML specificities to awareness reported to Lack of cybersecurity awareness strategy and ensure all ML stakeholders are incident receiving it response teams 37 SECURING MACHINE LEARNING ALGORITHMS December 2021

C ANNEX: IMPLEMENTING SECURITY CONTROLS

Security controls Examples for operational implementation References The literature provides the following techniques: - Adversarial Training Add some adversarial - Ensemble Adversarial Training 13, 23, 48, 51, 59, 65, 72, 95, 108, examples to the training - Cascade Adversarial Training 162, 200, 201, 202, 211, 215 dataset - Principled Adversarial Training - Gradient Band Based Adversarial Training The NIST 800-53 and the ISO 27001/2 provides several points: Apply a RBAC model, - Manage access permissions and authorisations, ISO 27001/2 respecting the least privilege incorporating the principles of least privilege and NIST 800-53, 162 principle separation of duties - Manage the identity of the users (Couple lifecycle management processes and procurement processes etc.) The NIST 800-53 and the ISO 27001/2 provides several 148 Apply documentation points: requirements to Artificial - Define change management processes, integrating the ISO 27001/2 Intelligence projects update of the documentation NIST 800-53 The literature provides the following techniques: - Data randomisation Apply modifications on inputs 64, 65, 108, 208 - Input transformation - Input denoising Assess the exposure level of the model used The NIST 800-53 and the ISO 27001/2 provides several Assess the regulations and points: ISO 27001/2 laws the ML application must - Identify applicable legislation NIST 800-53, 162 comply with - Meet the requirements of GDPR for personal data The literature provides the following techniques: - Interpret models with some tools Build explainable models - Use model more explainable like regression instead of 164, 225, 226, 227 Deep Neural Network for supervised learning when it is necessary 38 SECURING MACHINE LEARNING ALGORITHMS December 2021 Security controls Examples for operational implementation References The NIST 800-53 and the ISO 27001/2 provides several points: - Manage the exemptions by following it industrially, also including remediation plans - Make an inventory of the infrastructure equipment, the applications (Define, document, improve and review a Check the vulnerabilities of the regular process to make inventory) components used so that they ISO 27001/2 - Manage the maintenance, the obsolete assets etc. have an appropriate security NIST 800-53 (Define a process and continuously improve it, define a level roadmap to replace obsolescent technologies) - Implement a vulnerability management policy (Control regularly its implementation) - Perform and manage vulnerability scans on servers OS, middleware, database and network infrastructure (Perform regularly automatics scans) For poisoning, the literature provides the following technique: - Bagging or weight Bagging - TRIM algorithm For evasion, the literature provides the following Choose and define a more 35, 65, 108, 110, 112, 113, 114, technique: resilient model design 143, 196, 206, 213 - Randomisation - Stability terms into objective function - Adversarial perturbation-based regulariser - Input gradient regularisation - Defensive distillation - Random feature nullification The NIST 800-53 and the ISO 27001/2 provides several points: - Coordinate the compliance process with legal and audit functions Conduct a risk analysis of the - Identify legal requirements (e.g. GDPR or NIS for ISO 27001/2 ML application European countries) NIST 800-53 - Establish a methodology to manage identified risks - Establish a formal methodology to analyse risk - Define and monitor the IT resource availability (Formalise a capacity management plan) The literature provides the following techniques: - Data sanitisation Control all data used by the ML - RONI and tRONI technics 114, 116, 118, 142, 162, 197, 228 model - Point out important data and put a human in the loop (Human in the loop) The NIST 800-53 and the ISO 27001/2 provides several points: - Formalise a dashboard, bringing together a series of indicators enabling the state of the information system to Define and monitor indicators be judged in relation to the objectives set. ISO 27001/2 for proper functioning of the - Take actions in case of deviation from the objective NIST 800-53 model - Manage changes on assets, ensure changes will not impact the production and detect any changes in assets' baseline configuration - Guarantee the integrity of the code at all stage (Perform integrity control etc.) 39 SECURING MACHINE LEARNING ALGORITHMS December 2021 Security controls Examples for operational implementation References The literature provides the following technique: Enlarge the training dataset 68, 150 - Data augmentation The NIST 800-53 and the ISO 27001/2 provides several Ensure appropriate protection points: ISO 27001/2 are deployed for test - Protect data when there are in non-production NIST 800-53 environments as well environment (implement desensitisation measures etc.) Ensure that the model is sufficiently resilient to the environment in which it will operate. The NIST 800-53 and the ISO 27001/2 provides several points: - Apply a methodology for data classification. Review the classification regularly Ensure ML applications comply - Implement measures to detect data leakage on the ISO 27001/2 with Data Security Internet (Antivirus, Data right management solution on all NIST 800-53 requirements sensitive folders) - Secure sensitive data in transit (Deploy mechanisms to detect bypasses on all networks) - Deploy security solutions on network points to prevent data leaks (DLP etc.) The NIST 800-53 and the ISO 27001/2 provides several points: Ensure ML applications comply - Define a policy regarding users’ authentication (Define with identity management, an authentication policy that considers the sensitivity of ISO 27001/2 authentication and access resources and the connection context for all types of NIST 800-53, 162 control policies account, use Multi-Factor Authentication) - Define a remote access policy (Verify security configuration, authenticate connected devices) The NIST 800-53 and the ISO 27001/2 provides several points: - Manage the maintenance, the obsolete assets etc. Ensure ML applications comply (Define a process and continuously improve it, define a with protection policies and are roadmap to replace obsolescent technologies) ISO 27001/2 integrated to security - Implement a vulnerability management policy (Control NIST 800-53 operations processes regularly its implementation) - Perform and manage vulnerability scans on servers OS, middleware, database and network infrastructure (Perform regularly automatics scans) The NIST 800-53 and the ISO 27001/2 provides the Ensure ML applications comply ISO 27001/2 following point: with security policies NIST 800-53 - Define policies for information security The NIST 800-53 and the ISO 27001/2 provides several points: Ensure ML applications comply - Integrate the security into contracts (Define a security ISO 27001/2 with third parties’ security insurance plan for strategic contracts representing high NIST 800-53 requirements risks for company security. Communicate roles and responsibilities to every new third party) - Monitor and review third parties services 40 SECURING MACHINE LEARNING ALGORITHMS December 2021 Security controls Examples for operational implementation References The NIST 800-53 and the ISO 27001/2 provides several points: - Integrate the security into contracts (Define a security insurance plan for strategic contracts representing high risks for company security. Communicate roles and responsibilities to every new third party) Ensure ML projects follow the ISO 27001/2 global process for integrating - Define and manage a patch management policy NIST 800-53 security into projects - Manage the interconnections with external systems (establish a formal process to regularly review the exhaustiveness of interconnections inventory) - Integrate and manage security protection for applications (firewalls, WAF, reverse proxy) - Perform security controls on application The NIST 800-53 and the ISO 27001/2 provides several points: Ensure reliable sources are ISO 27001/2 - Manage the interconnections with external systems used NIST 800-53 (establish a formal process to regularly review the exhaustiveness of interconnections inventory) The literature provides the following techniques: - Classification parity - Calibration Ensure that models are - Anti-classification 217, 229 unbiased - Having a diverse dataset - Some other technics: samples bias, measurement error… The literature provides the following techniques: - Model design adapted like PATE for Deep Neural Ensure that models respect Network based classifier differential privacy to a 63, 108, 112, 194, 203, 208, 214 - Data randomisation sufficient degree - Randomisation - Objective function perturbation The NIST 800-53 and the ISO 27001/2 provides the Implement processes to following point: ISO 27001/2 maintain security levels of ML - Perform technical and organisational audits regularly on NIST 800-53 components over time critical scope and develop an action plan after each audit Implement tools to detect if a The literature provides the following technique: data point is an adversarial 65, 207 - Adding detector subnetworks example or not The NIST 800-53 and the ISO 27001/2 provides several points: - Create an inventory of the infrastructure equipment, the applications (Define, document, improve and review a Include ML applications into ISO 27001/2 regular process to make inventory) asset management processes NIST 800-53 - Classify information - Manage the maintenance, the obsolete assets etc. (Define a process and continuously improve it, define a roadmap to replace obsolescent technologies) 41 SECURING MACHINE LEARNING ALGORITHMS December 2021 Security controls Examples for operational implementation References The NIST 800-53 and the ISO 27001/2 provides several points: - Include ML projects into the Business Continuity Plan - Include ML projects into the Cybersecurity Disaster Include ML applications into Recovery Plan ISO 27001/2 detection and response to - Define a backup strategy for ML projects (and test it) NIST 800-53 security incident processes - Define a strategy for public relations during recovery (Identify and train possible spokespersons, Adapt communication responses to different categories of interlocutors) The NIST 800-53 and the ISO 27001/2 provides several points: - Include ML projects into the Business Continuity Plan - Include ML projects into the Cybersecurity Disaster Integrate ML applications into Recovery Plan ISO 27001/2 the overall cyber-resilience - Define a backup strategy for ML projects (and test it) NIST 800-53, 162 strategy - Define a strategy for public relations during recovery (Identify and train possible spokespersons, Adapt communication responses to different categories of interlocutors) The NIST 800-53 and the ISO 27001/2 provides the Integrate ML specificities to ISO 27001/2 following point: existing security policies NIST 800-53 - Review policies for information security Integrate ML specificities to The NIST 800-53 and the ISO 27001/2 provides several awareness strategy and ensure points: ISO 27001/2 all ML stakeholders are - Organise training sessions NIST 800-53 receiving it - Perform locally cyber risks reporting Integrate poisoning control The literature provides the following technique: after the "model evaluation" 198 - STRIP technique phase The NIST 800-53 and the ISO 27001/2 provides the Reduce the available ISO 27001/2 following point: information about the model NIST 800-53 - Implement a classification policy Reduce the information given The literature provides the following technique: 89, 145 by the model - Gradient Masking Use federated learning to 194 minimise risk of data breaches Use less easily transferable 65, 215 models The literature provides the following techniques: Use methods to clean the - Data sanitisation training dataset from suspicious - RONI and tRONI technics 114, 162, 210 samples - Point out important data and put a human in the loop (Human in the loop) 42 SECURING MACHINE LEARNING ALGORITHMS December 2021

D ANNEX: REFERENCES

d is m e ing e ing is ng rv Type of data rn rn rni orc a ingested e e Lea inf l uperv Lea nsupe d e nt S U e R e dat n o Title Source ti Author a ta n Index c s a o li e D ti ng ri ion a ng ion ion ub t e d s ri ge s ic e rdi P a s ns ideo e if e V Tex ture s duct wa Im gre s lust e m e im e Ti R la C D R R truc C S

Adversarial Machine https://www.morga • Yevgeniy Learning, Synthesis nclaypool.com/doi/ Vorobeychik 1 Lectures on Artificial abs/10.2200/S008 2018 X • Murat Intelligence and 61ED1V01Y20180 Kantarcioglu Machine Learning 6AIM039

https://web.stanfor • Trevor Hastie The Elements of 2 d.edu/~hastie/Pap 2001 • Robert Tibshirani X X X X X X X X X Statistical Learning ers/ESLII.pdf • Jerome Friedman

XGBoost: a scalable https://arxiv.org/pdf • Tianqi Chen 3 2016 X X tree boosting system /1603.02754.pdf • Carlos Guestrin

EfficientNet: Rethinking https://proceedings Model Scaling for • Mingxing Tan 4 .mlr.press/v97/tan1 2019 X X Convolutional Neural • Quoc V. Le 9a/tan19a.pdf Networks

BERT: Pre-training of • Jacob Devlin Deep Bidirectional • Ming-Wei Chang https://arxiv.org/pdf 5 Transformers for 2019 • Kenton Lee X X /1810.04805.pdf Language • Kristina

Understanding Toutanova

• Tom B. Brown

• Benjamin Mann • Nick Ryder

• Melanie Subbiah • Jared Kaplan

• Prafulla Dhariwal • Arvind

Neelakantan • Pranav Shyam

• Girish Sastry • Amanda Askell

• Sandhini Agarwal • Ariel Herbert-

Voss Language Models are https://arxiv.org/pdf • Gretchen Krueger 6 2020 X X Few-Shot Learners /2005.14165.pdf • Tom Henighan • Rewon Child

• Aditya Ramesh • Daniel M. Ziegler

• Jeffrey Wu • Clemens Winter

• Christopher

Hesse • Mark Chen • Eric Sigler

• Mateusz Litwin • Scott Gray

• Benjamin Chess • Jack Clark

• Christopher

43 SECURING MACHINE LEARNING ALGORITHMS December 2021 d is m e ing e ing is ng rv Type of data rn rn rni orc a ingested e e Lea inf l uperv Lea nsupe d e nt S U e R e dat n o Title Source ti Author a ta n Index c s a o li e D ti ng ri ion ng ion ion t d s a ri ub ge e s P a ic e ns rdi ideo s if e Tex e ture s duct wa V Im gre s lust e m e im e Ti R la C D R R truc C S

Berner • Sam McCandlish

• Alec Radford • Ilya Sutskeve

• Dario Amodei

• Preetum

Nakkiran Deep Double Descent: • Gal Kaplun https://arxiv.org/pdf 7 Where Bigger Models 2019 • Yamini Bansal X X /1912.02292.pdf and More Data Hurt • Tristan Yang

• Boaz Barak • Ilya Sutskever

• Kaiming He Deep Residual https://arxiv.org/pdf • Xiangyu Zhang 8 Learning for Image 2015 X X /1512.03385.pdf • Shaoqing Re Recognition • Jian Sun

• Kazuki Osawa

• Siddharth

Swaroop https://papers.nips. • Anirudh Jain Practical Deep cc/paper/2019/file/ • Runa 9 Learning with Bayesian b53477c2821c1bf0 2019 X X Eschenhagen Principles da5d40e57b870d3 • Richard E. Turner 5-Paper.pdf • Rio Yokota • Mohammad

Emtiyaz Khan

Deep Inside Convolutional • Karen Simonyan

Networks: Visualising https://arxiv.org/pdf • Andrea Vedaldi 10 2014 X X Image Classification /1312.6034.pdf • Andrew

Models and Saliency Zisserman Maps

https://web.stanfor d.edu/class/psych2 Reinforcement learning: • Richard S. Sutton 11 09/Readings/Sutto 1998 X X X X An introduction • Andrew G. Barto nBartoIPRLBook2n

dEd.pdf

• Yuri Burda Exploration by Random https://arxiv.org/pdf • Harrison Edwards 12 2018 X X Network Distillation /1810.12894.pdf • Amos Storkey

• Oleg Klimov

DeepFool: a simple • Seyed-Mohsen and accurate method to https://arxiv.org/pdf • Moosavi-Dezfooli 13 fool deep neural 2016 X /1511.04599.pdf • Alhussein Fawzi networks, in arXiv, July • Pascal Frossard 2016

• Yingce Xia

• Di He • Tao Qin Dual Learning for https://arxiv.org/pdf 14 2016 • Liwei Wang X X Machine Translation /1611.00179.pdf • Nenghai Yu

• Tie-Yan Liu • Wei-Ying Ma

44 SECURING MACHINE LEARNING ALGORITHMS December 2021 d is m e ing e ing is ng rv Type of data rn rn rni orc a ingested e e Lea inf l uperv Lea nsupe d e nt S U e R e dat n o Title Source ti Author a ta n Index c s a o li e D ti ng ri ion ng ion ion t d s a ri ub ge e s P a ic e ns rdi ideo s if e Tex e ture s duct wa V Im gre s lust e m e im e Ti R la C D R R truc C S

• Jingwei Xu

• Huazhe Xu Video Prediction via https://arxiv.org/pdf 15 2020 • Bingbing Ni X X Example Guidance /2007.01738.pdf • Xiaokang Yang

• Trevor Darrell

Context-aware • Ardhendu Behera

Attentional Pooling https://arxiv.org/pdf • Zachary Wharton 16 2021 X X (CAP) for Fine-grained /2101.06635.pdf • Pradeep Hewage

Visual Classification • Asish Bera

Ilya On the importance of http://proceedings. Sutskever/James initialisation and 17 mlr.press/v28/suts 2013 Martens /George X X X X X momentum in deep kever13.pdf Dahl/Geo↵rey learning Hinton

MPN: MULTIMODAL PARALLEL NETWORK • Jiashuo Yu https://arxiv.org/pdf 18 FOR AUDIO-VISUAL 2021 • Ying Cheng X X X /2104.02971.pdf EVENT • Rui Feng

LOCALISATION

Zero-Gradient Constrained • Kazuki https://arxiv.org/pdf 19 Optimisation for 2021 Naganuma X X /2104.02845.pdf Destriping of 3D • Shunsuke Ono

Imaging Data

• Chen Min

• Jiaolong Xu Attentional Graph https://arxiv.org/pdf • Liang Xiao 20 Neural Network for 2021 X X /2104.02576.pdf • Dawei Zhao Parking-slot Detection • Yiming Nie

• Bin Dai

Identity and Posture • Vandad Recognition in Smart https://arxiv.org/pdf 21 2019 Davoodnia X X Beds with Deep /2104.02159.pdf • Ali Etemad Multitask Learning

• Abdulaziz M.

A Combined CNN and Alayba https://arxiv.org/pdf 22 LSTM Model for Arabic 2018 • Vasile Palade X X /1807.02911.pdf Sentiment Analysis • Matthew England

• Rahat Iqbal

• Mathias Lechner

• Ramin Hasani Adversarial Training is https://arxiv.org/ab • Radu Grosu 23 Not Ready for Robot 2021 X X X s/2103.08187 • Daniela Rus Learning • Thomas A.

Henzinger

http://iwqos2018.ie

Improved Adam ee- 24 Optimizer for iwqos.org/files/201 2018 • Zijun Zhang X X X X X

Deep Neural Networks 8/05/Improved_Ad

am_Optimizer.pdf

https://ieeexplore.i Vision-Based Fall eee.org/stamp/sta • Oussema Keskes 25 Detection Using ST- 2021 X X mp.jsp?tp=&arnum • Rita Noumeir GCN ber=9351913

45 SECURING MACHINE LEARNING ALGORITHMS December 2021 d is m e ing e ing is ng rv Type of data rn rn rni orc a ingested e e Lea inf l uperv Lea nsupe d e nt S U e R e dat n o Title Source ti Author a ta n Index c s a o li e D ti ng ri ion ng ion ion t d s a ri ub ge e s P a ic e ns rdi ideo s if e Tex e ture s duct wa V Im gre s lust e m e im e Ti R la C D R R truc C S

DBSCAN++: Towards https://arxiv.org/pdf • Jennifer Jang 26 fast and scalable 2019 X X /1810.13105.pdf • Heinrich Jiang density clustering

MeanShift++:

Extremely Fast Mode- Seeking With https://arxiv.org/pdf • Jennifer Jang 27 2021 X X X Applications to /2104.00303.pdf • Heinrich Jiang

Segmentation and

Object Tracking

Self-correcting Q- https://arxiv.org/pdf • Rong Zhu 28 2021 X X Learning /2012.01100.pdf • Mattia Rigotti

• Taha Mansouri

• Mohamadreza A New Algorithm for https://arxiv.org/ftp/ Sadeghimoghada 29 Hidden Markov Models arxiv/papers/2102/ 2021 X X X X m Learning Problem 2102.07112.pdf • Iman Ghasemian

Sahebi

Deep Reinforcement Learning Aided Monte • Tz-Wei Mo https://arxiv.org/pdf 30 Carlo 2021 • Ronald Y. Chang X X X /2102.00178.pdf Tree Search for MIMO • Te-Yi Kan

Detection

Hard-Clustering with • Johannes Blomer https://arxiv.org/pdf 31 Gaussian Mixture 2016 • Sascha Brauer X X X X X X /1603.06478.pdf Models • Kathrin Bujna

• Marcel R.

Analysis of Ackermann https://arxiv.org/pdf 32 Agglomerative 2014 • Johannes Blomer X X /1012.3697.pdf Clustering • Daniel Kuntze

• Christian Sohler

Towards Evaluating the https://arxiv.org/pdf • Nicholas Carlini 33 Robustness of Neural 2017 X X /1608.04644.pdf • David Wagner Networks

DELVING INTO • Yanpei Liu TRANSFERABLE https://arxiv.org/pdf • Xinyun Chen 34 ADVERSARIAL 2017 X X /1611.02770.pdf • Chang Liu EXAMPLES AND • Dawn Song BLACK-BOX ATTACKS

• Chaowei Xiao

• Bo Li Generating Adversarial https://arxiv.org/pdf • Jun-Yan Zhu 35 Examples with 2019 X X /1801.02610.pdf • Warren He Adversarial Networks • Mingyan Liu

• Dawn Song

• Roman

Werpachowski Detecting Overfitting via https://arxiv.org/pdf 36 2019 • András György X X Adversarial Examples /1903.02380.pdf • Csaba

Szepesvári

Cybersecurity https://www.enisa. • Georgia Dede

Challenges in the europa.eu/publicati • Ronan Hamon

37 Uptake of Artificial ons/enisa-jrc- 2021 • Rossen X X X

Intelligence in cybersecurity- Naydenov

Autonomous Driving challenges-in-the- • Henrik Junklewitz

46 SECURING MACHINE LEARNING ALGORITHMS December 2021 d is m e ing e ing is ng rv Type of data rn rn rni orc a ingested e e Lea inf l uperv Lea nsupe d e nt S U e R e dat n o Title Source ti Author a ta n Index c s a o li e D ti ng ri ion ng ion ion t d s a ri ub ge e s P a ic e ns rdi ideo s if e Tex e ture s duct wa V Im gre s lust e m e im e Ti R la C D R R truc C S

uptake-of-artificial- • Apostolos

intelligence-in- Malatras autonomous- • Ignacio Sanchez

driving

Rapid Object Detection https://www.cs.cm using a Boosted u.edu/~efros/cours • Paul Viola 38 2001 X X Cascade of Simple es/LBMV07/Paper • Michael Jones

Features s/viola-cvpr-01.pdf

Detection of Advanced https://arxiv.org/ftp/ • Sanjay Sharma Malware 39 arxiv/papers/1903/ 2019 • C. Rama Krishna X X X by Machine Learning 1903.02966.pdf • Sanjay K. Sahay Techniques

https://www.theseu MACHINE LEARNING s.fi/bitstream/handl METHODS FOR e/10024/123412/T • Kateryna 40 MALWARE 2017 X X X hesis_final.pdf?seq Chumachenko DETECTION AND uence=1&isAllowe CLASSIFICATION d=y

• Bojan Kolosnjaji

Adversarial Malware • Ambra Demontis

Binaries: Evading Deep • Battista Biggio https://arxiv.org/pdf 41 Learning for Malware 2018 • Davide Maiorca X X X /1803.04173.pdf Detection in • Giorgio Giacinto

Executables • Claudia Eckert

• Fabio Roli

Generating Adversarial Malware Examples for https://arxiv.org/pdf • Weiwei Hu 42 2017 X X X Black-Box Attacks /1702.05983.pdf • Ying Tan

Based on GAN

Exploring Adversarial • Octavian Suciu https://arxiv.org/pdf 43 Examples in Malware 2018 • Scott E. Coull X X X /1810.08280.pdf Detection • Jeffrey Johns

• Felix Kreuk

Deceiving End-to-End • Assi Barak

Deep Learning Malware https://arxiv.org/pdf • Shir Aviv-Reuven 44 2019 X X X Detectors using /1802.04528.pdf • Moran Baruch

Adversarial Examples • Benny Pinkas

• Joseph Keshet

•Edward Raff

•Jon Barker

Malware Detection by https://arxiv.org/pdf •Jared Sylvester 45 2017 X X X Eating a Whole EXE /1710.09435.pdf •Robert Brandon

•Bryan Catanzaro

•Charles Nicholas

Black-box attacks against rnn based https://arxiv.org/pdf •Weiwei Hu 46 2017 X X X malware detection /1705.08131.pdf •Ying Tan

algorithms

Generic black-box end- • Shai Rosenberg to-end attack against https://arxiv.org/pdf • Asaf Shabtai 47 rnns and other API calls 2018 X X X /1707.05970.pdf • Lior Rokach based malware •Yuval Elovici classifiers

47 SECURING MACHINE LEARNING ALGORITHMS December 2021 d is m e ing e ing is ng rv Type of data rn rn rni orc a ingested e e Lea inf l uperv Lea nsupe d e nt S U e R e dat n o Title Source ti Author a ta n Index c s a o li e D ti ng ri ion ng ion ion t d s a ri ub ge e s P a ic e ns rdi ideo s if e Tex e ture s duct wa V Im gre s lust e m e im e Ti R la C D R R truc C S

• Raphael Labaca-

Castro • Luis Muñoz-

González Universal Adversarial •Feargus https://arxiv.org/pdf 48 Perturbations for 2021 Pendlebury X X X /2102.06747.pdf Malware •Gabi Dreo

Rodosek •Fabio Pierazzi

• Lorenzo

Cavallaro

MDEA: Malware • Xiruo Wang Detection with https://arxiv.org/pdf 49 2020 • Risto X X X Evolutionary /2002.03331.pdf Miikkulainen Adversarial Learning

Attack and Defense of • Jack W. Stokes

Dynamic Analysis- • De Wang, Mady https://arxiv.org/pdf 50 Based, Adversarial 2017 Marinescu X X X /1712.05919.pdf Neural Malware • Marc Marino

Classification Models • Brian Bussone

Robust Android • Hemant Rathore Malware Detection https://arxiv.org/pdf • Sanjay K. Sahay 51 System against 2021 X X X /2101.12031.pdf • Piyush Nikam Adversarial Attacks • Mohit Sewak using Q-Learning

Binary Black-box • Mohammadreza

Evasion Attacks Ebrahimi Against Deep Learning- • Ning https://arxiv.org/pdf 52 based Static Malware 2020 Zhang, James Hu X X X /2012.07994.pdf Detectors with • Muhammad Taqi

Adversarial Byte-Level Raza Language Model • Hsinchun Chen

MalFox: Camouflaged •Fangtian Zhong

Adversarial Malware • Xiuzhen Cheng

Example Generation https://arxiv.org/pdf • Dongxiao YuBei 53 2020 X X X Based on C-GANs /2011.01509.pdf Gong Against Black-Box • Shuaiwen Song

Detectors • Jiguo Yu

• Ishai Rosenberg

• Shai Generating End-to-End Meir, Jonathan Adversarial Examples https://arxiv.org/pdf 54 2020 Berrebi X X X for Malware Classifiers /2009.13243.pdf • Ilay Gordon Using Explainability • Guillaume Sicard

• Eli (Omid)David

Adversarial EXEmples: • Luca Demetrio

A Survey and • Scott E. Coull

Experimental • Battista Biggio https://arxiv.org/pdf 55 Evaluation of Practical 2020 • Giovanni Lagorio X X X /2008.07125.pdf Attacks on Machine • Alessandro

Learning for Windows Armando Malware Detection • Fabio Roli

Adversarial Machine • Ihai Rosenberg

Learning Attacks and https://arxiv.org/pdf • Asaf Shabtai 56 2020 X X X Defense Methods in the /2007.02407.pdf • Yuval Elovici

Cyber Security Domain • Lior Rokach

48 SECURING MACHINE LEARNING ALGORITHMS December 2021 d is m e ing e ing is ng rv Type of data rn rn rni orc a ingested e e Lea inf l uperv Lea nsupe d e nt S U e R e dat n o Title Source ti Author a ta n Index c s a o li e D ti ng ri ion ng ion ion t d s a ri ub ge e s P a ic e ns rdi ideo s if e Tex e ture s duct wa V Im gre s lust e m e im e Ti R la C D R R truc C S

• Wei Song Automatic Generation • Xuezixiang Li of Adversarial https://arxiv.org/pdf • Sadia Afroz 57 Examples for 2020 X X X /2003.03100.pdf • Deepali Garg Interpreting Malware • Dmitry Kuznetsov Classifiers • Heng Yin

• Aminollah

Khormali COPYCAT: Practical • Ahmed Adversarial Attacks on https://arxiv.org/pdf 58 2019 Abusnaina X X X Visualisation-Based /1909.09735.pdf • Songqing Chen Malware Detection • DaeHun Nyang

• Aziz Mohaisen

Effectiveness of • Robert Adversarial Examples https://arxiv.org/pdf 59 2019 Podschwadt X X X and Defenses for /1909.04778.pdf • Hassan Takabi Malware Classification

• Shashank Srikant

• Sijia Liu

• Tamara Generating Adversarial Mitrovska Computer Programs https://arxiv.org/pdf 60 2021 • Shiyu Chang X X X using Optimiser /2103.11882.pdf • Quanfu Fan Obfuscations • Gaoyuan Zhang

• Una-May

O'Reilly

• Ecenaz Erdemir Adversarial Robustness https://arxiv.org/pdf • Jeffrey Bickford 61 with Non-uniform 2021 X X X /2102.12002.pdf • Luca Melis Perturbations • Sergul Aydore

• Alexandre Araujo Robust Neural https://hal.archives • Laurent Meunier Networks using -ouvertes.fr/hal- 62 2019 • Rafael Pinot X X Randomiser 02380184v2/docu • Benjamin Adversarial Training ment Negrevergne

•Rafael Pinot

• Laurent Meunier

Theoretical evidence • Alexandre Araujo

for adversarial https://arxiv.org/pdf • Hisashi Kashima 63 2019 X X robustness through /1902.01148.pdf • Florian Yger

randomisation • Cédric Gouy-

Pailler • Jamal Atif

• Cihang Xie

Mitigating Adversarial • Jianyu Wang https://arxiv.org/pdf 64 Effects Through 2017 • Zhishuai Zhang X X /1711.01991.pdf Randomisation • Zhou Ren

• Alan Yuille

•Tong Chen https://cybersecurit Adversarial attack and • Jiqiang Liu y.springeropen.co defense in • Yingxiao Xiang 65 m/track/pdf/10.118 2019 X X X X X reinforcement learning- • Wenjia Niu 6/s42400-019from AI security view • Endong Tong and 0027-x.pdf Zhen Han

49 SECURING MACHINE LEARNING ALGORITHMS December 2021 d is m e ing e ing is ng rv Type of data rn rn rni orc a ingested e e Lea inf l uperv Lea nsupe d e nt S U e R e dat n o Title Source ti Author a ta n Index c s a o li e D ti ng ri ion ng ion ion t d s a ri ub ge e s P a ic e ns rdi ideo s if e Tex e ture s duct wa V Im gre s lust e m e im e Ti R la C D R R truc C S

TextAttack: A Framework • John X. Morris for Adversarial Attacks, • Eli Lifland

Data https://arxiv.org/pdf • Jin Yong Yoo 66 2020 X X Augmentation, and /2005.05909.pdf • Jake Grigsby

Adversarial Training in • Di Jin NLP • Yanjun Qi

• Anirban

Chakraborty • Manaar Alam Adversarial Attacks and https://arxiv.org/pdf • Vishal Dey 67 2018 X X Defences: A Survey /1810.00069.pdf • Anupam

Chattopadhyay • Debdeep

Mukhopadhyay

DATA • Ferhat Ozgur

AUGMENTATION Catak BASED MALWARE https://arxiv.org/pdf • Javed Ahmed 68 2021 X X X DETECTION USING /2010.01862.pdf • Kevser Sahinbas CONVOLUTIONAL • Zahid Hussain

NEURAL NETWORKS Khand

https://gala.gre.ac. • Nikolaos

uk/id/eprint/25226/ Pitropakisa 7/25226%20LOUK • Emmanouil A Taxonomy and AS_Taxonomy_An Panaousisb Survey of Attacks 69 d_Survey_Of_Atta 2019 • Thanassis X X Against Machine cks_Against_Mach Giannetsosc Learning ine_Learning_%28 • Eleftherios

AAM%29_2019.pd Anastasiadisd f • George Loukase

• Andrew Ilyas

• Shibani Santurkar Adversarial examples • Dimitris Tsipras https://arxiv.org/pdf 70 are not bugs, they are 2019 • Logan Engstrom X X /1905.02175.pdf features. • Brandon Tran

• Aleksander

Madry

• Rémi Bernhard • Pierre-Alain

Moellic Impact of Spatial • Martial Mermillod Frequency Based https://arxiv.org/pdf 71 2021 • Yannick Bourrier X X Constraints on /2104.12679.pdf • Romain Adversarial Robustness Cohendet • Miguel Solinas

• Marina Reyboz

• Vikash Sehwag • Saeed

Mahloujifar Improving Adversarial https://arxiv.org/pdf • Tinashe Handina 72 Robustness Using 2021 X X /2104.09425.pdf • Sihui Dai Proxy Distributions • Chong Xiang

• Mung Chiang • Prateek Mittal

• Guoqiu Wang Improving Adversarial https://arxiv.org/pdf • Huanqian Yan 73 Transferability with 2021 X X /2105.04834.pdf • Ying Guo Gradient Refining • Xingxing Wei

50 SECURING MACHINE LEARNING ALGORITHMS December 2021 d is m e ing e ing is ng rv Type of data rn rn rni orc a ingested e e Lea inf l uperv Lea nsupe d e nt S U e R e dat n o Title Source ti Author a ta n Index c s a o li e D ti ng ri ion ng ion ion t d s a ri ub ge e s P a ic e ns rdi ideo s if e Tex e ture s duct wa V Im gre s lust e m e im e Ti R la C D R R truc C S

• Guiyu Tian Poisoning MorphNet for https://arxiv.org/pdf • Wenhao Jiang 74 Clean-Label Backdoor 2021 X X /2105.04839.pdf • Wei Liu Attack to Point Clouds • Yadong Mu

• Siyue Wang High-Robustness, Low- • Xiao Wang Transferability https://arxiv.org/pdf 75 2021 • Pin-Yu Chen X X Fingerprinting of Neural /2105.07078.pdf • Pu Zhao Networks • Xue Lin

Detecting Adversarial • Yao Li

Examples with https://arxiv.org/pdf • Tongyi Tang 76 2021 X X Bayesian Neural /2105.08620.pdf • Cho-Jui Hsieh

Network • Thomas C. M. Lee

• Mingfu Xue Robust Backdoor • Can He Attacks against Deep https://arxiv.org/pdf 77 2021 • Shichang Sun X X Neural Networks in /2104.07395.pdf • Jian Wang Real Physical World • Weiqiang Liu

secml-malware: A

Python Library for https://arxiv.org/pdf • Luca Demetrio 78 Adversarial Robustness 2021 X X X /2104.12848.pdf • Battista Biggio Evaluation of Windows

Malware Classifiers

• Nicolas M. Müller Defending against • Simon adversarial denial-of- https://arxiv.org/pdf 79 2021 Roschmann X X service data poisoning /2104.06744.pdf • Konstantin attack Böttinger

• Weiyi Zhang

• Shuning Zhao Attack on practical • Le Liu speaker verification https://arxiv.org/pdf • Jianmin Li 80 system using universal 2021 X X /2105.09022.pdf • Xingliang Cheng adversarial • Thomas Fang perturbations Zheng • Xiaolin Hu

Exploiting • Faiq Khalid •

Vulnerabilities in Deep Muhammad https://arxiv.org/pdf 81 Neural Networks: 2021 Abdullah Hanif X X /2105.03251.pdf Adversarial and Fault- • Muhammad

Injection Attacks Shafique

Dynamic Defense • Ruoxi Qin Approach for Adversarial https://arxiv.org/ftp/ • Linyuan Wang Robustness in Deep 82 arxiv/papers/2105/ 2021 • Xingyuan Chen X X Neural Networks via 2105.02803.pdf • Xuehui Du Stochastic Ensemble • Bin Yan Smoothed Model

• Bangjie Yin

• Wenxuan Wang

Adv-Makeup: A New • Taiping Yao

Imperceptible and https://arxiv.org/pdf • Junfeng Guo 83 2021 X X Transferable Attack on /2105.03162.pdf • Zelun Kong

Face Recognition • Shouhong Ding

• Jilin Li

• Cong Liu

51 SECURING MACHINE LEARNING ALGORITHMS December 2021 d is m e ing e ing is ng rv Type of data rn rn rni orc a ingested e e Lea inf l uperv Lea nsupe d e nt S U e R e dat n o Title Source ti Author a ta n Index c s a o li e D ti ng ri ion ng ion ion t d s a ri ub ge e s P a ic e ns rdi ideo s if e Tex e ture s duct wa V Im gre s lust e m e im e Ti R la C D R R truc C S

• Micah Goldblum Adversarial Attacks on • Avi Machine Learning https://arxiv.org/pdf 84 2020 Schwarzschild X X X Systems for High- /2002.09565.pdf • Ankit B. Patel Frequency Trading • Tom Goldstein

M. Hausknecht & Al., Deep Recurrent Q- • Matthew https://arxiv.org/pdf 85 Learning for Partially 2017 Hausknecht X X X /1507.06527.pdf Observable MDPs, in • Peter Stone

arXiv, January 2017

• Hyrum S.

Anderson Learning Malware • Hyrum S. https://arxiv.org/pdf 86 Models via 2018 Anderson X X X /1801.08917.pdf Reinforcement Learnin • Bobby Filar

• David Evans

• Phil Roth

Wild patterns: ten years after the rise of https://arxiv.org/pdf • Battista Biggioa 87 2018 X X adversarial Machine /1712.03141.pdf • Fabio Rolia

Learning

https://www.resear chgate.net/publicat • Qiang Liu

A survey on security ion/323154427_A_ • Pan Li

threats and defensive Survey_on_Securit • Wentao Zhao

88 techniques of machine y_Threats_and_De 2018 • Wei Cai X X X X X X X X X

learning: a data driven fensive_Technique • Shui Yu

view s_of_Machine_Lea • Victor C. M.

rning_A_Data_Driv Leung

en_View

• Han Xu

• Yao Ma Adversarial attacks and • Haochen Liu defenses in images, https://arxiv.org/pdf 89 2020 • Debayan Deb X X X X x X X graphs and text: a /1909.08072.pdf • Hui Liu review • Jiliang Tang

• Anil K. Jain

https://www.resear chgate.net/publicat • Daniel Lowd 90 Adversarial Learning 2005 X X ion/221654486_Ad • Christopher Meek

versarial_learning

• Ling Huang

https://citeseerx.ist. • Anthony D.

psu.edu/viewdoc/d Joseph Adversarial Machine- 91 ownload?doi=10.1. 2011 • Blaine Nelson X X Learning 1.360.168&rep=re • Benjamin I. P.

p1&type=pdf Rubinstein • J. D. Tygar

https://www.resear

chgate.net/publicat A Survey of ion/327074374_A_ Adversarial Machine 92 Survey_of_Advers 2018 • Vasisht Duddu X X X X X X X X X X Learning in Cyber arial_Machine_Lea Warfare rning_in_Cyber_W

arfare

52 SECURING MACHINE LEARNING ALGORITHMS December 2021 d is m e ing e ing is ng rv Type of data rn rn rni orc a ingested e e Lea inf l uperv Lea nsupe d e nt S U e R e dat n o Title Source ti Author a ta n Index c s a o li e D ti ng ri ion ng ion ion t d s a ri ub ge e s P a ic e ns rdi ideo s if e Tex e ture s duct wa V Im gre s lust e m e im e Ti R la C D R R truc C S

Intelligence artificielle et https://www.wavest cybersécurité : protéger one.com/app/uploa • Carole Meziat 93 2019 X X X X X dès maintenant le ds/2019/09/IA- • Laurent Guille

monde de demain cyber-2019.pdf

• Ian J. Goodfellow Explaining and https://arxiv.org/pdf • Jonathon Shlens 94 Harnessing Adversarial 2020 X X /1412.6572.pdf • Christian Examples Szegedy

https://papers.nips. Feature Crosscc/paper/2014/file/ • Yevgeniy Substitution in 95 8597a6cfa74defcb 2014 Vorobeychik X X Adversarial de3047c891d78f90 • Bo Li Classification -Paper.pdf

• Wenqi Wei

• Ling Liu

Adversarial Examples • Margaret Loper

in Deep Learning: http://arxiv.org/abs/ • Stacey Truex 96 2018 X X Characterisation and 1807.00051 • Lei Yu

Divergence • Mehmet Emre

Gursoy • Yanzhao Wu

Maximal Jacobianhttp://arxiv.org/abs/ • Rey Wiyatno 97 based Saliency Map 2018 X X 1808.07945 • Anqi Xu Attack

Robustness and • Huan Xu https://www.jmlr.or Regularisation of • Constantine 98 g/papers/volume10 2009 Support Vector Caramanis /xu09b/xu09b.pdf Machines • Shie Mannor

• Xinyun Chen Targeted Backdoor • Chang Liu Attacks on Deep https://arxiv.org/pdf 99 2017 • Bo Li X X Learning Systems /1712.05526.pdf • Kimberly Lu Using Data Poisoning • Dawn Song

Towards evaluating the https://arxiv.org/ab • Nicholas Carlini 100 robustness of neural 2016 X X s/1608.04644 • David Wagner networks

• Reuben Feinman

• Ryan R. Curtin Detecting adversarial https://arxiv.org/ab 101 2017 • Saurabh Shintre X X samples from artifacts s/1703.00410 • Andrew B.

Gardner

Single Headed

Attention RNN: Stop https://arxiv.org/pdf 102 2019 • Steven Merity X X Thinking With Your /1911.11423.pdf

Head

• Pauline Luc Semantic Segmentation https://arxiv.org/pdf • Camille Couprie 103 using Adversarial 2016 X X /1611.08408.pdf • Soumith Chintala Networks • Jakob Verbeek

• Ming Pang

Unorganiser Malicious https://arxiv.org/pdf • Wei Gao 104 2018 Attacks Detection /1610.04086.pdf • Min Tao

• Zhi-Hua Zhou

53 SECURING MACHINE LEARNING ALGORITHMS December 2021 d is m e ing e ing is ng rv Type of data rn rn rni orc a ingested e e Lea inf l uperv Lea nsupe d e nt S U e R e dat n o Title Source ti Author a ta n Index c s a o li e D ti ng ri ion ng ion ion t d s a ri ub ge e s P a ic e ns rdi ideo s if e Tex e ture s duct wa V Im gre s lust e m e im e Ti R la C D R R truc C S

• CHRISTOPHE

ANDRIEU https://link.springer • NANDO DE An Introduction to .com/content/pdf/1 FREITAS 105 MCMC for Machine 2003 0.1023/A:1020281 • ARNAUD Learning 327116.pdf DOUCET • MICHAEL I.

JORDAN

• Choon Hui Teo http://people.csail. • Amir Globerson Convex Learning with mit.edu/gamir/pubs 106 2007 • Sam Roweis X X Invariances /TeoGloRowSmo0 • Alexander J. 7.pdf Smola

Scalable Optimisation of Randomiser http://proceedings. • Bo Li

107 Operational Decisions mlr.press/v38/li15a 2015 • Yevgeniy X x

in Adversarial .pdf Vorobeychik

Classification Settings

https://www.scienc • Kui Ren Adversarial Attacks edirect.com/scienc • Tianhang Zheng 108 and Defenses in Deep 2020 X X X X X e/article/pii/S20958 • Zhan Qin Learning, 0991930503X • Xue Liu

Exploring the Space of https://arxiv.org/pdf • Pedro Tabacof 109 2016 X X Adversarial Images /1510.05328.pdf • Eduardo Valle

• Nicolas Papernot

Distillation as a • Patrick McDaniel

Defense to Adversarial https://arxiv.org/pdf • Xi Wu 110 2016 X X Perturbations against /1511.04508.pdf • Somesh Jha

Deep Neural Network • Ananthram

Swami

• Xingjun Ma

• Bo Li

• Yisen Wang

• Sarah M. Erfani Characterizing • Sudanthi adversarial subspaces https://arxiv.org/pdf 111 2018 Wijewickrema X X using local intrinsic /1801.02613.pdf • Grant dimensionality Schoenebeck • Dawn Song

• Michael E. Houle

• James Bailey

Defense still has a long

way: Adversarial Examples Are Not https://arxiv.org/pdf • Nicholas Carlini 112 2017 X X Easily Detected: /1705.07263.pdf • David Wagner

Bypassing Ten

Detection Methods

• Aleksander

Madry Towards Deep • Aleksandar Learning Models https://arxiv.org/pdf 113 2019 Makelov X X Resistant to Adversarial /1706.06083.pdf • Ludwig Schmidt Attacks • Dimitris Tsipras

• Adrian Vladu

54 SECURING MACHINE LEARNING ALGORITHMS December 2021 d is m e ing e ing is ng rv Type of data rn rn rni orc a ingested e e Lea inf l uperv Lea nsupe d e nt S U e R e dat n o Title Source ti Author a ta n Index c s a o li e D ti ng ri ion ng ion ion t d s a ri ub ge e s P a ic e ns rdi ideo s if e Tex e ture s duct wa V Im gre s lust e m e im e Ti R la C D R R truc C S

https://towardsdata

science.com/poiso Poisoning attacks on 114 ning-attacks-on- 2019 • Ilja Moisejevs X Machine Learning Machine-Learning- 1ff247c254db

A Unified Framework • Xuanqing Liu for Data Poisoning • Si Si https://arxiv.org/pdf 115 Attack to Graph-based 2019 • Xiaojin Zhu X X X /1910.14147.pdf Semi-supervised • Yang Li

Learning • Cho-Jui Hsieh

• Mengchen Zhao https://personal.ntu Data Poisoning Attacks • Bo An .edu.sg/boan/pape 116 on Multi-Task 2018 • Yaodong Yu X X X rs/AAAI18_MTL.pd Relationship Learning • Shulin Liu f • Sinno Jialin Pan

• Chang Liu

Robust High- • Bo Li https://arxiv.org/pdf 117 Dimensional Linear 2016 • Yevgeniy X X /1608.02257.pdf Regression Vorobeychik • Alina Oprea

• Jacob Steinhardt Certified Defenses for https://arxiv.org/pdf 118 2017 • Pang Wei Koh X X X Data Poisoning Attacks /1706.03691.pdf • Percy Liang

• Ian J. Goodfellow • Jean Pouget-

Abadie • Mehdi Mirza

Generative adversarial https://arxiv.org/pdf • Bing Xu 119 2014 X X nets /1406.2661.pdf • David Warde-

Farley • Sherjil Ozair

• Aaron Courville • Yoshua Bengio

• Qiang Liu

A Survey on Security • Pan Li https://ieeexplore.i Threats and Defensive • Wentao Zhao eee.org/stamp/sta 120 Techniques of Machine 2018 • Wei Cai X X X X mp.jsp?tp=&arnum Learning: A Data • Shui Yu ber=8290925 Driven View • Vctor C. M.

Leung

https://www.enisa. europa.eu/publicati Artificial Intelligence • Apostolos ons/artificial- 121 Cybersecurity 2021 Malatras intelligence- Challenges • Georgia Dede cybersecuritychallenges

https://ieeexplore.i A Taxonomy of ML for eee.org/stamp/sta 122 2020 • Martin Maas X X X X Systems Problems mp.jsp?tp=&arnum ber=9153088

55 SECURING MACHINE LEARNING ALGORITHMS December 2021 d is m e ing e ing is ng rv Type of data rn rn rni orc a ingested e e Lea inf l uperv Lea nsupe d e nt S U e R e dat n o Title Source ti Author a ta n Index c s a o li e D ti ng ri ion ng ion ion t d s a ri ub ge e s P a ic e ns rdi ideo s if e Tex e ture s duct wa V Im gre s lust e m e im e Ti R la C D R R truc C S

Deep Unsupervised Learning for • Arjun Kaushik

Generaliser • Mehrazin https://arxiv.org/pdf 123 Assignment Problems: 2021 Alizadeh X X X /2103.14548.pdf A Case-Study of • Omer Waqar

User-Association in • Hina Tabassum

Wireless Networks

https://elie.net/blog Attacks against /ai/attacks-against- 124 machine learning — an 2018 • Elie Bursztein X machine-learningoverview an-over

https://towardsdata

science.com/how-

How to attack Machine to-attack-machine-

Learning ( Evasion, learning-evasion- 125 2019 • Alex Polyakov X X Poisoning, Inference, poisoning-

Trojans, Backdoors) inference-trojans-

backdoors-

a7cb5832595c

https://docs.lib.pur • Yingqi Liu

due.edu/cgi/viewco • Shiqing Ma Trojaning Attack on 126 ntent.cgi?article=2 2017 • Yousra Aafer X X X X Neural Networks 782&context=cstec • Wen-Chuang Lee

h • Juan Zhai

Evaluating Input Perturbation Methods • Lukas Brunke https://arxiv.org/pdf 127 for Interpreting CNNs 2021 • Prateek Agrawal X X /2101.10977.pdf and Saliency Map • Nikhil George

Comparison

Capsule Network is Not • Jindong Gu https://arxiv.org/pdf 128 More Robust than 2021 • Volker Tresp X X /2103.15459.pdf Convolutional Network • Han Hu

MagFace: A Universal • Qiang Meng

Representation for https://arxiv.org/pdf • Shichao Zhao 129 2021 X X Face Recognition and /2103.06627.pdf • Zhida Huang

Quality Assessment • Feng Zhou

Merge and Label: A https://www.aclweb novel neural network • Joseph Fisher 130 .org/anthology/P19 2019 X X architecture for nested • Andreas Vlachos -1585.pdf NER

• Soroush Mehrir

• Kundan Kumarrr

SampleRNN: An • Ishaan Gulrajanir

Unconditional End-to- https://arxiv.org/pdf • Rithesh Kumarr 131 2017 X End Neural Audio /1612.07837.pdf • Shubham Jainr

Generation Model • Jose Sotelor

• Aaron Courviller

• Yoshua Bengio

Generating Black Metal and Math Rock: https://arxiv.org/pdf • Zack Zukowski 132 2018 X Beyond Bach, /1811.06639.pdf • CJ Carr

Beethoven, and Beatles

56 SECURING MACHINE LEARNING ALGORITHMS December 2021 d is m e ing e ing is ng rv Type of data rn rn rni orc a ingested e e Lea inf l uperv Lea nsupe d e nt S U e R e dat n o Title Source ti Author a ta n Index c s a o li e D ti ng ri ion ng ion ion t d s a ri ub ge e s P a ic e ns rdi ideo s if e Tex e ture s duct wa V Im gre s lust e m e im e Ti R la C D R R truc C S

• Haotong Qina

• Ruihao Gonga

Binary Neural https://arxiv.org/pdf • Xianglong Liu 133 2020 X X Networks: A Survey /2004.03333.pdf • Xiao Baie

• Jingkuan Songc

• Nicu Sebed

Towards Accurate • Xiaofan Lin https://arxiv.org/pdf 134 Binary Convolutional 2017 • Cong Zhao X X /1711.11294.pdf Neural Network • Wei Pan

• Yunus Saatchi https://arxiv.org/pdf 135 Bayesian GAN 2017 • Andrew Gordon X X X X X /1705.09558.pdf Wilson

• Neda Tavakoli A Comparison of https://par.nsf.gov/ • Sima Siami- ARIMA and LSTM in 136 servlets/purl/10186 2018 Namini X X Forecasting Time 768 • Akbar Siami Series Namin

• John Schulman

•Filip Wolski Proximal Policy https://arxiv.org/pdf 137 2017 •Prafulla Dhariwal, X Optimisation Algorithms /1707.06347.pdf •Alec Radford,

•Oleg Klimov

https://op.europa.e

u/en/publication-

detail/- Ethics Guidelines for 138 /publication/d3988 2019 Trustworthy AI 569-0434-11ea-

8c1f-

01aa75ed71a1

• Battista

BiggioIgino https://link.springer • Igino Corona

Evasion Attacks against .com/content/pdf/1 • Davide Maiorca

139 Machine Learning at 0.1007%2F978-3- 2017 • Blaine Nelson X X

Test time 642-40994- • Nedim Srndie

3_25.pdf • Pavel Laskov

• Giorgio Giacinto

• Fabio Roli

Robustness

Evaluations of https://www.mdpi.c • Corey Dunn Sustainable Machine om/2071- • Nour Moustafa 140 Learning Models 2020 X X 1050/12/16/6434/h • Benjamin against Data Poisoning tm Turnbull Attacks in the Internet

of Things

Defending network https://www.scienc intrusion detection • Marek Pawlicki edirect.com/scienc 141 systems against 2020 • Michał Choraś e/article/abs/pii/S0 adversarial evasion • Rafał Kozik 167739X20303368 attacks

Defending SVMs against Poisoning • Hu Ding https://arxiv.org/pdf 142 Attacks: the Hardness 2021 • Fan Yang X X /2006.07757.pdf and DBSCAN • Jiawei Huang

Approach

57 SECURING MACHINE LEARNING ALGORITHMS December 2021 d is m e ing e ing is ng rv Type of data rn rn rni orc a ingested e e Lea inf l uperv Lea nsupe d e nt S U e R e dat n o Title Source ti Author a ta n Index c s a o li e D ti ng ri ion ng ion ion t d s a ri ub ge e s P a ic e ns rdi ideo s if e Tex e ture s duct wa V Im gre s lust e m e im e Ti R la C D R R truc C S

• Battista Biggio Bagging classifiers for http://pralab.diee.u • Igino Corona fighting poisoning nica.it/sites/default/ 143 2011 • Giorgio Fumera X X X attacks in adversarial files/biggio11- • Giorgio Giacinto classification tasks mcs.pdf • Fabio Roli

• Tom B. Brown

• Dandelion Mané https://arxiv.org/pdf 144 Adversarial Patch 2018 • Aurko Roy X X /1712.09665.pdf • Martín Abadi

• Justin Gilmer

Model Inversion Attacks • Matt Fredrikson https://dl.acm.org/d that Exploit Confidence • Somesh Jha 145 oi/pdf/10.1145/281 2015 X X X Information and Basic • Thomas 0103.2813677 Countermeasures Ristenpart

Membership Inference • Reza Shokri

Attacks against https://arxiv.org/pdf • Marco Stronati 146 2017 X X X X Machine Learning /1610.05820.pdf • Congzheng Song

Models • Vitaly Shmatikov

STRIDE-AI: An

Approach to Identifying https://github.com/ • Lara Mauri 147 Vulnerabilities of LaraMauri/STRIDE 2021 • Ernesto Damiani Machine Learning -AI

Assets

https://www.ai4eu.

eu/news/meaningf

ul-artificial- For a meaningful 148 intelligencetowards 2018 • Cedric Villani Artificial Intelligence -french-artificial-

and-european-

strategy

https://www.ai4eu.

eu/news/strategic-

Strategic Action Plan action-plan- 149 2019 for Artificial Intelligence artificial-

intelligence-

netherlands

• Hamid Eghdal-

zadeh • Khaled Koutini

• Paul Primus

• Verena On Data Augmentation https://arxiv.org/pdf Haunschmid 150 and Adversarial risk: An 2020 X X /2007.02650.pdf • Michal empirical Analysis Lewandowski • Werner Zellinger

• Bernhard

A.Moser • Gerhard Widmer

https://ieeexplore.i Review of Deep eee.org/stamp/sta • Ajay Shrestha 151 Learning Algorithms 2019 mp.jsp?tp=&arnum • Ausif Mahmood and Architectures ber=8694781

58 SECURING MACHINE LEARNING ALGORITHMS December 2021 d is m e ing e ing is ng rv Type of data rn rn rni orc a ingested e e Lea inf l uperv Lea nsupe d e nt S U e R e dat n o Title Source ti Author a ta n Index c s a o li e D ti ng ri ion ng ion ion t d s a ri ub ge e s P a ic e ns rdi ideo s if e Tex e ture s duct wa V Im gre s lust e m e im e Ti R la C D R R truc C S

Learning in a Large • Benjamin I. P.

Function Space: Rubinstein https://arxiv.org/pdf 152 Privacy-Preserving 2009 • Peter L. Bartlett X /0911.5708.pdf Mechanisms for SVM •Ling Huang

Learning • Nina Taft

• Liang Tong

Improving Robustness • Bo Li

of ML Classifiers • Chen Hajaj https://arxiv.org/pdf 153 against Realizable 2019 • Chaowei Xiao X X /1708.08327.pdf Evasion Attacks Using • Ning Zhang

Conserved Features • Yevgeniy

Vorobeychik

• Chaowei Xiao

• Jun-Yan Zhu

Spatially Transformed https://arxiv.org/pdf • Bo Li 154 2018 X X Adversarial Examples /1801.02612.pdf • Warren He

• Mingyan Liu

• Dawn Song

• Arjun Nitin

Exploring the Space of Bhagoji https://arxiv.org/pdf 155 Black-box Attacks on 2015 • Warren He X X /1712.09491.pdf Deep Neural Networks • Bo Li

• Dawn Song

• Kevin Eykholt

• Ivan Evtimov

• Earlence

Robust Physical-World Fernandes Attacks on Deep https://arxiv.org/pdf • Bo Li 156 2018 X X Learning Visual /1707.08945.pdf • Amir Rahmati

Classification • Chaowei Xiao

• Atul Prakash

• Tadayoshi Kohno

• Dawn Song

• PIERRE-

FRANÇCOIS

Hybrid Isolation Forest - MARTEAU https://arxiv.org/pdf 157 Application to Intrusion 2017 • SAEID SOHEILY- X X X X X X /1705.03800.pdf Detection KHAH • NICOLAS

BÉCHET

• Neda Tavakoli The Performance of https://par.nsf.gov/ • Sima Siami- LSTM and BiLSTM in 158 servlets/purl/10186 2019 Namini Forecasting Time 554 • Akbar Siami Series Namin

• Maurras Togbe

• Mariam Barry Anomaly Detection for https://hal.archives • Aliou Boly Data Streams Based on -ouvertes.fr/hal- • Yousra 159 Isolation 2020 X X X 02874869/docume Chabchoub Forest using Scikitnt • Raja Chiky multiflow • Jacob Montiel

• Vinh-Thuy Tran

59 SECURING MACHINE LEARNING ALGORITHMS December 2021 d is m e ing e ing is ng rv Type of data rn rn rni orc a ingested e e Lea inf l uperv Lea nsupe d e nt S U e R e dat n o Title Source ti Author a ta n Index c s a o li e D ti ng ri ion ng ion ion t d s a ri ub ge e s P a ic e ns rdi ideo s if e Tex e ture s duct wa V Im gre s lust e m e im e Ti R la C D R R truc C S

• Wei Wang Robust Differentiable https://arxiv.org/pdf • Zheng Dang 160 2021 SVD /2104.03821.pdf • Yinlin Hu • Pascal Fua

Accurate Stock Price https://arxiv.org/ftp/ Forecasting Using • Jaydip Sen 161 arxiv/papers/2103/ 2021 X X Robust and Optimiser • Sidra Mehtab 2103.15096.pdf Deep Learning Models

• Christian Vulnerabilities of Berghoff https://arxiv.org/pdf ConnectioNIST 800-53 162 /2003.08837.pdf 2020 • Matthias Neu X X X X X X AI Applications: Evaluation and Defence • Arndt Von

Twickel

• Zhijie Deng

LiBRe: A Practical • Xiao Yang https://arxiv.org/pdf 163 Bayesian Approach to 2021 • Shizhen Xu X X /2103.14835.pdf Adversarial Detection • Hang Su • Jun Zhu

• Christian

Berghoff,

• Battista Biggio

• Elisa Brummel

• Vasilios Danos

• Thomas Doms

• Heiko Ehrich

• Thorsten

Gantevoort

• Barbara Hammer

• Joachim Iden https://www.bsi.bu nd.de/SharedDocs/ • Sven Jacob

Downloads/EN/BSI • Heidy Khlaaf

Towards Auditable AI /KI/Towards_Audit • Lars Komrowski 164 2021 Systems able_AI_Systems. • Robert Kröwing pdf?__blob=public • Jan Hendrik ationFile&v=4 Metzen

• Matthias Neu

• Fabian Petsch

• Maximilian

Poretschkin

• Wojciech Samek

• Hendrik Schäbe

• Arndt von

Twickel

• Martin Vechev

• Thomas

Wiegand

• Ian J. Goodfellow • David Warde-

https://arxiv.org/pdf Farley 165 Maxout network 2013 X X /1302.4389.pdf • Mehdi Mirza

• Aaron Courville • Yoshua Bengio

60 SECURING MACHINE LEARNING ALGORITHMS December 2021 d is m e ing e ing is ng rv Type of data rn rn rni orc a ingested e e Lea inf l uperv Lea nsupe d e nt S U e R e dat n o Title Source ti Author a ta n Index c s a o li e D ti ng ri ion ng ion ion t d s a ri ub ge e s P a ic e ns rdi ideo s if e Tex e ture s duct wa V Im gre s lust e m e im e Ti R la C D R R truc C S

• Victor

Shepardson A Taxonomy of ML https://berryvilleiml. 166 2019 • Gary McGraw Attacks com/taxonomy/ • Harold Figueroa

• Richie Bonett

BadNets: Identifying • Tianyu Gu

Vulnerabilities in the https://arxiv.org/pdf • Brendan Dolan- 167 2019 X X Machine Learning /1708.06733.pdf Gavitt Model Supply Chain • Siddharth Garg

https://towardsdata AI Security and science.com/ai- 168 Adversarial Machine 2019 • Alex Polyakov and-ml-security- Learning 101 101-6af8026675ff

https://www.resear • Indira Kalyan chgate.net/publicat Generative Adversarial Dutta ion/344519514_Ge 169 Networks in Security: A 2020 • Bhaskar Ghosh nerative_Adversari Survey • Michael Totaro al_Networks_in_S • Albert H. Carlson ecurity_A_Survey

https://www.bsi.bu

nd.de/SharedDocs/ Secure, Robust and Downloads/EN/BSI • Bundesamt für

170 transparent application /KI/Secure_robust_ 2021 Sicherheit in der

of AI and_transparent_a Informationstechnik

pplication_of_AI.pd

f

• Bryant Chen

• Wilka Carvalho

• Nathalie

Detecting Backdoor Baracaldo http://ceur- Attacks on Deep Neural • Heiko Ludwig 171 ws.org/Vol- 2018 X X X Networks by Activation • Benjamin 2301/paper_18.pdf Clustering Edwards • Taesung Lee

• Ian Molloy

• Biplav Srivastava

Detection of adversarial • Andrea Paudice

training examples in • Luis Munozhttps://arxiv.org/pdf 172 Poisoning Attacks 2018 Gonzalez X X X /1802.03041.pdf Trough Anomaly • Andras Gyorgy

Detection • Emil C.Lupu

https://blogs.scienti

ficamerican.com/o When AI Misjudgement 173 bservations/when- 2018 • Douglas Yeung is not an accident ai-misjudgment-is-

not-an-accident/

https://www.media. • Samuel

mit.edu/publication G.Finlayson Adversarial attacks on s/adversarial- • Jonathan Zittrain 174 medical machine 2029 attacks •Joi Ito learning -on-medical- • Andrew L.Beam

machine-learning/ • Isaac S.Kohane

61 SECURING MACHINE LEARNING ALGORITHMS December 2021 d is m e ing e ing is ng rv Type of data rn rn rni orc a ingested e e Lea inf l uperv Lea nsupe d e nt S U e R e dat n o Title Source ti Author a ta n Index c s a o li e D ti ng ri ion ng ion ion t d s a ri ub ge e s P a ic e ns rdi ideo s if e Tex e ture s duct wa V Im gre s lust e m e im e Ti R la C D R R truc C S

https://thenewstack Camouflaged grafiti on .io/camouflagedroad signs can fool 175 graffiti-road-signs- 2017 • Kimberley Mok X X machine learning can-fool-machinemodels learning-models/

https://www.idgcon

nect.com/article/35

Deepfakes and deep 83356/deepfakes-

fraud: the new security and-deep-fraud- 176 challenge the-new-security- 2020 • Sadia Sajiad

of misinformation and challenge-of-

impersonation misinformation-

and-

impersonation.html

Stealing https://arxiv.org/pdf • B.Wang 177 Hyperparameters in 2019 X X X X /1802.05351.pdf • N.Z.Gong Machine Learning

Explanation-Guided • Giorgio Severi https://www.usenix Backdoor Poisoning • Jim Meyer 178 .org/system/files/se 2020 X X Attacks Against • Scott Coull c21-severi.pdf Malware Classifiers • Alina Oprea

• Gamaleldin F.

Elsayed Adversarial Examples • Shreya Shankar

that Fool both • Brian Cheung //arxiv.org/pdf/1802 179 Computer 2018 • Nicolas Papernot X X .08195.pd Vision and Time- • Alex Kurakin

Limited Humans • Ian Goodfellow

• Jascha Sohl-

Dickstein

With Great training https://www.usenix • Bolun Wang comes great .org/system/files/co • Bimal Viswanath vulnerability: 180 nference/usenixse 2018 • Yuanshun Yao Practical Attacks curity18/sec18- • Haitao Zheng against transfer wang.pdf • Ben Y.Zhao learning

• Nicolas Papernot

• Patrick McDaniel

Practical black-box • Ian Goodfellow https://arxiv.org/pdf 181 attacks against 2017 • Somesh Jha X X /1602.02697.pdf machine learning • Z. Berkay Celik

• Ananthram

Swami

• Seyed-Mohsen

Moosavi-Dezfooli Universal adversarial https://arxiv.org/pdf 182 2017 • Alhussein Fawzi X X perturbations /1610.08401.pdf • Omar Fawzi

• Pascal Frossard

Benchmarking neural • Dan Hendrycks network robustness to https://arxiv.org/pdf 183 2019 • Thomas X X common corruptions /1903.12261.pdf Dietterich and perturbations

62 SECURING MACHINE LEARNING ALGORITHMS December 2021 d is m e ing e ing is ng rv Type of data rn rn rni orc a ingested e e Lea inf l uperv Lea nsupe d e nt S U e R e dat n o Title Source ti Author a ta n Index c s a o li e D ti ng ri ion ng ion ion t d s a ri ub ge e s P a ic e ns rdi ideo s if e Tex e ture s duct wa V Im gre s lust e m e im e Ti R la C D R R truc C S

https://www.resear chgate.net/publicat

ion/341792988_Se Securing Artificial curing_Artificial_Int

Intelligence, Part 1 The elligence_Part_1_ 184 attack surface of The_attack_surfac 2019 • Sven Herping

machine learning and e_of_machine_lear its implications ning_and_its_impli

cations/link/5ed506 4a299bf1c67d3238

f4/download

Support vector http://proceedings. • Battista Biggio

185 machines under mlr.press/v20/biggi 2011 • Blaine Nelson X X X adversarial label noise o11/biggio11.pdf • Pavel Laskov

• Avi Just How Toxic is Data Schwarzschild

Poisoning? A Unified https://arxiv.org/pdf • Micah Golddblum 186 Benchmark for /2006.12557.pdf 2020 X X • Arjun Gupta Backdoor and • John P.Dickerson Data Poisoning Attacks • Tom Goldstein

• Huang Xiao

• Battista Biggio Is feature selection https://arxiv.org/pdf • Gavin Brown 187 secure against training 2018 /1804.07933.pdf • Giorgio Fumera data poisoning? • Claudia Eckert

• Fabio Roli

You Autocomplete Me: http://pages.cs.wis • Roei Schuster

Poisoning c.edu/~jerryzhu/ssl • Congzheng Song 188 2021 X X Vulnerabilities in Neural /pub/Mei2015Mach • Eran Tromer

Code Completion ine.pdf • Vitaly Shmatikov

• Yujie Ji • Xinyang Zhang Model-reuse attacks on https://arxiv.org/pdf 189 2018 • Shouling Ji X X X X X deep learning systems /1812.00483.pdf • Xiapu Luo

• Ting Wang

https://ieeexplore.i A survey on transfer • Sinno Jialin Pan 190 eee.org/document/ 2009 learning • Qiang Yang 5288526

• Ilia Shumailov

• Yiren Zhao Sponge examples: https://arxiv.org/pdf • Daniel Bates 191 Energy-Latency Attacks /2006.03463.pdf 2021 X X • Nicolas Papernot on Neural Network • Robert Mullins

• Ross Anderson

• Jiongcong Chen . Impact analysis of • Gaoqi Liang https://link.springer false data injection • Zexiang CAI .com/content/pdf/1 192 attacks 2016 • Chunchao Hu 0.1007%2Fs40565 on power system static • Yan Xu -016-0223-6.pdf security assessment • Fengji Luo

• Junhua Zhao

63 SECURING MACHINE LEARNING ALGORITHMS December 2021 d is m e ing e ing is ng rv Type of data rn rn rni orc a ingested e e Lea inf l uperv Lea nsupe d e nt S U e R e dat n o Title Source ti Author a ta n Index c s a o li e D ti ng ri ion ng ion ion t d s a ri ub ge e s P a ic e ns rdi ideo s if e Tex e ture s duct wa V Im gre s lust e m e im e Ti R la C D R R truc C S

https://www.resear chgate.net/publicat

The ND2DB attack: ion/250195790_Th Database content e_ND2DB_attack_ • Ariel Futoransky

193 extraction using timing Database_content 2007 • Damian Saura X attacks on the indexing _extraction_using_ • Ariel Waissbein

algorithms timing_attacks_on _the_indexing_alg

orithms

La nouvelle technologie 194 de protection des 2020 • Théo Ryffel

données

Reliable evaluation of adversarial robustness https://arxiv.org/pdf • Francesco Croce 195 with an ensemble of /2003.01690.pdf 2020 X X • Matthias Hein diverse parameter-free

attacks

• M.Jagielski Manipulating Machine • Alina Oprea

Learning: Poisoning • Battista Biggio https://arxiv.org/pdf 196 attacks and 2021 • Chang Liu X X /1804.00308.pdf countermeasures for • Cristina Nita-

regression learning Rotaru • Bo Li

Using Machine http://pages.cs.wis teaching to identify c.edu/~jerryzhu/ma • Shike Mei 197 Optimal training-set chineteaching/pub/ 2015 X X X • Xiaojin Zhu attacks on Machine Mei2015Machine.p

Learners df

• Yansong Gao • Chang Xu STRIP: A defence • Derui Wang against trojan attacks https://arxiv.org/pdf 198 2020 • Shiping Chen X X on deep neural /1902.06531.pdf • Damith C. networks Ranasinghe •Surya Nepal

• Blaine Nelson • Marco Barreno

• Fuching Jack Chi https://people.eecs • Anthony

Misleading learners: .berkeley.edu/~tyg D.Joseph 199 Co-opting your spam ar/papers/SML/mis 2009 • Benjamin I.P X X

filter leading.learners.pd Rubinstein f •Udam Saini

• Charles Sutton • J.D Tygar

•Kai Xia

Cascade Adversarial • Taesik Na

Machine learning https://arxiv.org/pdf • Jong Hwan Ko 200 2018 X X Regulariser with a /1708.02582.pdf • Saibal

unified embedding Mukhopadhyay

64 SECURING MACHINE LEARNING ALGORITHMS December 2021 d is m e ing e ing is ng rv Type of data rn rn rni orc a ingested e e Lea inf l uperv Lea nsupe d e nt S U e R e dat n o Title Source ti Author a ta n Index c s a o li e D ti ng ri ion ng ion ion t d s a ri ub ge e s P a ic e ns rdi ideo s if e Tex e ture s duct wa V Im gre s lust e m e im e Ti R la C D R R truc C S

• Tong Chen

Gradient band-based • Wenjia Niu

adversarial training for • Yingxiao Xiang https://arxiv.org/pdf 201 generaliser attack 2018 • Xiaoxuan Bai X X /1807.06752.pdf immunity of A3C path • Jiqiang Liu

finding. •Zhen Han

•Gang Li

Certifying Some • Adam Sinha

Distributional • Hongsoek https://arxiv.org/pdf 202 Robustness 2020 Namkoong X X /1710.10571.pdf with Principled • Riccardo Volpi

Adversarial Training • John Duchi

https://www.jmlr.or • Kamalika Differentially Private g/papers/volume12 Chaudhuri 203 Empircal Risk 2011 X X /chaudhuri11a/cha • Claire Monteleoni Minimisation udhuri11a.pdf • Anand D.Sarwate

•Elham Tabassi

•Kevin J. Burns A Taxonomy and https://csrc.nist.gov •Michael Terminology of 204 /publications/detail/ 2019 Hadjimichael Adversarial Machine nistir/8269/draft •Andres D. Learning Molina-Markham •Julian T. Sexton

Deep Defense: Training • Ziang Yan https://arxiv.org/pdf 205 DNNs with improved 2018 • Yiwen Guo X X /1803.00404.pdf adversarial Robustness • Changshui Zhang

Improving the Adversarial Robustness • Andrew Slavin

and Interpretability of https://arxiv.org/pdf Ross 206 2017 X X Deep Neural Network /1711.09404.pdf • Finale Doshi-

by Regularizing their Velez

input Gradients

• Jan Hendrik

Metzen On detecting https://arxiv.org/pdf 207 2017 • Tim Genewein X X Adversarial perturbation /1702.04267.pdf • Volker Fischer

• Bastian Bischoff

• Nicolas Papernot https://ieeexplore.i SoK: Security and • Patrick McDaniel eee.org/stamp/sta 208 privacy in machine 2017 • Arunesh Sinha mp.jsp?tp=&arnum learning • Michael P. ber=8406613 Wellman

• Qinglong Wang

Random Feature • Wenbo Guo

Nullification for https://arxiv.org/pdf • Kaixuan Zhang 209 2016 X X Adversary Resistant /1610.01239v1.pdf • Xinyu Xing

Deep Architecture • C. Lee Giles

• Xue Liu

https://people.eecs • Marco Barreno .berkeley.edu/~adj/ The security of machine • Blaine Nelson 210 publications/paper- 2010 X X learning • Anthony D. files/SecML- Joseph MLJ2010.pdf

65 SECURING MACHINE LEARNING ALGORITHMS December 2021 d is m e ing e ing is ng rv Type of data rn rn rni orc a ingested e e Lea inf l uperv Lea nsupe d e nt S U e R e dat n o Title Source ti Author a ta n Index c s a o li e D ti ng ri ion ng ion ion t d s a ri ub ge e s P a ic e ns rdi ideo s if e Tex e ture s duct wa V Im gre s lust e m e im e Ti R la C D R R truc C S

• Florian Tramer

• Alexey Kurakin Ensemble adversarial https://arxiv.org/pdf • Nicolas Papernot 211 training: attacks and 2018 X X /1705.07204.pdf • Ian Goodfellow defenses • Dan Boneh

• Patrick McDaniel

Improving the • Stephen Zheng

robustness of Deep https://arxiv.org/pdf • Yang Song 212 2016 X X Neural Networks via /1604.04326.pdf • Thomas Leung

Stability training • Ian Goodfellow

• Nicolas Papernot

Distillation as a defense • Patrick McDaniel

to adversarial https://arxiv.org/pdf • Xi Wu 213 2016 X X perturbations against /1511.04508.pdf • Somesh Jha

deep neural networks • Ananthram

Swami

• Nicolas Papernot

• Shuang Song

• Ilya Mironov Scalable Private https://arxiv.org/pdf 214 2016 • Ananth X X Learning with Pate /1802.08908.pdf Raghunathan • Kunal Talwar

• Ulfar Erlingsson

https://seclab.stanf

Gradient masking in ord.edu/AdvML201 215 2017 • Nicolas Papernot machine learning 7/slides/17-09-aro-

aml.pdf

The Quest for https://hal.archive Statistical Significance: s-ouvertes.fr/hal- 216 Ignorance, Bias and 2018 •Joshua Abah 01758493/docume Malpractice of nt Research Practitioners

The measure and https://arxiv.org/pdf • Sam Corbettmismeasure of fairness 217 / 2018 Davies A critical review of fair 1808.00023.pdf • Sharad Goel machine learning

Adversarial Policy • Xian Wu https://www.usenix Training Against Deep • Wenbo Guo 218 .org/system/files/se 2021 X X X Reinforcement • Hua Wei c21-wu-xian.pdf Learning • Xinyu Xing

https://link.springer

Early stopping - But .com/ 219 2012 • Lutz Prechelt When? book/10.1007%2F

978-3-642-35289-8

A pitfal and solution in https://icml.cc/Conf

multi-class feature erences 220 2004 • George Forman X X selection for text /2004/proceedings/

classification papers/107.pdf

• Kamalika Near-Optimal Chaudhuri Algorithms for https://arxiv.org/ 221 2013 • Anand D. X X Differentially-Private pdf/1207.2812.pdf Sarwate principal components • Kaushik Sinha

66 SECURING MACHINE LEARNING ALGORITHMS December 2021 d is m e ing e ing is ng rv Type of data rn rn rni orc a ingested e e Lea inf l uperv Lea nsupe d e nt S U e R e dat n o Title Source ti Author a ta n Index c s a o li e D ti ng ri ion ng ion ion t d s a ri ub ge e s P a ic e ns rdi ideo s if e Tex e ture s duct wa V Im gre s lust e m e im e Ti R la C D R R truc C S

Differentially private model selection via http://proceeding • Adam Smith

222 Stability argument and s.mlr.press/v30/Gu 2013 • Abhradeep X

the robustness of the ha13.pdf Thakurta

Lasso

• Shehzeen

Hussain WaveGuard: https://www.usenix • Paarth Neekhara Understanding and 223 .org/system/files/se 2021 • Shlomo Dubnov X X Mitigating Audio c21-hussain.pdf • Julian McAuley, Adversarial Examples • Farinaz

Koushanfar

http://researchers.li lle.inria.fr/abellet/te Privacy Preserving 224 aching/private_ma 2020 •Aurélien Bellet Machine Learning chine_learning_co

urse.html

Interpretability of https://www.wavest Machine Learning one.com/app/uploa What are the ds/2019/09/Wavest •Alexandre Vérine 225 2019 challenges in the era of one_Interpretability •Stephan Mir

automated decision- _machine_learning Making Progresses? .pdf

https://www.analyti

6 Python Libraries to csvidhya.com/blog/ interpret Machine 2020/03/6-python- 226 2020 •Purva Huilgol Learning Models and libraries-interpret- Build Trust machine-learning-

models/

• Cristoph Molnar

• Gunnar König • Julia Herbinger

• Timo Freiesleben • Susanne Dandl Pitfalls to avoid when https://arxiv.org/pdf • Christian 227 Interpreting Machine / 2020 A.Scholbeck Learning Models 2007.04131.pdf • Giuseppe

Casalicchio • Moritz Grosse-

Wentrup • Bernd Bischl

https://towardsdata science.com/7-

7 steps to ensure and steps-to-ensure- 228 2019 • Stéphanie Shen sustain Data quality and-sustain-data-

quality- 3c0040591366

67 SECURING MACHINE LEARNING ALGORITHMS December 2021 d is m e ing e ing is ng rv Type of data rn rn rni orc a ingested e e Lea inf l uperv Lea nsupe d e nt S U e R e dat n o Title Source ti Author a ta n Index c s a o li e D ti ng ri ion ng ion ion t d s a ri ub ge e s P a ic e ns rdi ideo s if e Tex e ture s duct wa V Im gre s lust e m e im e Ti R la C D R R truc C S

https://techcrunch. com/2018/11/06/3ways-to-avoidbias-in-machinelearning/?guccount er=1&guce_referre r=aHR0cHM6Ly93 d3cuZ29vZ2xlLmN vbS8&guce_referr er_sig=AQAAAIfXh Three ways to avoid VIDfTYv80Vxw4JV 229 bias in machine aKFZyt_3_2DTapB 2018 • Vince Lynch learning QQjW8C1vzjPTQq ViKdAE5O- BV1Q5J5waGMcY o4yu2R4QBOr9H1 7RpApdX9vlXDUlo _MS28Q4GD8qCX qhogX534lJcR7DP rzwANTY8WPnJX 5GXVmytlHxM0ZK 9Ym2ANzKGSnae 6QgH

https://www.kdnug 5 Ways to deal with the gets.com/2019/06/ •Alexandre 230 lack of data in Machine 5-ways-lack-d 2019 Gonfalonieri Learning ata-machinelearning.html

https://towardsdata science.com/artifici Artificial Intelligence is al-intellige Crucial TO the Success nce-is-crucial-to- 231 2019 • Amit Makhija of Your Business and the-success-ofhere is why your-businesslearn-whyd5b96fa3564d

https://www.kaggle 7 Simple Techniques to •Devendra Kumar 232 .com/learn- 2020 Prevent Overfitting Yadav forum/157623

68 -N -EN -153 -21 -06 TP ABOUT ENIS A The European Union Agency for Cybersecurity, ENISA, is the Union’s agency dedicated to achieving a high common level of cybersecurity across Europe. Established in 2004 and strengthened by the EU Cybersecurity Act, the European Union Agency for Cybersecurity contributes to EU cyber policy, enhances the trustworthiness of ICT products, services and processes with cybersecurity certification schemes, cooperates with Member States and EU bodies, and helps Europe prepare for the cyber challenges of tomorrow. Through knowledge sharing, capacity building and awareness raising, the Agency works together with its key stakeholders to strengthen trust in the connected economy, to boost resilience of the Union’s infrastructure, and, ultimately, to keep Europe’s society and citizens digitally secure. More information about ENISA and its work can be found here: www.enisa.europa.eu. ISBN: 978-92-9204-543-2 DOI: 10.2824/874249