Big Data Threat Landscape

Big Data Threat Landscape and Good Practice Guide

JANUARY 2016 www.enisa.europa.eu European Union Agency For Network And Information Security About ENISA

The European Union Agency for Network and Information Security (ENISA) is a centre of network and information security expertise for the EU, its member states, the private sector and Europe’s citizens. ENISA works with these groups to develop advice and recommendations on good practice in information security. It assists EU member states in implementing relevant EU legislation and works to improve the resilience of Europe’s critical information infrastructure and networks. ENISA seeks to enhance existing expertise in EU member states by supporting the development of cross-border communities committed to improving network and information security throughout the EU. More information about ENISA and its work can be found at www.enisa.europa.eu.

Authors

This report was authored by a group of experts, and edited by ENISA:

Authors: Ernesto Damiani (CINI), Claudio Agostino Ardagna (CINI), Francesco Zavatarelli (CINI), Evangelos Rekleitis (ENISA), Louis Marinos (ENISA).

Editor: Evangelos Rekleitis (ENISA)

Contact

For contacting the authors please use isdp@enisa.europa.eu

For media enquiries about this paper, please use press@enisa.europa.eu.

Acknowledgements

We would like to express our gratitude to the working group supported ENISA work for their valuable contribution to this report, including: W.M.P van der Aalst, Eindhoven University of Technology, The Netherlands and Panayotis Kikiras, AGT Group, Germany. Acknowledgement should also be given to ENISA colleagues who helped in this project.

Legal notice

Notice must be taken that this publication represents the views and interpretations of the authors and editors, unless stated otherwise. This publication should not be construed to be a legal action of ENISA or the ENISA bodies unless adopted pursuant to the Regulation (EU) No 526/2013. This publication does not necessarily represent state-of the-art and ENISA may update it from time to time.

Third-party sources are quoted as appropriate. ENISA is not responsible for the content of the external sources including external websites referenced in this publication.

This publication is intended for information purposes only. It must be accessible free of charge. Neither ENISA nor any person acting on its behalf is responsible for the use that might be made of the information contained in this publication.

Copyright Notice

© European Union Agency for Network and Information Security (ENISA), 2015 Reproduction is authorised provided the source is acknowledged.

Table of Contents

Executive Summary 5

1. Introduction 7

Policy context 8

Scope 8

Target audience 8

Methodology 9

Structure of this document 9

2. Big Data Environments 10

Big Data architecture 10

3. Big Data assets 12

Big Data asset taxonomy 12

Big Data asset categories 14

4. Big Data threats 17

ENISA threat taxonomy 17

Mapping threats to Big Data assets 19

4.2.1 Threat Group: Unintentional damage / loss of information or IT assets 20 4.2.2 Threat Group: Eavesdropping, Interception and Hijacking 23 4.2.3 Threat Group: Nefarious Activity/Abuse 23 4.2.4 Threat Group: Legal 27 4.2.5 Threat Group: Organisational threats 29

5. Threats agents 30

6. Good practices 33

7. Gap analysis 43

Annex A: Full list of Big Data taxonomy 48

Annex B: Full Big Data asset taxonomy structure 53

Annex C: Full list of threats affecting Big Data 54

Annex D: Full Big Data threat taxonomy structure 56

Annex E: Big Data analytics for security 57

Annex F: Summary of threat taxonomies 61

Executive Summary

The term Big Data is often used loosely to designate the palette of algorithms, technology and systems employed for collecting data of unprecedented volume and variety, and extracting value from them by massively parallel computation of advanced analytics. The sources of Big Data are many and diverse. Distributed multimedia sensors on the Internet of Things, mobile telecommunication devices and networks, distributed business processes, and Web-based applications are all candidate data providers/generators. As Big Data usage has increased over the years, the various algorithms, technologies, and systems are gradually reaching a level of development and maturity suitable for widespread adoption.

Experience has shown that Big Data applications can provide a dramatic increase in the efficiency and 1,2 effectiveness of decision-making in complex organizations and communities . It is expected that it will constitute an important part of a thriving data-driven economy, with applications ranging from science and business to military and intelligence . However, besides its benefits or in some cases because of them, Big Data also bears a number of security risks. Big Data systems are increasingly becoming attack targets by threat agents, and more and more elaborate and specialized attacks will be devised to exploit vulnerabilities and weaknesses.

This Threat Landscape and Good Practice Guide for Big Data provides an overview of the current state of security in the Big Data area. In particular, it identifies Big Data assets, analyses exposure of these assets to threats, lists threat agents, takes into account published vulnerabilities and risks, and points to emerging good practices and new researches in the field. To this aim, ongoing community-driven efforts and publicly available information have been taken into account.

The study analyses threats to all identified Big Data asset classes. Highlights include:

 Big Data threats include, but are not limited to, threats to ordinary data. The high level of replication in Big Data storage and the frequency of outsourcing Big Data computations introduce new types of breach, leakage and degradation threats that are Big Data-specific.  Big Data is having significant privacy and data protection impacts. The creation of links at data collection (a.k.a. “ingestion”) time is a key requirement for parallelization – and therefore performance - of Big Data analytics, but the additional information it creates may increase the impact of data leakages and breaches.  The interests of different asset owners (e.g., data owners, data transformers, computation and storage service providers) in the Big Data area are not necessarily aligned and may even be in conflict. This creates a complex ecosystem where security countermeasures must be carefully planned and executed.  As in many other areas of ICT, starting to apply basic privacy and security best practices would significantly decrease overall privacy and security risks in the Big Data area. At this still early stage of this emerging paradigm, embracing the Security-by-default principle can prove to be both highly

practical and beneficial; as opposed to the cost and effort required to provide ad hoc solutions later on.

This guide finally provides a gap analysis presenting a comparison between identified Big Data threats and identified Big Data countermeasures. In this context, the lack of current Big Data countermeasures and pressing needs in the development of next-generation countermeasures are discussed. In particular, the question arises of the trend of current countermeasures of adapting existing solutions against traditional data threats to the Big Data environments, mostly focusing on the volume of the data. This practice mainly targets scalability issues and clearly does not fit the Big Data peculiarities (5V- Volume, Variety, Value…) resulting in partial and ineffective approaches. A set of recommendations for next-generation countermeasures concludes the guide. Among these recommendations, we remark i) to depart from current approaches for traditional data, defining Big Data-specific solutions, ii) to identify gaps and needs for current standards, planning the definition of standardization activities, iii) to focus on training of specialized professionals, iv) to define tools for security and privacy protection of Big Data environments, v) to clearly identify Big Data assets simplifying the selection of solutions mitigating risks and threats.

Aligning to its mandate ENISA published two more reports studying the impact of Big Data in the more specialized areas of data protection and privacy (“Privacy by design in big data” ) and critical infrastructures .

1. Introduction

In this reports ENISA elaborates on threats related to Big Data, a technology that has gained much traction in recent years and is expected to play a significant role affecting various aspects of our sociatey, ranging from health, food security, climate and resource efficiency to energy, intelligent transport systems and smart cities . The European Commission has acknowledged the potential impact of Big Data in a “thriving data-driven economy” by outlining a strategy on Big Data . According to estimates, the value of just personal data of EU citizens has “the potential to grow to nearly €1 trillion annually by 2020” (sic). It is thus conceivable that data will continue to be a significant economic drive. But also in science and research Large and nowadays Big Data continue to proliferate and many agencies and institutions in Europe and around the globe have or are planning to launch Big Data projects to facilitate scientific data analysis and exploitation . Big Data technologies are also being used in military applications; such as fighting terrorism; assisting in combat; gathering and analysing intelligence from heterogeneous sources, including battlefield data and open sources . In addition, many existing data intensive environments have in recent years adopted a Big Data approach. To name just a few examples, Facebook is thought to store one of the biggest datasets worldwide, storing more than 300 petabytes of both structured and unstructured data; Twitter recently decided to tap directly into its own raw data using Big Data analytics and the world’s telecommunications capacity was already by 2007 near 65 Exabytes (without signs of this trend declining) ; straining existing storage and analytic processes and technologies.

Given that Big Data approaches make use of extremely novel and high tech ICT systems, with little time to mature against cyber-attacks it is not suprising that attacks are showing an increased trend in both number, sophistication and impact. But because of the loose use of definitions and the unwillingness of affected organizations to disclose attack data, accurate estimates are not easy to come up with. Additionally, as more and more businesses and organizations venture into the Big Data field, attackers will have more incentives to develop specialized attacks against Big Data. Somewhat paradoxically, Big Data approaches can also be used as a powerful tool to combat cyber threats by offering security professionals valuable insigts in threats and incident management.

Being an ENISA deliverable in the area of Threat Landscape, this report constitutes a detailed threat assessment in the area of Big Data, based on input from the ENISA Threat Landscape activities. The rationale behind this piece of work is to “deepen” the generic threat assessment by taking into account the specificities of Big Data.

Policy context

Threat analysis and emerging trends in cyber security are an important topic in the Cyber Security Strategy 14 15 for the EU . Moreover, the new ENISA regulation highlights the need of analysing current and emerging risks and dictates that “the Agency, in cooperation with Member States and, as appropriate, with statistical bodies and others, collects relevant information”. More specifically, it is stated that it should “enable effective responses to current and emerging network and information security risks and threats”.

To this end the ENISA Work Programme 2015 included this study on “Big Data Threat Landscape and Good Practice Guide” as one of this year’s deliverables (“WPK1.1-D2: Risk Assessment on two emerging technology/application areas” that focuses on Big Data).

The report aims to identify emerging trends in cyber-threats and to provide a concise state of the art analysis of the cyber threat and security issues of Big Data; consolidating existing and open literature and available information, and contributing to a cyber security public and private initiatives by addressing industry concerns in the area..

Scope

This report contributes to the definition of a threat landscape, by providing an overview of current and emerging threats applicable to Big Data technologies, and their associated trends. Several Big Data definitions exist in the literature and the area is constantly being shaped by advantages in methods, tools, and new applications, thus it is not possible to take into consideration all Big Data systems. The research done focuses on assets, threats and controls applicable to prominent, important and/or widely used Big Data systems.

The goal is to deepen our understanding of the threats that affect Big Data and to provide good practices and recommendations for those threats that are considered important or emerging.

Target audience

It is expected that this report will be useful for performing detailed Risk Assessments (RA) and Risk Management (RM) by Big Data providers and operators according to their particular needs and for Big Data consumers in drafting their SLAs. The asset and threat taxonomies presented here are to be expanded by asset owners, based on the particular Big Data system instantiation at hand, before being used as input to RA/RM and cyber threat exposure analysis.

Moreover, the presented Big Data threat landscape will be of use to policy-makers for understanding the current state of threats and respective mitigation practices and measures in the area.

Further, the extensive research of relevant existing literature in Big Data security and threat research means this study will be of particular interest to researchers and institutions working in the field.

Methodology

This study and its outcome are based on desk research and review of conference papers, articles, technical blogs and a variety of other open sources of information relevant to Big Data. This report identifies the majority of sources consulted; the details of all documentary sources consulted during this study are available on request. More than one hundred documentary sources were identified through a number of search methods, including specialist search engines for academic sources and journal articles. The sources collected are all in English.

The overall work went through a three-step process as follows: The first step “Information collection” was about the identification and collection of relevant information, in particular the assets and threats. The second step “Assessment, Guidelines and Gap Analysis” performed an analysis about the collected information to identify current and emerging trends and then elaborated countermeasures in a Big Data scenario. The third step “Good practices definition” was focused on findings, current practices, and needs that formalized the Big Data threat landscape report. A final note, all referenced web resources were last accessed in November 2015.

Structure of this document

The structure of document is as follows: in section 2 we define Big Data and describe an abstract architecture upon which the study is based; in section 3 we present an asset taxonomy for Big Data; in section 4 we identify threats against Big Data, based on the threat taxonomy used by ENISA in “Threat Landscape and Good Practice Guide” reports, and map these threats to Big Data assets; in section 5 we consider which threat agents are more relevant to Bog Data attacks; in section 6 we present a set of recommendations and good practices for Big Data; we conclude in section 7 with a gap analysis.

In addition 6 annexes are provides at the end of the report.

Annex A contains the Big Data asset taxonomy in full depth; including all identified asset groups, asset types, assets and asset details.

Annex B contains the detailed Big Data asset taxonomy diagram.

Annex C contains the Big Data threat taxonomy in full detail; including all identified threat groups/types correlated to threat agents and affected Big Data assets.

Annex D contains the detailed Big Data threat taxonomy diagram.

Annex E contains a concise presentation on how Big Data analytics can assist security professionals in analysing threats and attacks and detecting intrusion and fraud cases.

Annex F contains a summary of existing threat taxonomies, which were used along with ENISA’s threat taxonomy to drive this study.

2. Big Data Environments

The term Big Data describes the vast amount of data in our information-driven world. In a 2001 research report and related lectures, the META Group (now Gartner) defined the data growth challenges and opportunities as being three-dimensional, i.e. increasing Volume (amount of data), Velocity (speed of data in and out), and Variety (range of data types and sources). Gartner, and then the industry, used this "3Vs" model for describing Big Data: "Big data is high volume, high velocity, and/or high variety information assets that require new forms of processing to enable enhanced decision making, insight discovery and process optimization. ”. Additionally, some new Vs have been added by some organizations to further define Big Data: "Veracity" (data authenticity since the quality of captured data can vary greatly and an accurate analysis depends on the veracity of source data), “Variability” (data meaning is often changing, and the data can show inconsistency at times, and this can hamper the process of handling and managing the data effectively) and “Value” (the potential revenue of Big Data). This being a developing field, several other alternative or complenetary definitions have been proposed, in an effort to capture different nuances attributed to Big Data; such as its evolutianary nature: “datasets whose size is beyond the ability of typical database software tools to capture, store, manage, and analyze.” (sic) . Given that the field is still not mature, for the purposes of this report we take into account the different ways Big Data is defined.

While a great scientific opportunity exists with Big Data, this growth is outpacing the technological advances in computational power, storage, analysis and analytics. Furthermore a real concern is arising 4,21, about the security of this massive amount of digital information, the data protection and privacy issues and the protection of the (critical) infrastructure supporting it .

Big Data architecture

The architecture in Error! Reference source not found. is a high-level conceptual model that facilitates the discussion of security requirements in Big Data and introduces the terminology used in this report. It does not represent the system architecture of a specific Big Data system, nor it is tied to any specific vendor products, services, or reference implementation, but rather it is a tool for describing some common Big Data components; i.e. the Big Data environment. In our vision the notion of Big Data architecture can be

detailed into five layers: “Data sources”, “Integration process”, “Data storage”, “Analytics and computing models“, “Presentation”.

The function of each layer is as follows:

The “Data sources” layer consists of disparate data sources, ranging from sensor streaming data, to structured information such as relational databases, and to any sort of unstructured and semi-structured data.

The “Integration process” layer is concerned with acquiring data and integrating the datasets into a unified form with the necessary data pre-processing operations.

The “Data storage” layer consists of a pool of resources such as distributed file systems, RDF stores, NoSQL and NewSQL databases, which are suitable for the persistent storage of a large number of datasets.

The “Analytics and computing models” layer encapsulates various data tools, such as Map Reduce, which run over storage resources and include the data management and the programming model . The “Presentation” layer enables the visualisation technologies.

Cloud computing can be deployed as the infrastructure layer for Big Data systems to meet some infrastructure requirements, such as cost-effectiveness, elasticity, and the ability to scale up or down .

3. Big Data assets

Assets can be abstract assets (like processes or reputation), virtual assets (for instance, data), physical assets (cables, a piece of equipment), human resources, money”. An item of our taxonomy is either a description of data itself, or describes assets that generate, process, store or transmit data chunks and, as such, is exposed to cyber-security threats. For information security considerations, this study focuses on assets that are related mainly to information and communication technology (ICT) under the scope of Big Data.

A major source of information for this study is the work made by the NIST Big Data Public Working Group (NBD-PWG) , which is developing consensus on important and fundamental questions related to Big Data. They have produced two draft Volumes (Volume 1 about Definitions and Volume 2 about Taxonomy). Another source of information is the report “Big Data Taxonomy” , issued by Cloud Security Alliance (CSA) Big Data Working Group in September 2014. In that document, CSA proposes a six-dimensional taxonomy for Big Data, pivoted around the nature of the data to be analysed. The objective is to help “navigate the myriad choices in compute and storage infrastructures as well as data analytics techniques” and the proposed structure is mainly intended as a high-level taxonomy for decision makers.

Specifically, most of the terminology used in this report for high level asset types (Data, Infrastructure, Analytics, and Security and Privacy techniques) comes, with some small modifications, from the CSA taxonomy; where our term Infrastructure also comprises of the other two CSA main categories; viz. Compute Infrastructure and Storage Infrastructure. Another high-level type, Roles, comprises human resources and other related assets, as in previous ENISA thematic studies .

Big Data asset taxonomy

Error! Reference source not found. gives an overview of the Big Data assets structure into relevant categories according to their use. The full list of identified the Big Data assets is given in Annex A.

Big Data asset categories

With the following list we attempt to identify some of the known Big Data valuable assets in a hierarchical manner. The first and second level category items (asset group and asset type) can be thought of as intuitively clear, but we give a brief description of them nevertheless. Numbers in brackets refer to Error! Reference source not found.. The full taxonomy, with further levels in the taxonomy such assets and asset details, is presented in Annex A .

Data – This is the core category of the Big Data taxonomy and includes:

Metadata, i.e. schemas, indexes, data dictionaries and stream grammars’ data (which often but not necessarily come together with stream data).

Structured data, i.e. database records structured according to a data model, as for example a relational or hierarchical schema; structured identification data, as for example users’ profiles and preferences; linked open data; inferences and re-linking data structured according to standard formats.

Semi-structured and unstructured data, for example logs, messages and web (un)formatted data (Web and Wiki pages, e-mail messages, SMSs, tweets, posts, blogs, etc.), files and documents (e.g. PDF files and Office suite data in Repositories and File Servers), multimedia data (photos, videos, maps, etc.), and other non-textual material besides multi-media (medical data, bio-science data and raw satellite data before radiometric/geometric processing, etc.).

Streaming data, i.e. single-medium streaming (for example in-motion sensor data) and multimedia streaming (remote sensing data streams, etc.).

Volatile data, i.e. data that are either in motion or temporarily stored, as, for example, network routing data or data in devices’ random access memory.

Infrastructure – The term infrastructure comprises software, hardware resources denoting both physical and virtualized devices, the basic computing infrastructure with its batch and streaming processes and the storage infrastructure with all sort of database management systems, ranging from old-style relational databases to NoSQL or NewSQL, as well as Semantic Web tools. Specifically, the Infrastructure first level category includes:

Software, including operating systems, device drivers, firmware, server-side software packages (as Web and Application Server software) and applications. Applications sub-category includes software implementation as back-end services and all sorts of functionalities that utilize other assets in order to fulfil a defined task, such as for example asset management tools, requirements gathering applications, billing services and tools to monitor performances and SLAs.

Hardware (physical and virtual), i.e. servers (physical devices and hardware nodes, all the virtualized hardware, including virtual Data Centres with their management consoles and virtual machines, as well as the physical hardware supporting their provisioning), clients, network devices (for example, physical switches, virtual switches and virtual distributed switches, etc.), media and storage devices (the various

types of disk storage, etc.), data gathering devices (sensors, remote platforms as airborne platforms or drones, etc.), Human Interface Devices (HID) and mobile devices.

Computing Infrastructure Models, this category includes paradigms of abstract processing architectures, on whether the processing can be done in batch mode, for example MapReduce; on real-time/near realtime streaming data, as for example Sketch or Hash-based models; or follow a unified approach supporting both, as for example Cloud Dataflow.

Storage Infrastructure Models, this category includes paradigms of abstract storage architectures, including Big Files and triples-based models.

Big Data Analytics – This category includes models which define protocols and algorithms for Big Data analysis, like procedures, models, algorithms definitions down to the source code, and analysis’ results. The category includes:

Data analytics algorithms and procedures, which include algorithm source code with their set-up parameters, configuration and thresholds, metrics, the model definitions, advanced techniques that streamline the data preparation stage of the analytical process.

Analytical results, either in textual or in graphical mode (e.g. spatial layouts, abstract, interactive and real time visualizations).

Security and Privacy techniques – This category name includes the term “techniques” to remark that the security-related assets it includes are the ones of interest to attackers and therefore more subject to unauthorized disclosure and leakage, as for example security best practice documents, cryptography algorithms and methods, information about the access control model used, etc. The category includes the following sub-categories:

Infrastructure Security, i.e. the first aspect of a Big Data ecosystem security, which deals with how to secure the distributed computation systems and the data stores, with security Best Practices and policy set-ups.

Data Management, i.e. documents and techniques about how to secure Data Storage and Logs, and documentation about granular audits and data life cycle (Data provenance).

Integrity and reactive security, which deals with all the practices, techniques, and documents related to End Point validation and filtering and the monitoring of real-time security, including incident handling and information forensics.

Data Privacy, i.e. all the techniques put in place to protect privacy as it is requested by law, for example cryptographic methods and access control.

Roles - This terminology for this category was introduced by the NIST Big Data Public Working Group and includes:

Data provider, such as enterprises, organizations, public agencies, academia, network operators and endusers.

Data consumer, partly overlapping the previous category, but from a different scope, and including enterprises, organizations, public agencies, academia and end-users.

Operational roles, i.e. system orchestrators (business leader, data scientists, architects, etc.), Big Data application providers (application and platform specialists), Big Data framework providers (Cloud provider personnel), security and privacy specialists, technical management (in-house staff, etc.).

We remark that leaving the taxonomy unbalanced (some sub trees, like those rooted in Data and Infrastructure are deeper than others) is a deliberate choice. Indeed some leaf subcategories of our taxonomy, such as Models definitions, could be used to integrate external taxonomies designed for different reasons, such as data science ones .

Another remark is that most of the categories and sub-categories could be related to data, rather than Big Data. For example, relational databases are a very typical and common resource in every enterprise infrastructure, not necessarily storing big data volumes. Even when relational databases have big volume size, they are often manageable through traditional hardware clusters, appliances and software tools. Another example is applications’ random-access memory (featured in volatile data category), i.e. the data that is temporarily in memory due to processing operations. This memory is often (though not invariably, as witnessed by the success of in-memory processing systems) not large, compared to massive data sizes of in-memory databases.

Nevertheless, we included these assets in our taxonomy for completeness of information. Data stored in relational databases, often very valuable for data owners, might be used in some cases as data source for analytics, while leakage of RAM content could compromise login credentials and cryptographic keys, paving the way to dangerous attacks to Big Data.

The presented asset taxonomy should only be considered as a snapshot of the complex range of Big Data assets and could as such not be exhaustive.

4. Big Data threats

ENISA threat taxonomy

In this section, we introduce the major characteristics of the ENISA threat taxonomy. The ENISA taxonomy is a comprehensive one, with a special focus on cyber-security threats; i.e., threats applying to information and communication technology assets. Additional non-ICT-stemming threats have been considered to cover threats to physical assets and also both natural disasters [not directly triggered by humans] and environmental disasters directly caused by human.

The threat taxonomy has been developed by the ENISA Threat Landscape (ETL) Group and is a consolidation of threats previously considered in other thematic reports and extensive research. The taxonomy includes threats applicable to the Big Data assets and only these are depicted in figure 4-1. In the following subsection, threats specific to Big Data that were identified through extensive literature that have been assigned to the relevant categories defined in ENISA’s Threat Taxonomy are mapped to the previously discussed Big Data Asset Taxonomy.

Mapping threats to Big Data assets

In this section, we discuss the threats that can be mapped to the Big Data asset taxonomy presented in the previous chapter. This analysis is based on an extensive review of actual threat incidents and attacks to Big Data presented in articles, technical blogs, conference papers, as well as online surveys for gathering supplemental information. Our review was driven by the ENISA generic threat taxonomy presented in the previous section.

In general terms, threats, such as network outage or malfunctions of the supporting infrastructure, may heavily affect Big Data. In fact, since a Big Data has millions of pieces of data and each piece may be located in a separate physical location, this architecture leads to a heavier reliance on the interconnections between servers. Past ENISA thematic reports have dealt in depth with threats such as outages and malfunctions, which affect network communication links . For this reason, in this report, we don’t take these threats into account. Also, we chose not to dwell on physical attacks (deliberate and intentional), natural and environmental disasters, and failures / malfunction (e.g. malfunction of the ICT supporting infrastructure), since their effects are strongly mitigated by the intrinsic redundancy of Big Data, though Big Data owners deploying their systems in private clouds or other on-premise infrastructure should take these attacks under serious consideration .

In general, a threat is “any circumstance or event with the potential to adversely impact an asset through unauthorized access, destruction, disclosure, modification of data, and/or denial of service” . Given the definition we gave of Big Data (Volume, Velocity, Variety, Veracity, Variability and Value), a threat to a Big Data asset can be considered as any circumstance or event that affects, often simultaneously, big volumes of data and/or data in various sources and of various types and/or data of great value.

We also identify two different sub-categories of threats: “Big Data Breach” and “Big Data Leak” , orthogonal to the used threat taxonomy. A breach occurs when “a digital information asset is stolen by attackers by breaking into the ICT systems or networks where it is held/transported” . We can define “Big Data Breach” as the theft of a Big Data asset executed by breaking into the ICT infrastructure. A Big Data Leak on the other hand, can be defined as the (total or partial) disclosure of a Big Data asset at a certain stage of its lifecycle. A Big Data Leak can happen for example in inadequate design, improper software adaptation or when a business process fails. In terms of the attacker model, a Big Data Breach requires pro-active hostile behaviour (the break-in), while a Big Data Leak can be exploited even by honest-butcurious attackers.

4.2.1 Threat Group: Unintentional damage / loss of information or IT assets

This group includes Information leakage or sharing due to human errors, unintentional intervention or erroneous use of administration of systems (misconfiguration), loss of devices.

Threat: Information leakage/sharing due to human error

Accidental threats are those not intentionally posed by humans. They are due to misconfiguration, skillbased slips and clerical errors (for example pressing the wrong button), misapplication of valid rules (poor patch management, use of default user names and passwords or easy-to-guess passwords), and

knowledge-based mistakes (software upgrades and crashes, integration problems, procedural flaws) .

Information leakage due to misconfiguration can be a common problem: according to a recent study , erroneous system administration setups led to numerous weaknesses in four different Big Data technologies; viz. Redis, MongoDB, Memcache and ElasticSearch. According to the same study most of these new products “are not meant to be exposed to the Internet. […] These technologies' default settings tend to have no configuration for authentication, encryption, authorization or any other type of security controls that we take for granted. Some of them don't even have a built-in access control.”

Furthermore, in the past, there have been reported incidents of inappropriate sharing of files containing possible sensitive and confidential information, which affected even very popular online services like

Dropbox . This is also confirmed by many surveys .

The assets targeted by these threats include asset group “Data”, and asset “Applications and Back-end services” (such as for example “Billing services”).

Threat: Leaks of data via Web applications (unsecure APIs)

Various sources claim that Big Data is often built with little security . New software components are usually provided with service-level authorization, but few utilities are available to protect core features and application interfaces (APIs). Since Big Data applications are built on web services models, APIs may be vulnerable to well-known attacks, such as the Open Web Application Security Project (OWASP) Top Ten list , with few facilities for countering common web threats.

The security software vendor Computer Associate (CA) and other sources report data breaches, due to insecure APIs, in many industries, especially in social networks, mobile photo-sharing and video-sharing services, as Facebook, Yahoo and Snapchat.

For example, a threat of this category may consist in injection attacks to Semantic Web technologies through SPARQL code injection . Security flaws are rather common in new Big Data languages like SPARQL, RDQL (both are read-only query languages) and SPARUL (or SPARQL/Update, which has modification capabilities). The use of these new query languages introduces vulnerabilities already found in a bad use of old-style query languages, since attacks like SQL, LDAP and XPath injection are already well known and still dangerous . Libraries of these new languages provide tools to validate user input and minimize the risk. However, “main ontology query language libraries still do not provide any mechanism to avoid code injection” and without these mechanisms, attackers’ arsenal might get enhanced with SPARQL, RDQL and SPARQL injections . Other new Big Data software products, as for example Hive, MongoDB and CouchDB, also suffer from traditional threats such as code execution and remote SQL injection .

The assets targeted by these threats belong to group “Data” and asset type “Storage Infrastructure models” (such as “Database management systems (DBS)” and “Semantic Web tools”)

Threat: Inadequate design and planning or incorrect adaptation

Techniques for improving Big Data analytics performance and the fusion of heterogeneous data sources increase the hidden redundancy of data representation, generating ill-protected copies. This challenges traditional techniques to protect confidentiality and the effect of redundancy must be taken into account. As already stated, Big Data redundancy can be seen as a threat mitigation technique for physical attacks, disasters and outages , however in some cases it signals a system weakness, being a risk booster for Big Data leaks. In other words, if our Big Data storage replicates data records ten times and distributes the copies to ten storage nodes for some reason (e.g., to speed up the analytics pipeline), the ten nodes may end up with different levels of security robustness (e.g., different security software versions) and this will increase the probability of data disclosure and data leaks. This can be considered a specific weakness of Big Data designs.

On the other hand we can also note that even the redundancy and the replication that are necessary features to enhance Big Data functionality, are not always a failsafe against data loss. For example Hadoop, the well-known framework for Big Data processing, replicates data three times by default, since this protects against inevitable failures of commodity hardware. However, a corrupted application could destroy all data replications . Also, recent studies put forward the idea that Hadoop redundancy could even be a non-linear risk booster for Big Data leakages .

Even the design of the Hadoop Distributed File System (HDFS) signals problems as reported by literature . HDFS is the basis of many Big Data large-scale storage systems and is used by social networks. HDFS clients perform file system metadata operations through a single server known as the Namenode, and send and retrieve file system data by communicating with a pool of nodes. The loss of a single node should never be fatal, but the loss of the Namenode cannot be tolerated . Big social networks, such as Facebook, suffered this problem and took countermeasures against the threat (Hadoop installed at Facebook includes one of the largest single HDFS cluster, more than 100 PB physical disk space in a single HDFS file system).

One more threat related to the design is the lack of scalability of some tools. For example NIST reports that original digital rights management (DRM) techniques were not built to scale and to meet demands for the forecasted use of the data and “DRM can fail to operate in environments with Big Data characteristics— 58 59 especially velocity and aggregated volume” .

The assets that are targeted by these threats belong to asset groups “Data” and “Big Data analytics”, and to asset types “Software”, “Computing Infrastructure models“ and “Storage Infrastructure models”.

4.2.2 Threat Group: Eavesdropping, Interception and Hijacking

This group includes threats that rely on alteration/manipulation of the communications between two parties. These attacks do not require installing additional tools or software on the victims’ infrastructure.

Threat: Interception of information

A common issue that affects any ICT infrastructure is when offenders can intercept communications between nodes by targeting the communication links. Various sources claim that inter-node communication with new Big Data tools is often unsecured , that it is not difficult to hijack a user session or gain unauthorized access to services in social networks as Facebook and Twitter , and that there is evidence of flaws in communication protocols .

Big Data software distributions (for example Hadoop, Cassandra, MongoDB , Couchbase) rarely have the protocols that ensure data confidentiality and integrity between communicating applications (e.g., TLS and SSL) enabled by default or configured properly (e.g., changing default passwords).

The assets targeted by this threat belong to asset groups “Data” and “Roles”, and to asset “Applications and Back-end services”.

4.2.3 Threat Group: Nefarious Activity/Abuse

This group includes threats coming from nefarious activities. Unlike the previous group, these threats (often) require the attacker to perform some actions altering the victims’ ICT infrastructure; usually with the use of specific tools and software.

Threat: Identity fraud

Big Data systems store and manage credentials for accessing personal data and financial accounts with information such as credit card numbers and payment and billing details, which are targets for cyber criminals. Big Data systems also store profiling data that can describe user behaviour, preferences, habits, travel, media consumption at a high degree of detail, and may help attackers in more elaborate forms of impersonation fraud, creating big opportunities for identity thieves .

Since most Big Data systems are built on top of cloud infrastructure, a threat to users’ identity is, for example, when the control of a system interface, in either a Big Data system based on a large public cloud or in a widely used private cloud, gets lost . A successful attack on a console grants the attacker complete power over the victim's account, including all the stored data. The control interfaces could be initially compromised via novel signature wrapping and advanced XSS techniques, then privilege escalation may lead to identity fraud . While in traditional information systems the loss of control of a console interface could cause limited information leakage, in Big Data the effect is amplified and the impact is more severe.

Social engineering is not a new issue, but as social networking becomes important both for home users and businesses, attacks often involve social engineering. Attackers have been abusing social networks since they first came online. For example, XSS vulnerabilities on Twitter have been used to push malicious and fake tweets, while Internet malware has emerged on Facebook as a means of promoting malicious profiles .

The assets targeted by these threats are “Personal identifiable information”, “Applications and Back end services” (such as, for example, “Billing services”) and “Servers”.

Threat: Denial of service

Big Data components can be threaten by traditional denial of service (DoS) and distributed denial of service (DDoS) attacks. For example, such attacks may remove Big Data components from the network and then exploit its vulnerabilities or an attacker could exhaust the limited resources in a Hadoop cluster, leading to a significant decrease of system performance and causing the loss of the targeted resource to other cloud users . But, at the same time, countering mechanisms have been developed for/using Big

Data systems. For example administrators of Hadoop infrastructure can deploy specialized components to track DDOS attacks .

In the past this kind of attacks has led to some service outages for Amazon distributed storage, through elevated levels of authenticated requests and account validation . Furthermore, as already stated, also specific attacks against social networks such as Facebook have been mounted, exploiting some weaknesses of the Hadoop Distributed File system, for example the Namenode single server .

Assets targeted by this threat include the asset “Servers” (viz. Virtualized Data Centre”, “Physical Machine” and “Virtual Machine”) and the asset “Network”.

Threat: Malicious code / software / activity

These very generic threats affect almost all the ICT components of an infrastructure. Examples of these threats are: i) exploit kits, which allow virus and malware infections, ii) worms, which may be distributed by using the network to send copies to other nodes, iii) Trojans, which are pieces of malware that facilitate unauthorized access to a computer system, iv) backdoors and trapdoors, which are undocumented entry points into a computer program, generally inserted by a programmer to allow remote access to the program, v) service spoofing, which is an attack in which the adversary successfully masquerades as another by falsifying data and thereby gaining an illegitimate advantage , vi) web application attacks and injection attacks through code injection –examples of exploiting this threats to mount more elaborate attacks have already been discussed –.

After the deployment of the code, the attacker may manipulate infected devices. In Big Data, malware infected nodes may send targeted commands to other servers and disturb or manipulate their operations, worms may replicate themselves sending copies to other nodes and affect the behaviour of all components connected to the network. There is always the possibility that vendors of Big Data tools, or somebody else in the software chain, may have installed firmware with backdoors or some hidden functionality to facilitate access to the devices, in particular in the context of very new technologies such as NoSQL and NewSQL .

An example of hacking Big Data through a malicious code attack is reported in literature as faulty results of the Hadoop logging data system. System administrators use Hadoop server logs to identify potential attacks. A demo of this hack requires that a service, called Flume, streams logs into a SQL based Hadoop data store (Hcatalog). In this scenario, an attacker runs a malicious script and alters the results by

modifying the log data before Flume can stream them into Hcatalog . The logs can be corrupted even when Hadoop services seem to be working as expected.

Malicious software can be a threat also in distributed programming frameworks, which use parallel computation, and may have untrusted components. For example, MapReduce computational framework splits the input file into multiple chunks: in the first phase a mapper reads the data, performs computation, and outputs key/value pairs. In the second phase, a reducer works on these pairs and outputs the result. A key issue is how to secure the mappers , since untrusted mappers alter results. With large data sets, it becomes difficult to identify malicious mappers.

The assets targeted by this attack include “Database management systems (DBMS)” (such as the traditional “Relational SQL” databases, and the Big Data new tools “NoSQL” and “NewSQL”), and asset type “Computing infrastructure models”.

Threat: Generation and use of rogue certificates

Device signing and media encryption can be critically undermined by the use of rogue certificates allowing attackers the access to Big Data assets and communication links . These can then be used to access data storage and thus causing data leakage, intercept and hijack individuals’ secure Web-based communications, misuse of brand, and upload/download malware or force updates, which potentially contain undesired functionality for Big Data software and hardware components.

Social networks such as Facebook are affected. According to reports in some circumstances download flaws allowed attackers to plant a malicious file on a victim’s machine that looks like it is coming from a trusted Facebook domain .

Many assets are targeted by this threat: including asset groups “Data” and “Big Data analytics”, and assets “Software” and “Hardware”.

Threat: Misuse of audit tools / Abuse of authorizations / Unauthorized activities

Audit information is necessary to ensure the security of the system and understand what went wrong; it is also necessary due to compliance and regulation. The scope and the granularity of the audit might be different in a Big Data context and the effect of the misuse of such information may be amplified.

For example, key personnel at financial institutions require access to large data sets that contain personally identifiable data . Also, there can be massive breaches of privacy when employees of providers hosting social networks, using their administrative credentials, regularly access private user information . For this reason, it is important to keep security-relevant chronological records. Since the misuse and abuse

of authorization can become a common issue, it is necessary to protect a large number of assets containing granular audits, documentation of the security policies, logs and cryptographic keys (e.g. all the assets included in category “Security and privacy techniques” of our asset taxonomy).

The assets targeted by these threats include “identification record data”, “Database management systems (DBS)” (for example “NoSQL” and “NewSQL”) and asset group “Security and privacy techniques”.

Threat: Failures of business process

Failures of business process according to ENISA taxonomy are threats of damage and/or loss of assets due to improperly executed business process. In Big Data, this class includes all threats related to data integrity that can be favoured by Big Data storage policies. In particular, the highly-replicated and eventual consistency nature of big data represents a driver towards attacks to data integrity, where data items stored in different replicas can be inconsistent. This scenario is summarized in the new concept of Big Data degradation, which represents an increasing risk for Big Data correctness.

This scenario also defines a “Big Data Leak”, a total or partial disclosure of a Big Data asset at a certain stage of its lifecycle as opposed to a “Big Data breach” (e.g. a theft of an asset executed by breaking into the infrastructure). In our case Big Data can be unwillingly disclosed by the owner to the provider of an outsourced process, for example when computing data analytics . This disclosure of information, at a certain stage of the Big Data lifecycle, can be exploited by an honest, but curious attacker, even without hostile intention.

Also, several cases of inadequate anonymisation of users are reported. While data collection and aggregation uses anonymization techniques, individual users can be re-identified by leveraging other Big Data datasets, often available in the public domain . This is an emergent phenomenon introduced by Big Data variety that has the ability to infer identity from anonymized datasets by correlating with apparently innocuous public information. Examples related to de-identification of personally identifiable information 81 82 (PII) are given by the AOL case and by NIST Big Data publications in Web logs collection and analysis . For a more detailed study on deanonymizatuion and anonymity issues in Big Data systems see ENISA’s report “Privacy by design in big data: An overview of privacy enhancing technologies in the era of big data analytics “.

The assets targeted by this threat include asset groups “Data” and “Big Data analytics”.

4.2.4 Threat Group: Legal

This group includes threats due to the legal implications of a Big Data system such as violation of laws or regulations, the breach of legislation, the failure to meet contractual requirements, the unauthorized use of Intellectual Property resources, the abuse of personal data, the necessity to obey judiciary decisions and court orders.

Threat: Violation of laws or regulations / Breach of legislation / Abuse of personal data

Data storage in the European Union falls under the Data Protection directive: organizations are required to i) adhere to this compliancy law throughout the life of the data, ii) remain responsible for the personal data of their customers and employees, and iii) guarantee its security even when a third-party like a cloud provider processes the data on their behalf.

In the traditional data centric model, data is stored on-premise, and every organization has control over the information. In Big Data, instead, a real concern is arising about the security of this massive amount of digital information and the protection of the critical infrastructure supporting it, as demonstrated by a vast

literature about privacy risks .

We should also note that EU has stricter regulations regarding the collection of personal data than other countries, but sometimes multinationals operating in the EU are based in the United States. In this context, the most important privacy issues are how to protect individual privacy when the data is stored in multiple

sites, and how efficient the protection is .

Big Data also raises the potential issue of data residency . Data, when stored in cloud storage of providers that offer multi-national storage solutions, may fall under different legal jurisdictions. An example brought by the NIST Big Data Public Working Group regards the custody of pharmaceutical data beyond trial disposition, which is unclear, especially after firms merge or dissolve .

The assets targeted by this threat include asset groups “Data” (especially “identification record data”) and “Roles”.

4.2.5 Threat Group: Organisational threats

This group includes threats pertaining to the organizational sphere.

Threat: Skill shortage

The analysis of large datasets can underpin new waves of productivity growth and innovation, and unlock significant value. However, companies and policy makers must tackle significant hurdles, such for instance a possible shortage of skilled data scientists and managers .

The asset targeted by this threat is asset group “Roles”.

5. Threats agents

According to ENISA Threat Landscape 2013 , a threat agent is “someone or something with decent capabilities, a clear intention to manifest a threat and a record of past activities in this regard”. For Big Data asset owners it is crucial to be aware of which threats emerge from which threat agent group. This study does not develop a new glossary on threat agents, but utilises the ENISA Threat Landscape 2013’s consolidation of several publications .

The categorization of threat agents is as follows:

Corporations: they refer to organizations/enterprises that adopt and/or are engaged in offensive tactics. In this context, corporations are considered as hostile threat agents and their motivation is to build competitive advantage over competitors, who also make up their main target. Depending on their size and sector, corporations usually possess significant capabilities, ranging from technology up to human engineering intelligence, especially in their area of expertise.

Cyber criminals: they are hostile by nature. Moreover, their motivation is usually financial gain and their skill level is, nowadays, quite high. Cybercriminals can be organised on a local, national or even international level.

Cyber terrorists: they have expanded their activities and engage also in cyber-attacks. Their motivation can be political or religious, and their capability varies from low to high. Preferred targets of cyber terrorists are mostly critical infrastructures (e.g. public health, energy production, telecommunication), as their failures cause severe impact in society and government. It has to be noted, that in the public material analyses, the profile of cyber terrorists still seems to be blurred.

Script kiddies: they are unskilled individuals using scripts or programs developed by others to attack computer systems and networks, and deface websites.

Online social hackers (hacktivists): they are politically and socially motivated individuals that use computer systems to protest and promote their cause. Their typical targets are high profile websites, corporations, intelligence agencies and military institutions.

Employees: they refer to the staff, contractors, operational staff or security guards of a company. They can have insider access to company’s resources, and are considered as both non-hostile threat agents (i.e. distracted employees) and hostile agents (i.e., disgruntled employees). This kind of threat agents possesses a significant amount of knowledge that allows them to place effective attacks against assets of their organization.

Nation states: they can have offensive cyber capabilities and use them against an adversary. Nation states have recently become a prominent threat agent due to the deployment of sophisticated attacks that are considered as cyber weapons. From the sophistication of these malware, it can be confirmed that Nation states have a plethora of resources and they have a high level of skills and expertise .

All agents listed in this section, may have an interest in exploiting certain vulnerabilities in Big Data for different reasons. Only some specific threats come more typically from certain agents, as, for instance, the abuse of authorization that is related to corporation employee, who can use their administrative credentials to access systems. In the following table we propose a cross relation between threats and agents in Big Data. Annex C presents an overall mappings between assets, threat agents and threats.

● ● ○ ● ○ ● ● ● ● ● ● ● ● ● ● ● ○ ● ●

6. Good practices

In this section, we provide a discussion summarizing good practices to protect Big Data assets. A good practice is a method or technique that has consistently shown results superior to those achieved with other means, and that is used as a benchmark. To this aim, different sources have been collected, reviewed, and mapped to the previously identified Big Data threats. They specify vulnerabilities, recommendations, controls, countermeasures, and good practices published by institutions or working groups, and relevant for the protecting the assets and counteracting the threats in this report. The first result of our analysis is that publicly available information on Big Data security issues mainly originates from research and is based on requirements and generic assumptions, while materials of real-life experience are not often available. This is mainly due to the fact that development of Big Data infrastructures and their related security measures are at an early stage of maturity. In fact, on one side, many of Big Data infrastructures have been operational for a limited period of time; on the other side, Big Data security assessment is in many cases managed confidentially for reasons of competitiveness.

Generally speaking, Big Data being a collection of input channels from sensors, networks, storage and computing systems, and output to data consumers, there is shared responsibility for security and infrastructure management. Every party, such as a data provider or a data consumer, should be conscious that its own security also depends on the security of its neighbours. Countermeasures and good practices are expected to be implemented to increase security of single parties, and of other related parties when applicable.

One more source of potential controls and technical countermeasures stems from the use of Big Data analytics as a tool for increasing system and data security, and improving intrusion detection and prevention. For completeness a small presentation of the expected capabilities is given in Annex E: Big Data analytics for security.

7. Gap analysis

In this section, we provide a gap analysis for those cases where further research and investigations are required in the areas of Big Data threats, security, and good practice.

This analysis aims to close the gaps highlighted in the previous section and is summarised as follows. The use of cryptography may be not always sufficient and there are obvious risks associated to administrators and security professionals with equivalent privileges. This is especially true when threats related to information leakage and/or sharing due to human errors are considered. Furthermore, leaks of data via Web applications (unsecure APIs) and inadequate design/planning or improperly adaptation need an improved design of computing and storage infrastructure models, while streaming data from sensors may have issues of confidentiality that cannot be mitigated by current solutions. Personal identifiable information is at risk even when best practices are widely followed and calls for privacy-oriented defensive approaches. Malicious code and activities pose a risk to models of computing infrastructure and storage due to the difficulties of patch management in a Big Data heterogeneous environment, while violation of laws or regulations, breach of legislation and abuse of personal data may affect final users. All these breaches requires, on one side, Big Data specific countermeasures, and, on the other side, the involvement of policy makers to reflect changes in current IT environment in EU laws and legislations. Finally, a skill shortage in roles such as data scientists is foreseen.

We categorize the gaps into four groups: gaps (i) on data, (ii) on the use of cryptography (iii) on computing and storage models and (iv) on roles (e.g. administrators, data scientist, and final users).

Gaps on data protection

Major gaps are found due to threats to privacy (e.g., the identification of personal identifiable information) and to confidentiality of sensor data streams.

As already reported in this report, several cases of identity fraud due to traffic capture and data mining have been recorded in recent years. Big Data analysis has facilitated the intrusion of privacy by strengthening common techniques and further research in this field is required. Since countermeasures, discussed in the previous section, such as anonymization did not prove to be always effective against Big Data mining, new research efforts are made to devise better controls. For example, a promising topic, actively researched, is privacy-preserving data mining (PPDM). The basic idea of PPDM is to modify the data in such a way so as to perform data mining algorithms effectively without compromising the security of sensitive information contained in the data.

In addition, it is foreseeable to have streams of data from sensors certified when possible. Since centralized cryptography systems are hard to implement when a large number of sensors is involved, the use of Trusted Computing (TC) appears to be a promising technology. Trusted computing relies on Trusted 135 136 Platform Modules (TPMs) and related hardware to prove integrity of software, processes, and data. TPM chips are not expensive and could be fitted in sensors at build time. TPM-enabled devices could

provide reliable data streams. However, on the server side (e.g., Big Data cloud-based installations), the use of this new technology is more challenging since hardware TPM should be adapted to virtualized environments. A researched approach is based on the notion of virtual Trusted Platform Module (vTPM) , which provides secure storage and cryptographic functions of TPM to applications and operating systems running in virtual machines.

Other hardware-based security technologies include the development of new processors for the embedded smart sensors . These new processors include protected areas for storage of user authentication keys, as well as areas of the processor that are off-limits to unauthorized users.

Besides the above technically-oriented aspects of data protection gaps, in 2015 ENISA has conducted a privacy-oriented assessment of Big Data ”Privacy by design in big data” . In this work, more thorough privacy gaps have been identified and recommendations have been made. Highlights include: application of privacy by design, preservation of privacy by data analytics and the need for coherent and efficient privacy policies for big data. It is recommended to refer to this document in order to obtain full perspective of security and privacy issues of Big Data.

Use of cryptography in applications and back-end services

The use of cryptography in Big Data as a mitigation countermeasure can be challenging. Gaps related to the use of cryptography are mainly related to: i) performance and scalability, ii) protection of logical and physical fragments, such as data blocks.

In fact, in Big Data, cryptography adds complexity and negatively affects performance. New dedicated products and ad hoc solutions are under development, as for example the already discussed TC and TPM technologies, while some interesting new approaches to cryptography for Big Data applications as the notion of “cryptography-as-a-service” in cloud environments are emerging. In recent years, there has been a lot of discussion around novel, but still rather esoteric crypto-algorithms. Homomorphic 140 141 encryption , honey encryption and other proposals could, at least in theory, provide end-to-end data protection and confidentiality. As an example, assuming the existence of a fully homomorphic cryptoscheme, one could use public Big Data systems to perform analytics – with the expected speed or accuracy losses – without ever revealing the data to anyone else, not even the computation and storage service provider. Research is still ongoing but the interested reader can find a concise study of the current state of the art in ENISA’s “Privacy by design in big data: An overview of privacy enhancing technologies in the

era of big data analytics” .

Gaps on computing and storage models

Computing Infrastructure and storage models in Big Data face new challenges such as the lack of standardization and portability of security controls among different open source projects (e.g., different Hadoop versions) and Big Data vendors , and the poor design of security features.

Often, standards do not exist or are still under development. An example of lack of standards is brought by NIST Big Data Working Group for the shipping industry, which uses Big Data in the identification, transport, and handling of items in the supply chain . However, at the moment, the status of the shipped items (e.g., unique identification number, GPS coordinates, sensors information, etc.) is not passed through the entire chain. A unique identification schema is under development within an ISO technical committee.

From a security perspective, we note that in a traditional management system as, for example, in an SQL relational database, security has slowly evolved and many new controls have been proposed over the years. Unlike such solutions, the security of Big Data components has not undergone the same level of rigor or evaluation due to the immaturity of Big Data research and development.

Gaps on roles (administrators, data scientist, and final users)

As stated in the previous section, many roles can be critical in Big Data, in particular system administrators, data scientist, and users.

Big Data administrators and other privileged users are a big concern since they require access to corporate data systems when working on behalf of the cloud services provider. Moreover, they could use their grants to access key stores and other sensitive information . All the data scientist positions are unlikely to be filled in the near future, while users might not always be conscious of or care about the legal implications of data storage – legal implications that will vary large and wide around the world.

Awareness, education, and training are the keys to close these gaps concerning human resourses. Some new online educational web sites are offering specialised courses in Big Data, for example the Big Data 146 147 University sponsored by IBM, and MIT . The Big Data University is run by a community, which includes many IBM staff members, contributing voluntarily to the development of courses, and to enhancing the site; also Amazon is contributing to the initiative . Other courses are available at Massive Open Online

Recommendations

The above gaps naturally result in a set of recommendations that can be classified as general recommendations, technical recommendation and recommendation on human resources.

General recommendations: they target the main Big Data stakeholders such as owner of Big Data projects and policy makers. In particular, stakeholders should depart by the assumption that a Big Data environment is simply a traditional data environment focusing on large amount of data. Big Data is more than a simple scalability problem, and management tools and risk assessment countermeasures and solutions should consider and address all 5V characterizing a Big Data environment.

This consideration is important both for policy makers specifying laws and regulations targeting current ICT environment, and stakeholders managing Big Data platforms and analytics. Especially for the latters, it becomes fundamental to evaluate i) the current level of security by understanding the assets covered (and not covered) by existing security measures, ii) the effectiveness of the application of good practices adapted from traditional security and privacy tools and techniques.

General recommendation requires a parallel standardization effort supporting the definition of proper and specific Big Data tools and legislations.

Technical recommendations: they target owners of Big Data projects and developers of corresponding products.

Following general recommendation of being Big Data specific, stakeholders should limit as much as possible the practice of adapting existing products to Big Data. Big Data introduces completely novel environments with new assets, threats, risk, and challenges. As a consequence, new products are needed to provide effective countermeasures and increase the trustworthiness of Big Data environments. Such products must be put in the Big Data life cycle after a careful evaluation, through pilots, aimed to verify and prove their correct behaviour. Success of these new products passes from a commitment by thirdparty vendors to apply security measures and stay focused on any updates.

Moreover, developers of Big Data products should benefit from new tools providing security and privacy functionalities by default.

To conclude, as already specified in the general recommendations, international bodies are invited to support this shift to Big Data specific security and privacy solutions by starting a gap analysis on Big Data standards, and new standardization activities according to the identified gaps.

Recommendations on human resources: they target human resources managing and using Big Data assets. As in traditional environments, in fact, human resources are one of the main sources of threats, and include users that attack a system either maliciously or accidentally. To limit these scenarios, all involved parties should focus on training of specialized professionals. Big players should support education initiatives on Big Data to raise/train tomorrow's scientists, fostering information and communication technology security awareness and training programs. Private companies and governmental bodies should

encourage technical staff to attend offline/online courses from respected institutes to increase their competences. Final users should learn about their rights and threats to privacy attending courses and educational initiatives. Big Data administrator and other privileged users should cooperate with the international community to exchange on threats and promote the application of good practices as mitigation measures. Finally, Big Data administrator should rely on good practices, and report on their implementations choices in terms of considered assets, threat, countermeasures, and identified gaps.

Annex A: Full list of Big Data taxonomy Annex B: Full Big Data asset taxonomy structure Annex C: Full list of threats affecting Big Data

This section provides a list of all the threats that affect Big Data assets and that were examined in the previous sections of the document.

Annex D: Full Big Data threat taxonomy structure Annex E: Big Data analytics for security

In recent years, Big Data analytics have attracted the interests of the security community as the means to increase security protection, thanks to its promise to analyse and correlate security-related data efficiently and at unprecedented scale. Even if ICT security community has been analysing logs, network flows, system events, and other information sources to identify threats and detect malicious activities for many years, conventional technologies have not always proven to be adequate to support long-term, large-scale analytics for many reasons . First, performing analytics and complex queries on large, unstructured datasets is inefficient since conventional Business Intelligence tools are designed to analyse and manage data organized in fixed and predefined schemas. Second, the management of large data warehouses is very expensive and dedicated installation, and their deployment usually requires strong business motivations.

The above limitations can be mitigated by applying Big Data analytics techniques to huge amount of backend data, to the aim of uncovering security threats, attack patterns, and security exploits. This section details how the security analytics landscape is changing with the introduction of Big Data applications and tools, such as the Hadoop framework , which can deploy, clean, process, analyse, and query large amounts of structured and unstructured data efficiently.

In particular, we will examine Big Data analytics as a tool for increasing system and data security, and improving intrusion detection and prevention. This falls under the more general name of threat intelligence, which is defined as “evidence-based knowledge, including context, mechanisms, indicators, implications and actionable advice, about an existing or emerging menace or hazard to assets that can be used to inform decisions regarding the subject's response to that menace or hazard” .

Threat intelligence evolution started years ago, security companies started building anti-virus products, later on created signatures to capture bad emails, and finally introduced reputation through behavioural indicators to infer which files were malware and which messages were spams . More recently, when networks of compromised devices (botnets) could evade detection and defeat reputation, heavier reliance was put on other techniques, such as attackers’ analysis with a resulting aggregation of big amounts of data, to get a better view on the threat landscape. Nowadays, threat intelligence can be considered as a knowledge base, which represents the synthesis of information detailing potential threats with a solid understanding of network structure, operations, and activities . This knowledge base is a collection of indicators, often called “threat feeds”, which must be contextualised with a baseline of normal network activities. Big Data security analytics can be used to improve chances of detection and ascertain trends.

Big Data analytics can adopt different approaches. For example, Hadoop can be programmed to detect all data entering and exiting the network. This configuration has been used to pick up odd activities, such as when an infected PC or server starting communicating at unusual times . Another example uses Hadoop to monitor system logs : a service called Flume is used for streaming log data into Hcatalog, a SQL based data store, then Pig is used to query and refine the data and Elastic Search for a high-level visualization. A proposed Early Warning System methodological framework implements a systematic process to collect and aggregate security data internally, establish a number of baselines that identify normal behaviour, gather external intelligence (threat feeds) from third parties, and uses Big Data analytics to analyse this information for particular dangers.

Future directions for research in Big Data analytics for security aim at addressing business risks for the whole organisation . In fact, without a risk-based approach to security, organisations could waste valuable ICT resources for vulnerabilities that are not causing in reality big threats to the business. Also it will be important to filter security Big Data to the information that is just relevant to specific stakeholders’ roles and responsibilities.

In the following of this section we focus on Big Data analytical solutions for the following threats: anomaly detection, denial of service, fraud detection, and botnets.

Anomaly detection

Telecommunication providers observe Internet traffic and analyse it to study malicious activities. For this reason a large amount of data is collected and millions of global DNS requests, HTTP transactions and full packet information are correlated. Network operators analyse the data usually relying on detection tools based on anomaly detection techniques. These tools are based on a reference model of the normal traffic behaviour and compute the deviation from it. However, to cope with the global growth of Internet and obtain results in a reasonable time, the Internet traffic is usually sampled, even if sampling is inherently detrimental to anomaly detection.

Big Data tools promise to improve the analysis of these large datasets, by providing fast and accurate analytics. Recently, the research community has then put a lot of effort in the development of efficient tools using the MapReduce model. As an example, EU funded project NECOMA is investigating the benefits of MapReduce to achieve real-time anomaly detection with non-sampled traffic. The goal of the project is to provide high scalable and fault tolerant tools implementing features for anomaly detection. The project proposes a MapReduce-based framework that consists of two steps : first, the traffic is

divided into splits using a hash function and preserving both the spatial and temporal structures of the traffic; second, detectors identify anomalies in each split of data.

Denial of service

A special case of anomaly detection is the discovery of Distributed Denial of Service attacks (DDoS). For example, anomaly detection tools can identify DDoS attacks by observing numerous requests sent to the same service. However, detection performance and accuracy can be highly decreased by the availability of a sampled set of requests only . Also here the use of Big Data tools can help: various designs of 161 163 MapReduce based frameworks and of detection algorithm for the major flooding attacks (TCP-SYN, HTTP GET, UDP and ICMP) are reported.

Fraud detection

Fraud is a deliberate deception to obtain unfair or unlawful gain. The purpose of a fraud may be monetary gain or other benefits, such as retrieving certificates by way of false statements. New approaches based on Big Data analytics can be used to combat fraud, by i) correlating real-time and historical account activity, ii) relying on baselines to spot abnormal user behaviour uncovering trends and patterns in large amounts of data, iii) establishing patterns and relationships, and iv) making non-obvious connections between disparate sources of data . By these means, businesses can identify fraud risks at an early stage, thus preventing crime and solving investigations. The technical challenge about fraud analysis involves looking for behavioural patterns and building a profile of normal activities , accessing sparse financial information data and parsing unstructured text, and understanding discrepancies in customer transactions. We also notice that companies are not inclined to reveal real cases of fraud that have undergone, unless law requires them and real cases are not often reported.

The use of Big Data tools is becoming common for insolvency and forensic professional services, and for teams of dedicated investigators covering financial investigations . The crimes can be traced by analysing structured and unstructured data sources such as bank statements, PDF files, emails, invoices and spread sheets. The same article claims that “technology has enabled the investigation business to move from struggling with huge amounts of information stored in spread sheets to a faster, more accurate,

intelligence-led approach that helps to solve cases related to money laundering, missing trader fraud, and theft of company assets.”

Botnets

Network flow data collected by telecommunication providers are also analysed to identify malicious communications associated with botnets. Years of historical data on DNS transactions across enterprises are analysed by Big Data tools, applying custom analytics to help in identifying suspicious domain names used by botnets , infected hosts, and past intrusions. Another approach uses a distributed computing framework that examines host relationships using a combination of PageRank and clustering algorithms to track the command-and-control channels in the botnet.

Annex F: Summary of threat taxonomies

In this report we have used the ENISA threat taxonomy to map threats to Big Data assets. Obviously it is not the only threat taxonomy that may be used for this exercise. In this section we provide a very short review of some other taxonomies of threats and cyber attacks presented in international conferences and workshops (Error! Reference source not found.) that might be used as an alternative. The review showed that there are no specific threat taxonomies for Big Data assets.

Article #1 proposes a cyber attack taxonomy called AVOIDIT (Attack Vector, Operational Impact, Defence, Information Impact, and Target). The study uses five major classifiers to characterize the nature of an attack: (i) classification by attack vector, (ii) classification by operational impact, (iii) classification by defence, (iv) classification by informational impact, and (v) classification by attack target. This technique is useful to associate threats to Big Data with the corresponding attackers and foreseeable effects. The study made by Im and Baskerville in #2 focuses on human errors, which remain a significant and poorly recognized issue for information system security. The study (2005) proposes and validates a taxonomy of information security threats, which provides additional insight into human error as a significant source of

security risks. For this reason, taxonomies like the one proposed by ENISA take into account threats as unintentional damage, information leakage/sharing due to human errors, or configuration mistakes and errors during software development (unsecure APIs). The research in #3 offers a comprehensive taxonomy of Distributed Denial of Services (DDoS) attacks with the corresponding countermeasures and defence mechanisms. Article #4 proposes taxonomy with four dimensions, which provide a holistic classification, covering network and computer attacks. This taxonomy provides assistance in improving computer and network security as well as consistency in attack description. Article #5 proposes cyber-attack taxonomy and provides nine classes of cyber-attacks. The focus is on effectively classifying attacks. It also proposes taxonomy of cyber adversaries arranged in eight classes according to skill level. In the context of NSFsponsored work, #6 envisions combining modular threat libraries, i.e. structured or unstructured lists of threats to be coupled with attack taxonomies, which offer a classification scheme to help developers find relevant attacks more easily. The threat list is based on the notion of a threat pattern, which can be customized and instantiated in different architectural contexts to define specific threats to a system. The approach includes a method to construct pattern-based threat taxonomies for more specific system types and/or technology contexts by specializing one or more threat patterns.

ENISA

European Union Agency for Network and Information Security Science and Technology Park of Crete (ITE) Vassilika Vouton, 700 13, Heraklion, Greece

Athens Office

1 Vass. Sofias & Meg. Alexandrou Marousi 151 24, Athens, Greece

PO Box 1309, 710 01 Heraklion, Greece Tel: +30 28 14 40 9710 info@enisa.europa.eu www.enisa.europa.eu