ECSC 2020 Analysis Report

About ENISA

The European Union Agency for Cybersecurity, ENISA, is the Union’s agency dedicated to achieving a high common level of cybersecurity across Europe. Established in 2004 and strengthened by the EU Cybersecurity Act, the European Union Agency for Cybersecurity contributes to EU cyber policy, enhances the trustworthiness of ICT products, services and processes with cybersecurity certification schemes, cooperates with Member States and EU bodies, and helps Europe prepare for the cyber challenges of tomorrow. Through knowledge sharing, capacity building and awareness raising, the Agency works together with its key stakeholders to strengthen trust in the connected economy, to boost resilience of the Union’s infrastructure, and, ultimately, to keep Europe’s society and citizens digitally secure. More information about ENISA and its work can be found on its website www.enisa.europa.eu.

Contact

To contact the authors, please use ecsc@enisa.europa.eu. For media enquiries about this paper, please use press@enisa.europa.eu.

Authors

Adrián Belmonte Martin and Ioannis Agrafiotis – European Union Agency for Cybersecurity; contributions from the report on lessons learned and the public affairs strategy by Arttic and Weber Shandwick.

Acknowledgements

We would like to thank all our colleagues responsible for the national competitions and the preparation of the teams that participate in the European Cyber Security Competition. Without their untiring efforts, this competition would not be possible.

Legal notice

Notice must be taken that this publication represents the views and interpretations of ENISA, unless stated otherwise. This publication should not be construed to be a legal action of ENISA or the ENISA bodies unless adopted pursuant to Regulation (EU) No 2019/881. ENISA may update this publication from time to time.

Third-party sources are quoted as appropriate. ENISA is not responsible for the content of the external sources, including external websites, referenced in this publication. This publication is intended for information purposes only. It must be accessible free of charge. Neither ENISA nor any person acting on its behalf is responsible for the use that might be made of the information contained in this publication.

Copyright notice

© European Union Agency for Cybersecurity (ENISA), 2021 Reproduction is authorised provided the source is acknowledged. For any use or reproduction of photos or other material that is not under ENISA’s copyright, permission must be sought directly from the copyright holders.

ISBN: 978-92-9204-496-1 - DOI: 10.2824/284260

1

ECSC 2020 ANALYSIS REPORT

EXECUTIVE SUMMARY

The 7th edition of the European Cyber Security Challenge (ECSC), planned initially for 4 and 5 November 2020 in Vienna, had to be postponed because of the coronavirus disease 2019 pandemic. Considering that the organisation of this European event brings together more than 350 young people coming from all over Europe, the ECSC Steering Committee wants above all to safeguard the health of all participants.

Consequently, the committee, together with the Austrian national planners, with the support of the European Union Agency for Cybersecurity (ENISA) and the agreement of the future ECSC organising countries, decided to change the dates of the ECSC 2020 final and postpone the 7th edition to 2021.

Therefore, the 7th edition of the ECSC will take place in Prague, Czechia, in 2021. Austria will host the event in 2022.

Despite this, to maintain support for the growth of the competition, keep the spirit of the ECSC alive and prepare for the 2021 edition, various activities were performed during 2020, such as the first ENISA Hackfest, the initial steps towards the selection of Team Europe for the International Cyber Security Challenge (ICSC) planned for Athens in December 2021 and the launch of the ICSC Steering Committee, responsible for the international competition.

ENISA is currently hosting a number of platforms and performing several activities to support the ECSC hosting country and the future development of the project, including hosting the ECSC main website, hosting the ECSC planning platform, working on the public affairs strategy, creating challenges, overseeing the governance framework of the competition and providing support for the ECSC Steering Committee as the acting secretariat.

In order to ensure appropriate and transparent reporting to the ECSC Steering Committee, the following key observations were made by independent third-party observers who attended ECSC 2020 planning meetings. These observations were made based on the feedback collected from participants, members of the ECSC Steering Committee and attendees at the Hackfest event, including through an online evaluation survey and personal interviews.

 Concerning the way in which the 2020 cancellation was managed, it was stressed that the transparency shown by ENISA and its involvement of the ECSC Steering Committee in the decision-making were good. Decisions were made in agreement with all countries concerned, for example on how to communicate the cancellation of ECSC 2020.  There is no need for the committee to do things very differently. Some improvements could be made to accelerate the circulation of official documents when finalised and to clearly state who is to receive ECSC-related documents. Based on the responses to the organisers’ questionnaire, 77 % of respondents considered that the ECSC Steering Committee performs well.  A clearer dissemination plan would be helpful, but there is also a need to establish a transmission channel between the ECSC public affairs team and contractor on the one side and national teams on the other. Many ECSC participants would welcome messages and communication materials crafted at ECSC level. This should include infographics and other visual elements to help to ensure that posts on Twitter or on participants’ websites are eye-catching and engaging.

2

ECSC 2020 ANALYSIS REPORT

 Regarding ENISA Hackfest 2020, based on the questionnaire sent to participants, 76 % of respondents considered that the performance of the platform was good or excellent. No major issues or incidents with regard to the availability of network infrastructure were reported by respondents to the questionnaire or in interviews.

In addition, it seems worth noting that some of the open-ended comments made by the ECSC Steering Committee members who participated in the study concerned:

 the strengthening of the European dimension of the ECSC;  the need to preserve the ECSC’s spirit of fun;  the balance to be struck between opening the competition to as many countries as possible and logistical and financial issues;  the need to develop a sense of a community and to build a community inclusive of women and very young participants;  the need for reflection on ways to attract more women into the cybersecurity field;  the possibility of tapping into the pool of expertise formed by ECSC members.

The final of the 2021 edition of the ECSC will take place in Prague in September 2021; at least 22 countries are expected to participate. The latest updates will be published on the ECSC 2021 website (https://www.europeancybersecuritychallenge.eu/).

This report is not for public dissemination. It concerns only ENISA and the members of the ECSC Steering Committee, namely the representatives of the countries that participate in the ECSC.

3

ECSC 2020 ANALYSIS REPORT

TABLE OF CONTENTS

1. Introduction 6

1.1. Background to the ECSC 6 1.2. The ECSC set-up 7

2. Trends in ECSC attendance 8

2.1. ECSC attendance 8 2.2. Participation in the ECSC Steering Committee 9

3. ENISA’s contribution to ECSC 2020 10

3.1. Steering Committee management 10 3.2. Initial planning conference 10 3.3. COVID-19 impact assessment 11 3.4. International Cyber Security Challenge 12 3.5. ENISA Hackfest 2020 14 3.6. Platforms 22 3.7. Public affairs and media activities 24 3.8. Working groups 25

4. ECSC 2020 – lessons learned and maturity assessment 27

4.1. Governance and decision-making aspects 29 4.2. Public affairs – general perspective 30 4.3. Challenges – focus on ENISA Hackfest 2020 32 4.4. Compliance – focus on ENISA Hackfest 2020 34

5. Areas for improvement and recommendations 35

5.1. Governance and decision-making aspects 35 5.2. Public affairs – general perspective 35 5.3. Challenges – lessons learned from ENISA Hackfest 2020 36 5.4. Compliance – lessons learned from ENISA Hackfest 2020 37

6. ECSC 2021 38

4

ECSC 2020 ANALYSIS REPORT

ABBREVIATIONS

API application programming interface CET Central European Time COVID-19 coronavirus disease 2019 CIRCABC Communication and Information Resource Centre for Administrations, Businesses and Citizens CTF capture the flag ECSC European Cyber Security Challenge EFTA European Free Trade Association ENISA European Union Agency for Cybersecurity FAQ frequently asked question ICSC International Cyber Security Challenge KPI key performance indicator

5

ECSC 2020 ANALYSIS REPORT

1. INTRODUCTION

The growing need for IT security professionals is widely acknowledged. According to recent estimates, it is expected that more than 3.5 million cybersecurity professionals will be needed worldwide by 2021 ( ) to prevent, react to and protect citizens from cyber threats. Europe has to make an effort to attract talent to and retain it in cybersecurity and, at the same time, create solid and powerful educational, entrepreneurial and business structures in cybersecurity.

To help mitigate this shortage of skills, many countries have launched national cybersecurity In a nutshell, competitions targeting students, university graduates and even non-IT professionals, with a

ECSC is the

clear aim:

annual European event that brings

‘Identify new and young cyber talents and encourage young people to pursue a career in cyber

security.’ together young talent from across

The European Cyber Security Challenge (ECSC)

Europe to have

(https://www.europeancybersecuritychallenge.eu/) leverages on these competitions by adding a

fun and compete

pan-European layer. Top cybersecurity talents from each participating country meet to network and collaborate and, finally, compete against each other. Contestants are challenged to solve in cybersecurity. security-related tasks in different domains. Its main aim is to

highlight the

In a nutshell, ECSC is the annual European event that brings together young talent from across

importance of the

Europe to have fun and compete in cybersecurity. Its main aim is to highlight the importance of

national

the national competitions.

competitions. 1.1. Background to the ECSC

The project was initiated under the umbrella of the European Union’s cybersecurity strategy of February 2013, which stated that the European Commission would:

‘organise, with the support of ENISA, a cybersecurity championship in 2014, where university students will compete in proposing [network and information security] solutions.’

Since 2014, the European Union Agency for Cybersecurity (ENISA) has been supporting the organisation of the ECSC. ENISA actively organises the meetings of the governance structures, supports the development of the competition’s rules and challenges and is part of the ECSC jury. Since 2016, ENISA has been the acting secretariat of the ESCS Steering Committee.

In the 2018 edition, 200 participants (contestants, coaches and judges), representing 17 EU and European Free Trade Association (EFTA) countries (Austria, Belgium, Cyprus, Czechia, Denmark, Estonia, France, Germany, Greece, Italy, Liechtenstein, Norway, Poland, Romania, Spain, Switzerland and the United Kingdom), competed in the ECSC final in London.

The 2019 edition of the ECSC took place in the parliament building in Bucharest, Romania, from 9 to11 October 2019. For the first time, teams from 20 countries participated in the final (Austria, Cyprus, Czechia, Denmark, Estonia, France, Germany, Greece, Italy, Ireland, Liechtenstein, Luxembourg, the Netherlands, Norway, Poland, Portugal, Romania, Spain, Switzerland and the United Kingdom).The participants investigated vulnerabilities in web applications, binaries and

6

ECSC 2020 ANALYSIS REPORT

document files; solved crypto puzzles; and hack hardware systems. However, technical skills are just one part of the story. Time and resources were limited, and teamwork and presentation skills were also evaluated. The top teams in ECSC 2019 were the teams from Romania, Italy and Austria.

The 2020 edition of the ECSC, planned initially for 4 and 5 November in Vienna, had to be cancelled owing to the coronavirus disease 2019 (COVID-19) pandemic.

The 2021 edition of the ECSC will take place in Prague, Czechia, on 29 and 30 September.

1.2. The ECSC set-up

Each country is represented at the ECSC final by a team of 10 contestants, comprising the winners of the national competition. Half of them are 14–20 years old and half are 21–25 years old.

Two preparatory pilot phases of the ECSC were held in 2014 (in Austria) and 2015 (in Switzerland), attended by three and six countries, respectively. Since 2015, ENISA has used its experience and position to coordinate and organise the ECSC and enable it to reach full maturity.

The activities of the ECSC are supervised by a steering committee, composed of representatives of the attending countries. ENISA facilitates the meetings of this group and provides strategic guidance. The decision-making processes are described in the ECSC Charter, which is revised and approved every year by the ECSC Steering Committee.

ENISA is currently hosting a number of platforms and performing several activities to support the ECSC hosting country and the future development of the project, including hosting the ECSC main website, hosting the ECSC information-sharing platform, working on the public affairs strategy and creating challenges.

In addition, ENISA is working closely with the hosts of future editions in order to ensure appropriate and transparent reporting to the ECSC Steering Committee.

Each year, two ECSC Steering Committee meetings are held prior to the event:

 the initial planning conference in February or March,  the main planning conference in June or July.

ENISA organised these preparatory events and had overall responsibility for the efficient running of the project, including minuting meetings and following up on proposed actions.

7

ECSC 2020 ANALYSIS REPORT

2. TRENDS IN ECSC ATTENDANCE

2.1. ECSC attendance

Since the first edition of the ECSC in 2014, the countries that have attended are the following.

 2014 (3). Austria, Germany and Switzerland.  2015 (6). Austria, Germany, Switzerland, Spain, Romania and the United Kingdom.  2016 (10). Austria, Estonia, Germany, Greece, Ireland, Liechtenstein, Romania, Spain, Switzerland and the United Kingdom.  2017 (15). Austria, Cyprus, Czechia, Denmark, Estonia, Germany, Greece, Ireland, Italy, Liechtenstein, Norway, Romania, Spain, Switzerland and the United Kingdom.  2018 (17). Austria, Belgium, Cyprus, Czechia, Denmark, Estonia, France, Germany, Greece, Italy, Liechtenstein, Norway, Poland, Romania, Spain, Switzerland and the United Kingdom. 2019 (20). Austria, Cyprus, Czechia, Denmark, Estonia, France, Germany, Greece, Italy, Ireland, Liechtenstein, Luxembourg, the Netherlands, Norway, Poland, Portugal, Romania, Spain, Switzerland and the United Kingdom.  2020. Owing to the COVID-19 pandemic, the competition was cancelled, with the 2022 edition to be held in Vienna instead.

Figure 1 depicts the growth of the ECSC since ENISA became involved in the competition in 2014.

Figure 1: Number of countries participating in the ECSC, 2014–2020

8

ECSC 2020 ANALYSIS REPORT 2.2. Participation in the ECSC Steering Committee

In the past 12 months, the following new EU and EFTA countries have joined the committee:

 Croatia  Iceland  Latvia  Slovakia  Slovenia.

The full list of members participating in the ECSC Steering Committee is as follows:

 Austria  Belgium  Cyprus  Croatia  Czechia  Denmark  Estonia  France  Germany  Greece  Iceland  Italy  Ireland  Latvia  Liechtenstein  Luxembourg  Malta  the Netherlands  Norway  Poland  Portugal  Romania  Spain  Slovakia  Slovenia  Switzerland  the United Kingdom.

Total: 27 countries

9

ECSC 2020 ANALYSIS REPORT

3. ENISA’S CONTRIBUTION TO ECSC 2020

3.1. Steering Committee management

The activities of the ECSC are supervised by a steering committee composed of representatives of the attending countries. ENISA facilitates the meetings of this group and provides strategic guidance. The decision-making processes are described in the ECSC Charter, which is revised and approved every year by the ECSC Steering Committee.

During the preparations for ECSC 2020, ENISA was responsible for the following activities related to the management of the ECSC Steering Committee:

 platform maintenance and updates,  mailing list management,  updating and creating ECSC Steering Committee-related documentation,  supporting the organisation of the initial planning conference in Vienna,  organising follow-up meetings with the ECSC Steering Committee,  organising a follow-up meeting with the ECSC 2020 chair,  COVID-19 situational assessment,  accommodation, execution and follow-up of ECSC Steering Committee requests,  implementation of changes and suggestions collected through surveys and other feedback from previous editions,  management of the public affairs support contract,  management of the ECSC support contract,  updating the public affairs strategy,  creating various subcommittees and working groups to support new activities,  deciding on the methods for the creation of Team Europe for the International Cyber Security Challenge (ICSC),  organisation of a hot-wash meeting,  collecting feedback through a report on lessons learned and a survey of participants.

3.2. Initial planning conference

On 13 and 14 February 2020, the ECSC initial planning conference was held in Vienna. The following topics were discussed during the meeting:

 the ECSC 2019 experience,  presentation from Austria on plans for the 2020 edition (logistics, hotels, etc.),  lessons learned from ECSC 2019,  ECSC 2020 planning status,  Brexit and new countries,  update on the ECSC Connecting Europe Facility call,  update on the ICSC,  presentation of new platforms.

10

ECSC 2020 ANALYSIS REPORT

The following actions and decisions were agreed in the meeting:

 a code of conduct was to be created;  credentials and vouchers were to be provided to enable access to the Austrian platform that was to be used during the competition;  the presentations were to be eliminated in the next edition of the ECSC, with a subcommittee to be formed to submit proposals on the development of soft skills;  a calculation tool was to be provided for checking the ages of the participants;  two subcommittees were to be formed, with the following members:

o Subcommittee No 1 on ICSC decisions:  Austria  Cyprus  Czechia  Germany  Ireland  Luxembourg  Portugal  Spain  ENISA. o Subcommittee No 2 on soft skills:  Austria  Czechia  France  Germany  Greece  Norway  the Netherlands. After the event, the minutes of the meeting were circulated to the ECSC Steering Committee for review and approval.

3.3. COVID-19 impact assessment

During the months of April and May 2020, ENISA and the ECSC Steering Committee chair/organising country (Austria) were continuously assessing the situation in order to monitor the impact of the COVID-19 pandemic on the participant countries and have available the information required to enable the best decision to be taken on holding the final in Vienna.

During May 2020, ENISA led the collection of information from the participant countries regarding the following issues:

 impact of the situation on national competitions,  impact on meetings of national teams,  measures taken and special considerations required,  if the situation threatened national teams’ participation in the final.

Given the force majeure of the situation and the measures imposed by the Austrian government, finally Austria decided that the competition needed to be either postponed or cancelled, and an ad hoc meeting between ECSC organising countries was arranged in order to present and consult on the different options. After the situation was explained, Norway, the 2022 host, and the subsequent organising countries agreed to the postponement of their competitions for 1 year and yielded the 2022 edition to Austria. It was not possible to postpone until next year because Czechia already had organisational commitments in place.

Following this agreement between Norway, Italy and Poland, the ECSC final will take place in Austria in 2022, Norway in 2023, Italy in 2024 and Poland in 2025. The 2021 edition remains the same.

11

ECSC 2020 ANALYSIS REPORT

After this agreement, on 18 May, an extraordinary ECSC Steering Committee teleconference was arranged to announce the postponement and the new dates for the competitions and to formally approve the decision.

In addition, at this meeting special measures to mitigate the impact of the situation were adopted by the ECSC Steering Committee, such as raising by 1 year the maximum age for participation in the 2021 competition.

Regarding the official announcement of the changes, the ECSC Steering Committee proposed that the participating countries create a joint communication plan, define a common message and coordinate media activities. A dedicated teleconference was organised by ENISA’s public affairs team on 29 May. The official postponement announcement was launched on 3 June on the ENISA website (https://www.enisa.europa.eu/news/enisa-news/european-cyber-securitychallenge-2020-dates-changed), followed by various social media activities. In addition, the ECSC official website was modified accordingly.

Figure 2: News item on the ENISA website announcing the postponement of ECSC 2020

3.4. International Cyber Security Challenge

Building on the success of the ECSC, ENISA, with the help of other regional and international organisations, decided to design and host the ICSC. The aim of the challenge is to attract young talent and raise awareness in the global community of the education and skills needed in the area of cybersecurity.

12

ECSC 2020 ANALYSIS REPORT

To this end, a steering committee comprising representatives of government and regional institutions and universities and research centres will design a competition between teams from different regions. So far, teams from South-East Asia, Oceania, the United States, Latin America and Africa have expressed a clear interest in participating.

Teams will compete in a series of challenges (to be decided by the ICSC Steering Committee) in a number of areas, such as web application and system exploitation, cryptography, reverse engineering, hardware challenges, forensics and escape rooms. The ICSC brand is expected to be associated with the top cybersecurity talents of the world, and ENISA anticipates that the ICSC will become one of the world’s key incubators of cybersecurity entrepreneurship and top security experts.

ENISA is committed to organising the first ICSC final in Greece and supporting the development of the competition’s rules and governing body. Each region will be represented at the competition by a team comprising young talent from countries in the region. The regions will be responsible for selecting and training team members. ENISA proposes that the first ICSC

final be held during the second week of December 2021. The international competition will be held in Athens.

This competition can have a significant impact on strengthening cybersecurity; some of the objectives are the following:

 create and nurture a global multistakeholder cybersecurity challenge;  strengthen international cooperation on cybersecurity;  contribute to the world’s strategy on cyberspace;  promote capture the flag (CTF) competitions by leveraging to an international level;  engage a wider audience and increase the level of awareness of the importance of holding events such as the ECSC;  promote young talent across the world;  promote cybersecurity training at an international level;  develop various skills, including both technical skills and soft skills such as teamwork, presentation skills and public speaking;  promote professional careers in cybersecurity by putting the young professionals in the spotlight and highlighting the interesting career paths available, in order to address the shortage of cybersecurity professionals.

During early 2020, ENISA completed the creation of the ICSC Steering Committee to support management activities. The committee:

 is composed of representatives of the regions that have committed to competing in the ICSC;  is the ultimate decision-making body on the ICSC;  defines the role and composition of the ICSC secretariat.  provides leadership and direction on the scope and management of the work to be undertaken on matters that relate to the ICSC’s scope and objectives, and related risks and issues, as necessary;  acts in the best interests of the ICSC;  provides steering and guidance on ICSC activities, while respecting the overall responsibilities of all stakeholders;  is a delegated decision-making authority on the related activities within the scope of the ICSC;  keeps relevant stakeholders informed on the ICSC and related developments.  approves the ICSC Charter and the ICSC competition rules, accepts submissions from potential future hosting countries, decides on the dates and location of ICSC events, and makes decisions on any other issues arising, for example in the event of disputes.

13

ECSC 2020 ANALYSIS REPORT

Representatives of the various regions were engaged by ENISA during 2020 and began discussions to establish the required structures and the basis for the long-term development of future editions, namely:

 a consistent set of rules for the competition (ages and number of participants, requirements, etc.);  the final curricula for the competition;  the final format of the competition and the methods for maintaining a high level of transparency and trust between the regions.

ENISA, in the role of ICSC secretariat and organising body, executed the following actions in order to support ICSC activities:

 making initial contact with and engaging the International Steering Committee,  creating a mailing list,  creating a common roadmap,  creating of space in the collaboration platform (Communication and Information Resource Centre for Administrations, Businesses and Citizens (CIRCAB)),  organising follow-up meetings during the year,  starting preparatory activities for the competition,  developing key documentation,  organising surveys and collection of information to support the decision-making process,  carrying out activities to prepare for the final in December 2021.

3.5. ENISA HACKFEST 2020

On 16–18 November, ENISA hosted ENISA Hackfest 2020 (https://enisahackfest2020.cyberedu.ro/), a CTF event for cybersecurity professionals and students to connect and train the teams participating in the 2021 ECSC. Contestants were challenged on an individual level to solve cybersecurity problems in areas such as web security, mobile security, cryptography, reverse engineering and forensics. The Hackfest brought together more than 250 participants from 17 EU and EFTA countries. The event was held in a virtual format.

The event was to be used:

 as a testbed for future Team Europe selection activities;  by ECSC national teams to assess future players’ performance;  as training for ECSC participants;  to provide support to countries that cannot afford a national qualifying competition.

The event was hosted on a cloud-based platform (cyberEDU.ro) that can host large CTF competitions of various types, such as jeopardy, attack-and-defence and king-of-the-hill competitions, as well as competitions in other, more complex, formats such as cyber-range, classroom style, etc. The platform supports individual and team-based competitions. The technology can host events with as few as 10 players or as many as 5 000 and even more.

14

ECSC 2020 ANALYSIS REPORT

Figure 3: The ENISA Hackfest 2020 logo

3.5.1. Competition format

● Individual competition. ● Classroom style (teacher is visible only to his or her students). ● Jeopardy competition. ● All players registered to gain access to the challenges. ● Registration was available at the Hackfest website (http://enisa-hackfest2020.cyberedu.ro). ● Started on 16 November 2020 at 10:00 Central European Time (CET). ● Ended on 18 November 2020 at 10:00 CET. ● Team leaders received detailed reports after the event about each player, which included: ○ soft metadata such as nationality, age, sex, etc.; ○ technical statistics such as players’ results, categories of challenges, timestamps, etc.

3.5.2. Train the trainers event

To provide information to the team captains and coaches, a train the trainers event was organised by ENISA and the contractor; this event was expected to provide clarification on various key issues:

 roles during the competition,  rules,  communication with and support from the organisers,  challenges (types, difficulty, etc.),  complaint management,  scoring mechanisms,  user management,  reports and information,  the look and feel of the platform,  the registration process,  flag submission.

In addition, access to a demo platform with real challenges was provided to participants before the event started, for testing purposes, and a trainer’s manual was provided with all the relevant information.

15

ECSC 2020 ANALYSIS REPORT

The train the trainers event was organised virtually (using Cisco WebEx) on 6 November. A video and the manual were released on the ECSC collaboration platform and sent to the mailing list.

3.5.3. Hackfest results

● Total participants: 263. ● Total participants with positive score: 169. ● Total wrong attempts: 4 609. ● Total flags: 826. ● Gender statistics: ○ Men making attempts: 146. ○ Women making attempts: 3. ○ People who preferred not to say making attempts: 9. ● Average age of participants: 21.55 years. ● Number of countries participating: 17. ● Total challenges available: 24 challenges, 23 solved during the event, 21 solved by the best player. ● Fasted solve: 481 seconds, or 8 minutes and 1 second. ● Easiest challenge was solved by 128 players. ● Hardest challenge was solved by 2 players. ● On average, players solved 4.8876 challenges each during the event. ● Categories: Web security, Miscellaneous, Cryptography, Forensics, Steganography, Network security, Open-source intelligence, Mobile security, Memory analysis, Reverse engineering, Exploitation, Programming.

16

ECSC 2020 ANALYSIS REPORT

Table 1: Summary of challenges

Challenges Title Category Total solvers Final score

hello-nemo Miscellaneous, Forensics 128 50 downloader-v1 Web security 123 50 warmup-cat Miscellaneous 99 50 fair-dice Miscellaneous, Programming 71 50 imgur Web security 49 50 what-to-do Forensics 46 50 online-album Web security 41 100 Miscellaneous, Open-source treasure-map 40 110 intelligence s3-simple-secure-system Reverse engineering, Cryptography 39 120 lukas-skywalker-business Forensics 32 190 crypto Miscellaneous, Cryptography 31 200 api Web security 24 270 stargate Miscellaneous 21 300 posts Web security, Cross-site scripting 20 310 crow Web security 16 350 Forensics, Miscellaneous, Mobile investigator 14 370 security cerbebros Web security 6 450 crack-me Reverse engineering 6 450 spi-capture Miscellaneous 5 460 Network security, Forensics, ancient signal 5 460 Cryptography slot Pwn 4 470 middleman Reverse engineering 4 470 blindfold Cryptography 2 490

17

ECSC 2020 ANALYSIS REPORT 3.5.4. ENISA Hackfest 2020 – lessons learned

3.5.4.1. Scoring

Dynamic scoring continues to be one of the most objective and accurate ways to evaluate the difficulty of challenges based on the technical level of players. The proposed algorithm worked as expected and no malicious behaviour was detected.

The potential risks and disadvantages of dynamic scoring are that:

● providing the same challenges to a group of players with different experience could give different results; ● malicious players could register additional fake users and solve a particular challenge in order to decrease the total number of points, providing an unfair advantage to other players who solve other challenges.

Lesson learned. ENISA recommends considering the use of dynamic scoring for any type of jeopardy competition but also keeping in mind the potential risks.

3.5.5. Challenges

The Hackfest Consortium set 26 challenges with 2 backups (14 previously used and 12 new challenges) for the event. No significant disruption of infrastructure was detected during the competition, but certain Layer 7-oriented attacks (mostly denial of service) were addressed.

Players who had previously solved the reused challenges found this situation unattractive and their tendency was to search for write-ups or use their previous solutions, rather than trying to solve the challenge again. However, many players did prefer to try to solve the challenges again and actually learn from this experience.

Lesson learned. ENISA recommends not reusing challenges in future.

3.5.6. Communication

The consortium took an innovative approach to communication this year, using an official Discord group with several communication channels.

Access to each channel was granted by a custom-made bot developed by the consortium. The bot granted access to channels to players based on invitation links, in accordance with predefined user roles (player, country coordinator and organiser). Moreover, the bot monitored communication between players and removed any message that potentially contained a flag or part of it.

The benefits were:

● voice chat, which allowed networking between players; ● flexible application programming interfaces (APIs), to enable the creation of bots with different levels of capabilities; ● very good and granular user access.

The disadvantages were that:

● the bot developer needed to introduce failover/healing features, since Discord APIs can have downtime quite often. ● voice chat could be used for flag sharing, bypassing the bot’s control.

18

ECSC 2020 ANALYSIS REPORT

Lesson learned. Discord channels can be an alternative to Telegram or Slack for this type of event; however, certain issues must be taken into consideration.

Figure 4: The ENISA Hackfest 2020 platform

3.5.7. Support

In order to provide 24/7 technical support for 48 hours, the consortium used the following mechanisms.

● The challenge authors were available during the event and they answered questions when addressed. ● Each question answered was added to a common frequently asked questions (FAQs) document, which allowed the support team to answer common questions more quickly and focus on new or critical situations. ● During the night, at least one person was available at any point and was able to respond to all questions received.

Lesson learned. Developing a live FAQs document for challenges can speed up support resolutions during CTF competitions, which are quite intensive and exhausting for support teams if the event happens over several days.

Feature improvement. Another potential improvement would be adding a bot that can manage tickets, which could speed up the process, since a player would no longer have to wait for the identification of a staff member who was available to answer their question.

3.5.8. Complaints

Players and country coordinators had channels where they could make complaints. During the event, most of the conversations were challenge-related. The only question that could be construed as a complaint was related to the availability of rankings for individual players, so that participants could see how their country was situated in comparison with other countries.

19

ECSC 2020 ANALYSIS REPORT 3.5.9. Infrastructure and platform

The event was hosted on a cloud-based platform (cyberEDU.ro) that can host large CTF competitions in various formats. The platform and infrastructure had 100 % uptime during the event, with some minor issues during the first hour relating to reporting of results to country coordinators and organisers, which were immediately addressed by the consortium.

The benefits of using the platform, rather than other open-source/standard technologies, are as follows.

● Management of the competition. The platform allowed the consortium to set up and manage the event in minutes, with various custom fields at registration, multiple-level access (i.e.. organiser, teacher/country coordinator and player levels), dynamic scoring and more. ● Scalability. Since the entire application uses Kubernetes, the infrastructure can be scaled as needed for the existing load with no downtime recorded. ● Moderation. Having moderation support for registration allowed country leaders to accept or reject players. ● Self-healing challenges. The platform allowed players to vote for restart when a challenge seemed to be down or malfunctioning, and it automatically restarted the challenge when the system detected suspicious delays in response.

Lessons learned are the following.

● Self-healing challenges. These proved to be an improvement in terms of reducing staff effort required to manage the infrastructure; staff were able to solve downtime and malfunctioning from their phones. ● One-to-one resources for players. To further improve the experience of players, the platform used for future editions of the event should allow each player to run and restart their own instance of each challenge.

3.5.10. Miscellaneous lessons learned and recommendations

● Some players complained that organising the event over a weekend would have allowed them more involvement in the competition. ● Some players complained about the lack of prizes/stakes, sometimes stating that this was one of the main reasons for limited numbers of players representing their countries. ● Some players recommended not reusing challenges in future. ● Some players complained that participation by women needed to be considerably increased. ● The consortium recommends considering the possibility of (distributed) denial of service attacks on the infrastructure (the CTF platform and the challenges infrastructure) when creating such events, especially competitions with publicly available access, and enabling various security measures including: ○ backup infrastructure; ○ anti-denial of service attack solutions (e.g. websites should have Cloudflare or other traffic-filtering technologies in place); ○ auto-scaling infrastructures to address higher loads; ○ available response teams that can monitor the load and/or attacks and can respond properly in minutes; ○ dedicated resources for each player or for a number of players; ○ controlling access to infrastructures with tunnelling technologies such as OpenVPN/Wireguard.

20

ECSC 2020 ANALYSIS REPORT 3.5.11. Public affairs and communications

After the competition, ENISA created and sent to the ECSC mailing list a document intended to align communication activities with all media points of contacts in the various ECSC participant countries.

The document contained the following information:  links and guidelines on sharing information (text, hashtags, etc.),  a link to an ENISA news item (https://www.enisa.europa.eu/news/enisanews/enisa2019s-48h-hackfest-puts-europe2019s-cybersecurity-talent-to-the-test),  social media links: o Twitter post 1 (https://twitter.com/enisa_eu/status/1329365661928177664), o Twitter post 2 (https://twitter.com/enisa_eu/status/1329381551575019520), o Twitter post 3 (https://twitter.com/enisa_eu/status/1329381843263746048/photo/1), o Facebook post (https://www.facebook.com/ENISAEUAGENCY/posts/3108688432569452), LinkedIn post (https://www.linkedin.com/posts/european-union-agency-forcybersecurity-enisa_enisahackfest-cybersecurity-eu-activity- 6735131891386658816-g3wi),  statistics (some general information about the participation in the event),  the competition logo,  a screenshot of the platform,  visual material for social media, developed by ENISA’s public affairs team, as shown in Figure 5.

21

ECSC 2020 ANALYSIS REPORT

Figure 5: Social media material on ENISA Hackfest 2020

3.6. PLATFORMS

The updated list of ECSC-related platforms is the following:

 test scoreboard and contracts platform (only available from a certain point before the competition) (https://board.ecsc.eu),  test scoreboard code repository (https://github.com/enisaeu/ecsc-gameboard),  file-sharing platform (ownCloud) used for information sharing and as a contingency mechanism (https://storage.ecsc.eu/),  the ECSC website used to promote the event and provide real-time scoring information during the challenge to external interested parties (https://www.europeancybersecuritychallenge.eu/),

22

ECSC 2020 ANALYSIS REPORT

 the challenges platform on which previous years’ ECSC challenges are collected (https://challenges.ecsc.eu/),  the collaboration platform (https://cermit.enisa.europa.eu/ui/welcome).

During 2020, two new platforms were added to support ECSC activities, the ECSC challenges platform and the collaboration platform.

3.6.1. ECSC challenges platform

The idea was to create a simple, public-facing platform to provide access to all the challenges from past ECSCs and others that were not used during past events and could be used for training.

Figure 6: The ECSC challenges repository

3.6.2. ECSC collaboration platform

The objectives of this platform are to improve the exchange of information among ECSC Steering Committee members and to provide a single place to store all ECSC-related information.

The selected platform, Cermit/CIRCABC, is supported and maintained by the European Commission and uses the European Commission Authentication Service (ECAS) as an authentication service.

23

ECSC 2020 ANALYSIS REPORT

Figure 7: The ECSC collaboration platform

3.7. Public affairs and media activities

The following key media activities were carried out during 2020 in collaboration with the various media points of contact appointed from the ECSC Steering Committee:  coordinating the communication of the cancellation of the ECSC 2020 final,  coordinating the announcement of the ENISA Hackfest 2020 post-event activities,  updating attendees at the ECSC hot-wash meeting on ECSC public affairs activities. Figure 8: Update on ECSC public affairs activities presented at the ECSC hot-wash meeting

24

ECSC 2020 ANALYSIS REPORT

Figure 9: Phases of ECSC public affairs activities

3.8. WORKING GROUPS

During 2020, three new working groups were created, two of them as per the ECSC Steering Committee’s decision during the initial planning conference in Vienna.

3.8.1. International working group

This group is intended to support the creation of Team Europe for the ICSC, planned for December 2021. During 2020, it carried out the following actions.

 The working group created the mandate for the ICSC Subcommittee, which was approved by the ECSC Steering Committee on 20 July 2020. The aim of the document is to set out the delegation of responsibilities from the ECSC Steering Committee to the subcommittee, and in particular to specify: o the composition and the means of acceptance of the subcommittee; o the tasks, purpose, role, boundaries and responsibilities of the subcommittee; o the duration of the subcommittee’s mandate and the means of renewal.  It created a document on the trainer selection process (‘Coaches requirements and selection’), which was approved by the ECSC Steering Committee on 20 November 2020. The aim of the document is to set out the process for selecting the trainers for the future Team Europe, and in particular to specify: o the composition of the team of trainers and the means of selection; o the goals for Team Europe; o requirements for the trainers; o tasks, role, boundaries and responsibilities of the trainers; o the selection process; o the duration of the trainers´ mandate and the means of renewal.

25

ECSC 2020 ANALYSIS REPORT

 The working group created the call for trainers, launched on 23 November 2020.  It proceeded with the final selection of trainers, which was completed on 11 December 2020.  It started discussions about the selection mechanisms and the training path, in coordination with ENISA.

Figure 10: Relations between the international and European committees

3.8.2. Soft skills development working group

This working group was created to discuss and propose new ideas to replace the presentations as a soft skills development mechanism during the finals.

A space on the collaboration platform was created for this group; however, owing to the COVID- 19 pandemic and the cancellation of the 2020 edition, no relevant decisions were made on this topic.

3.8.3. Media and public affairs points of contact working group

This working group was created to address the identified need to establish a permanent coordination group for the alignment of public affairs, communication and media activities.

This list of contact points is expected to improve timeliness and collaboration with regard to communication activities.

The group was started to work on the communication of the cancellation of the 2020 final and on communication activities relating to ENISA Hackfest 2020.

26

ECSC 2020 ANALYSIS REPORT

4. ECSC 2020 – LESSONS LEARNED AND MATURITY ASSESSMENT

A report on lessons learned has been produced by an external contractor on behalf of ENISA. It builds to some extent on the report on lessons learned from ECSC 2019, for the sake of consistency, and highlights comments and views provided by a number of ECSC Steering Committee members and ECSC representatives recommended by ENISA, as well as by several ENISA Hackfest 2020 organisers and participants.

Figure 10: Process of building the lessons learned

The report aggregates and puts into perspective a number of comments and views from stakeholders who were prompted to reflect on Hackfest 2020 and on the maturity of the ECSC. This feedback was collected through the following methods.

 Two online EUSurvey-based questionnaires, which combined closed and open-ended questions, and which were intended respectively for participants in Hackfest 2020 (defined as the individuals who were selected at national level and joined the event to try to solve the challenges) and for the organisers of Hackfest 2020 (understood in a generic way and referring to the people behind the organisation of the Hackfest event, including ECSC Steering Committee members). In total, 56 people responded to the questionnaires, distributed as follows: 47 participants and 9 organisers.  A series of phone, Microsoft Teams or GoToMeeting interviews with ECSC Steering Committee members (or ECSC representatives acting as deputies) recommended by ENISA. In total, six interviews with eight people were conducted.

27

ECSC 2020 ANALYSIS REPORT

It is important to note that ENISA acted only as an intermediary in the data collection process. It provided support to encourage participation in the surveys and helped in identifying ECSC Steering Committee members to contact, but it refrained from any participation in the interviews in order to ensure the neutrality of the discussion.

Although the intention was to replicate the approach and methodology adopted for the maturity assessment conducted in 2019, the focus of which was ECSC 2019 (held in Bucharest, Romania), this report on lessons learned had to introduce some changes to the methodology owing to the specific circumstances caused by the COVID-19 pandemic, namely:

 no field research or direct observation-based inputs as part of the collected qualitative data;  revision of the high-level domains because of the irrelevance of those related to venue and logistics;  revision of some of the objectives to account for the change in event (Hackfest 2020 instead of ECSC 2020);  adoption of a distinction between a general perspective (still appropriate for governance and decision-making and public affairs high-level domains) and a focus on Hackfest 2020 (for challenges and compliance).

For the sake of consistency, the high-level domains, parameters and evaluation criteria previously defined have been kept for this report. The change of scope, with Hackfest 2020 replacing ECSC 2020, has, however, rendered it impossible to conduct a maturity assessment directly comparable to the 2019 maturity assessment.

The following domains were evaluated:

1. governance and decision-making aspects, 2. public affairs – general perspective, 3. challenges – focus on Hackfest 2020, 4. logistics – aspects of the competition related to the venue, catering, hotels, transportation, etc., 5. side events – aspects of the competition related to social events and networking meetings, 6. compliance – aspects of the competition related to compliance with laws and standards.

28

ECSC 2020 ANALYSIS REPORT 4.1. GOVERNANCE AND DECISION-MAKING ASPECTS

Feedback from Hackfest / maturity assessment, Parameter Objective questionnaires and interviews

29

ECSC 2020 ANALYSIS REPORT 4.2. PUBLIC AFFAIRS – GENERAL PERSPECTIVE

Feedback from Hackfest / maturity assessment and Parameter Objective questionnaires

Monitoring,

measurement

and analysis

30

ECSC 2020 ANALYSIS REPORT

Dissemination

plan

Key

Messages

and reach

31

ECSC 2020 ANALYSIS REPORT

Social media

and visibility

Website

4.3. CHALLENGES – FOCUS ON ENISA HACKFEST 2020

Feedback from Hackfest / maturity assessment and Parameter Objective questionnaires

32

ECSC 2020 ANALYSIS REPORT

parameter

specifically to the ECSC)

33

ECSC 2020 ANALYSIS REPORT 4.4. COMPLIANCE – FOCUS ON ENISA HACKFEST 2020

Parameter Objective Observations

34

ECSC 2020 ANALYSIS REPORT

5. AREAS FOR IMPROVEMENT AND RECOMMENDATIONS

5.1. GOVERNANCE AND DECISION-MAKING ASPECTS

Parameter Recommendations

ECSC Steering

5.2. PUBLIC AFFAIRS – GENERAL PERSPECTIVE

Parameter Recommendations

and analysis (KPIs)

35

ECSC 2020 ANALYSIS REPORT 5.3. CHALLENGES – LESSONS LEARNED FROM ENISA HACKFEST 2020

Parameter Recommendations

Design • Ensure enough diversity in the challenges. • A ‘very good challenge’, according to one of the respondents to the participants’ questionnaire would meet the following requirements: o very well designed

o a learning experience o innovative

o fair o fun.

• Try to harmonise flags’ formats, which can vary a lot depending on the supplier.

• Enhance the cooperative mindset and the European ‘branding’. • Small countries (e.g. the five smallest ones) should be allowed to cooperate against major teams. • Consider adding some boot2root challenges and create activities that members of teams from different countries are expected to collaborate on, instead of their having to compete against each other for the duration of the event. • Even if the event has to become virtual because of the COVID-19 pandemic, find ways to encourage interaction and collaboration.

36

ECSC 2020 ANALYSIS REPORT

Rules • Implement a section on the ECSC platform that details the ECSC rules and the good conduct expected of the participants. • Find a way to display the Charter of Good Conduct during the event. • Something as simple as a terminology brief describing the roles, functions and responsibilities of the various people taking part in ECSC events could be helpful in making people aware of the rules, especially those that apply more particularly to them depending on their function (captains, coaches, trainers, qualifiers, organisers, etc.). • The choice of a channel for communication among participants should be made in advance and thought should be put into how communication will be structured (e.g. who should have access to what, which spaces should be created). Once it has been decided on, registration with the communication channel could be made compulsory.

Enforcement • Identify a platform or a channel (within Discord, for instance) where an incidentreporting space could be created for participants to notify the jury of potential issues, such as cheating or unethical behaviour. • If using a Discord-type channel, check that participants do not use it to exchange flags. • Consider publishing exchanges on a Discord-type channel for transparency purposes.

Presentations • Explain clearly what the requirements for the presentations are and what the level of technical expertise of the jury panel is.

(NB: this

• Post an agenda giving the order and timing of the presentations on the ECSC

parameter

platform and website.

relates specifically to the ECSC)

Complexity • Participants are more and more demanding about the quality and complexity of challenges and tend to expect that considerable thought and care will have been put into their creation, for example by forensics professionals. • Do not reuse challenges, to ensure that no write-ups are available. • Avoid challenges that involve guessing or that are not originals. • Create more education-oriented challenges. • Check carefully for misspellings.

Scoring • Ensure that challenges that require a lot of work to find the solution are worth more than challenges that require guessing or random algorithms to be solved.

5.4. COMPLIANCE – LESSONS LEARNED FROM ENISA HACKFEST 2020

Parameter Recommendations

Data protection • Do not require participants to have a Google account to submit write-ups. • Do not display full names on the scoreboard.

37

ECSC 2020 ANALYSIS REPORT

6. ECSC 2021

The final of the 2021 edition of the ECSC will take place in Prague in September 2021; at least 22 countries are expected to participate. The latest updates will be published on the ECSC 2021 website (https://www.europeancybersecuritychallenge.eu/). Figure 11: The ECSC 2021 logo

38

-N -EN -123 -21 -03 TP

About ENIS A

The European Union Agency for Cybersecurity, ENISA, is the Union’s agency dedicated to achieving a high common level of cybersecurity across Europe. Established in 2004 and strengthened by the EU Cybersecurity Act, the European Union Agency for Cybersecurity contributes to EU cyber policy, enhances the trustworthiness of ICT products, services and processes with cybersecurity certification schemes, cooperates with Member States and EU bodies, and helps Europe prepare for the cyber challenges of tomorrow. Through knowledge sharing, capacity building and awareness raising, the Agency works together with its key stakeholders to strengthen trust in the connected economy, to boost resilience of the Union’s infrastructure, and, ultimately, to keep Europe’s society and citizens digitally secure. More information about ENISA and its work can be found on its website www.enisa.europa.eu.

ISBN: 978-92-9204-496-1 DOI: 10.2824/284260