BEREC Guidelines detailing Quality of Service Parameters

7 March 2024 Introduction

According to Article 4 of the BEREC Regulation , BEREC shall issue guidelines on the implementation of the European Union regulatory framework for electronic communications, as referred to in Directive (EU) 2018/1972 (EECC), on, among other things, relevant quality of service (QoS) parameters the applicable measurement methods, the content and format of publication of the information, and quality certification mechanisms, in accordance with Article 104 of EECC. In this context, in 2020, BEREC issued Guidelines detailing QoS parameters (hereafter Guidelines) to provide guidance to National Regulatory Authorities (NRAs) with respect to Article 104 of the EECC, taking into account, where appropriate, the parameters, definitions and measurement methods set out in Annex X. Section 7 of the previous Guidelines lays down a review period to commence 2 years from their adoption and publication. On that basis, BEREC is revising the Guidelines. The rationale for initially issuing the Guidelines as well as their update is to provide guidance to NRAs in respect to Article 104 of the EECC and to contribute to the consistent application of Article 104(2) and Annex X, with the aim of defining:

1. The relevant QoS parameters in relation to Interpersonal Communications Services (ICS) and Internet Access Service (IAS),

2. The parameters relevant for end-users with disabilities,

3. The applicable measurement methods for these QoS parameters, including, where appropriate, the ETSI and ITU standards set out in Annex X of the EECC in relation to ICS and IAS, respectively,

4. The content and format of the QoS information to be published, and

5. Possible quality certification mechanisms. BEREC, within its strategic priority 3: “Empowering end-users”, included in its Work Programme WP 2023 the task to update BEREC Guidelines detailing QoS parameters, according to points 1 and 2 above. Definitions used in this document follow Article 2 of the EECC, unless otherwise stated. (See also Annex 2 of the Guidelines). For the purposes of the revision of the Guidelines, a questionnaire was sent to the NRAs to collect their current practices as regards points 2 and 3 above. In addition to the questionnaire for the NRAs, the early call for stakeholders was published in order to request input from the market. Although most of the responses received were in the direction of needing more time to gather important information, early notification contributes to 2 good preparation for stakeholders for public consultation and also for BEREC to finalize Guidelines, to ensure consistency and effectiveness.

1 Policy principle, legal basis and scope of the BEREC Guidelines

1.1 Policy principle

1. In the ever-connected, globalized, digital environment that is developing faster and faster, electronic communication services play a key role in citizens’ everyday activities. As set out in the QoS regulation manual ITU 2017 pgs. 5 & 110 – “The profusion of ever-evolving technologies, networks, services and devices with different QoS capabilities further ads to the complexity of regulation in this area. Quality can be impacted by many factors at the network level and along the value chain. In this regard, a common approach to regulating QoS can enable greater quality prospects irrespective of the locations of the consumer and service provider”. Indeed, the correlation between quality and pricing of services suggests: “If the demand for services that require high QoS is very low (compared to demand for services not requiring QoS), then the willingness to pay for high QoS will be also very low. In such a case, telecommunication operators (which are in fact the ISPs nowadays) will have lower interest in QoS. When the demand for services that require high QoS is comparable with demand for services not requiring QoS, then the willingness to pay for QoS is higher”. 2. The QoS, as perceived by the end-user, is a crucial factor for both customers and service providers and, with the profusion of ever evolving technologies, networks and services with different levels of QoS, it is becoming increasingly more complex to manage, measure and regulate it. Indeed, quality can be impacted by many factors at the network level and along the value chain, including the device, hardware, infrastructure, service and applications . Hence, the QoS, that the provider is having control of, might be good or as declared, but the Quality of Experience (QoE) might be lower due to other aspects. As it was stated in the BEREC Report on Internet Ecosystem , users can assess whether the quality of their internet experience is rather good or bad, but it may be extremely difficult for such a user to assess whether e.g., quality issues when streaming videos are caused by an IAS provider’s open internet violations or result from IP interconnection disputes. 3. Regulatory development in the European electronic communications sector is intended to help improve the end-user experience, to lead to greater competition and investment, and to benefit all the different players in the digital ecosystem. This 3 development has resulted in the EECC and BEREC undertaking complex work aimed, inter alia, at achieving one of the very clear objectives of the EECC, i.e., empowering and protecting end-users. 4. The European harmonisation of QoS parameters for data collection and publication practices would result in substantive benefits, such as enabling comparability among Member States and providing better information on the European electronic communications market, while at the same time promoting the consistent application of regulatory obligations and improving transparency for end-users and public authorities in relation to QoS.

1.2 Legal basis and related BEREC documentation

5. The rationale of the Guidelines is, therefore, to contribute to a consistent and harmonized application of the provisions of Article 104 of the EECC .

6. More specifically, Article 104(1) of the EECC provides that NRAs in coordination with other competent authorities may require providers of IAS and of publicly available ICS to publish comprehensive, comparable, reliable, user-friendly and upto-date information for end-users on the quality of their services and on measures taken to ensure equivalence in access for end-users with disabilities.

7. Pursuant to information requirements for contracts set out in Article 102 and Annex VIII of the EECC, end-users should be informed, inter alia, of the different levels of the QoS, conditions for promotions and termination of contracts, applicable tariff plans and tariffs for services subject to particular pricing conditions.

8. At the same time, Recital 271 of the EECC provides that NRAs in coordination with other competent authorities should be empowered to monitor the QoS and to systematically collect information about the QoS offered by providers of IAS and of publicly available ICS, to the extent that the latter are able to offer minimum levels of service quality either through control of at least some elements of the network or by virtue of a service level agreement (SLA) to that end, including the quality related to the provision of services to end-users with disabilities. That information should be collected on the basis of criteria which allow comparability between service providers and between Member States. Providers of such electronic communications services, operating in a competitive environment, are likely to make adequate and up-to-date information on their services publicly available for reasons of commercial advantage. NRAs in coordination with other competent authorities should nonetheless be able to require publication of such information where it is demonstrated that such information is not effectively available to the public, including information on equal access and choice for end-users with disabilities.

9. Moreover, according to Article 104(1) of the EECC, NRAs in coordination with other competent authorities may require providers of internet access services (ISPs) and of publicly available interpersonal communications services to publish 4 comprehensive, comparable, reliable, user-friendly and up-to-date information for end-users on the quality of their services, to the extent that they control at least some elements of the network either directly or by virtue of a service level agreement to that effect, and on measures taken to ensure equivalence in access for end-users with disabilities. NRAs in coordination with other competent authorities may also require providers of publicly available interpersonal communication services to inform consumers if the quality of the services they provide depends on any external factors, such as control of signal transmission or network connectivity. In accordance with Article 104(2) of the EECC, NRAs in coordination with other competent authorities shall specify, taking utmost account of the Guidelines, the QoS parameters to be measured, the applicable measurement methods, and the content, form and manner of the information to be published, including possible quality certification mechanisms, using, where appropriate, the parameters, definitions and measurement methods set out in Annex X of the EECC.

10. In order to facilitate comparability across the European Union and to reduce compliance cost, according to Recital 272 of the EECC, BEREC should adopt guidelines on relevant QoS parameters which NRAs in coordination with other competent authorities should take utmost account of.

11. Moreover, Article 104(1) provides that “The measures to ensure quality of service shall comply with Regulation (EU) 2015/2120”. Article 4(1) of Regulation (EU) 2015/2120 contains specific transparency obligations for providers of IAS. In the BEREC Guidelines on the Implementation of the Open Internet Regulation adopted in 2022 , BEREC has further explained the transparency obligations contained in Article 4(1), first sentence, (a)-(e).

12. Focus on the QoS is also put in Roaming Regulation 2022/612, which establishes the requirement for the transparency of the information related to QoS while roaming. Given this, QoS is the important criterion not only for the end-user using services domestically, but within all the European Union, hence, the consistent application of parameters has become even more important.

13. The Guidelines focus on providing definitions and measurement methods for the IAS QoS parameters related to the network performance listed in Annex X of the EECC, i.e. latency, jitter and packet loss, taking into account previous BEREC reports, especially BEREC Net Neutrality Regulatory Assessment Methodology BoR (22) 72 , in order to ensure the consistency of BEREC’s documents.

14. Additional guidance can be found in the BEREC reports: Monitoring quality of Internet access services in the context of net neutrality BoR (14) 117 ; Net 5 neutrality measurement tool specification BoR (17) 179 and BoR (18) 32 Annex 1 .

1.3 Scope of the BEREC Guidelines

15. In light of the above, the Guidelines, in pursuing the goal of providing transparency to consumers on QoS, provide assistance to NRAs on the QoS parameters that NRAs in coordination with other competent authorities could specify for measurement by the providers, together with the applicable methodologies, the information to be published and the possible quality certification mechanisms.

16. Furthermore, QoS can be distinguished from QoE as QoS concerns the network and terminal equipment up to the user interface while QoE focuses on the entire service experience and includes the whole path from user to user including the enduser expectation, perception and context of use. QoS also includes the assistive equipment and the specific services provided to end-users with disabilities where equipment is provided by the ICS Provider. For more details on QoE see ITU-T Rec P.10/G.100 . Network performance (NP) is more limited in scope because it excludes terminal performance. Figure 1 shows the relationship between these terms. For the purpose of these Guidelines only QoS is taken into consideration.

17. The Guidelines focus on QoS parameters related to ICS and IAS as well as the corresponding measurement methods and certification mechanisms.

18. ICS can be provided by different technological means. Firstly, in the event where the provider has control over network elements (e.g., because he owns the network) or has an SLA with a network operator - it is possible for the provider of the ICS to give information on QoS parameters. For example, in the case of voice communications services which are provided as “traditional” (i.e. non-Over The Top 6 – OTT) telephony services or as specialised services (i.e. managed services), it is possible for the provider to indicate the corresponding QoS parameters in the case where voice connections are originated and terminated within the providers network while end-users use specific terminal equipment. In this scenario, a provider is obliged to fulfill the information requirements set out in Article 104(1) of the EECC, if an NRA in coordination with other competent authorities requires such.

19. Secondly, when the provider has neither control over network elements, nor has an SLA to that effect: this situation may arise if the interpersonal communication services are provided over the internet, e.g., number-independent ICS (NI-ICS). In this event, the quality of the ICS depends on the quality of the IAS and terminal equipment used. For example, a provider of a messaging service which also has a voice communications service functionality cannot indicate the QoS of the voice call because the quality of the voice call is influenced by the underlying IAS and terminal equipment used. According to Article 104(1) of the EECC, an NRA in coordination with other competent authorities may require the provider of the NI-ICS to inform consumers if the quality of the services they provide depends on any external factors, such as control of signal transmission, network connectivity and terminal equipment. If the NRA in coordination with other competent authorities requires so, a NI-ICS provider is obliged to inform consumers that the voice communications service quality depends, e.g., on the quality of the underlying IAS and the terminal equipment used. However, the NI-ICS provider cannot make a statement or guarantee a QoS as this is outside the area of his control.

20. Different standards and recommendations have been defined to detail methodologies to measure QoS of ICS and IAS. The measurement methods specified by NRAs in coordination with other competent authorities should be based, where appropriate, on standards or other documentation as set out in Tables 1, 1A, 2 and 3 of the Guidelines.

21. It should be noted that the measures, which an NRA in coordination with other competent authorities may define under Article 104(1) of the EECC and under Article 5 of Regulation (EU) 2015/2120 shall be coherent .

22. IAS and ICS providers should assess all factors that may impact the QoS levels available to end-users, for example, user environment or the bias brought by the location of test servers or interconnection issues, etc. Where possible, providers should take these factors into consideration during the measurement process.

23. This review will identify possible issues when applying the Guidelines, to and necessary updates considering the development of technologies, services and associated QoS parameters in recent years. This will be substantiated by the data collected from the NRAs taking into account their experience from the implementation of the Guidelines and therefore their usability and identify whether additional QoS parameters are necessary considering the industry’s evolution. 7

2 QoS Parameters & Measurement Methods for Interpersonal Communication Services and Internet Access Services

2.1 Parameters and measurement methods

24. According to Figure 1 as set out in section 2 of this document and referenced by the EC in its Report “Fixed and Mobile Convergence in Europe – Quality Measurements for 5G and Network Densification” QoS refers to “the effectiveness of performance of a system in support of end-user needs or that contributes positively to another system’s performance”. Furthermore, quality of emergency communications, according to the EC report “Study on technical solutions to ensure compatibility, interoperability, quality, reliability and continuity of emergency communications in the Union , “may be defined as the characteristics of the emergency communication that bear on its ability to satisfy the requirements related to the operational needs of emergency services, in particular i) swift communication with the most appropriate PSAP and ii) making available in a timely manner the contextual data, including caller location information”.

25. Αs set out in Annex 3 to the Guidelines, an extract of the most widely mandated existing QoS indicators across Member States include the following: • Voice - Call set-up time; Unsuccessful call rate; Speech transmission quality; Response time for calls to the operator, customer service and directory assistance; • Mobile - Network availability; Probability of successful connection in an area covered by the network; Dropped call ratio; • Customer service - Time between request for service and start of service; Fault frequency; Time to troubleshoot & eliminate faults; Frequency of complaints about billing; • Internet - Data transmision speed (upload and download); Latency (delay); Jitter; Packet loss rate.

26. Depending on the nature of the content to be exchanged (e.g., audio, video, text, data), different ICS quality parameters need to be specified. ETSI EG 202 057 multipart deliverable standards (see Annex X of the EECC) provide guidance on the basic approach to be applied by the providers to assess the various aspects of quality.

27. Providers of NI-ICS and NB-ICS as well as any other provider of electronic communication services cannot know and influence the technical characteristics of interconnected networks and terminal equipment used at the endpoints of the communication when these are not provided or selected by the providers of NI-ICS 8 and NB-ICS. Thus, providers can only specify estimates of the resulting communication quality of actual end-to-end communications. However, such providers are only subject to Article 104 of the EECC in so far as they control parts of the network or have an SLA with a network operator to that effect.

28. Typically, NI-ICS are designed to compensate for the varying transport quality of packet switched networks and the best effort packet forwarding principle. They do not have stringent requirements for network quality and just require an adequate overall performance level (e.g., a maximum delay value not to be exceeded for realtime communication).

29. Table 1 below lists QoS parameters, definitions, and measurement methods from Annex X of the EECC which shall be used, where appropriate, for ICS.

30. For completeness, BEREC has proposed ETSI definitions and measurement methods for two QoS parameters set out in Annex X of the EECC (the call set up failure probability parameter and the call signalling delays parameter), which currently do not have definitions and measurements methods provided in the Annex X, and which shall be used by NRAs in coordination with other competent authorities where appropriate (see Table 1 below).

31. Table 1A below lists QoS parameters from Annex X of the EECC for IAS together with definitions and measurement methods from ITU, IETF and BEREC Net Neutrality Regulatory Assessment Methodology BoR (22) 72 .

32. Table 2 below lists QoS parameters (Response time for operator services, Customer complaints resolution time, Number of customer complaints per data collection period and Data transmission speed (upload and download)) which are not provided in Annex X of EECC but are of interest to the end-users. Article 104 of the EECC contains a specific reference to Regulation (EU) 2015/2120: “the measures to ensure quality of service shall comply with Regulation (EU) . 2015/2120”

33. Annex X of the EECC contains some basic QoS parameters and measurement methods for IAS. Concerning network performance, besides latency (delay), jitter and packet loss which are included in the Annex X of the EECC, data transmission speed (upload and download) governed by Regulation (EU) 2015/2120, and other parameters might be also of interest for end-users.

34. In these Guidelines BEREC has considered measurement methods as set out in BEREC Net Neutrality Regulatory Assessment Methodology BoR (22) 72 .

35. It is important to note here that there is a degree of flexibility allowed when deciding which QoS parameters are to be measured by the providers and therefore should be specified by NRAs. To this end, NRAs in coordination with other competent authorities are free to choose among the QoS parameters listed in Table 1 and 9 Table 1A, those that are appropriate, taking into account national circumstances and other factors, such as, the meaningfulness and usefulness of the parameter, the underlying costs, time needed to implement the measurement and possible monitoring systems, changes required to adapt and modify current methodologies and allowing for the possibility of comparing new results with previous records. NRAs are therefore not obliged to specify the full list of parameters contained in Table 1 and Table 1A but can choose the ones that are particularly relevant for the needs of their country. For example, web page loading time, which is quality of experience parameter (QoE) and the operator providing the IAS service cannot have a full control of this parameter. Where NRAs in coordination with other competent authorities choose to impose relevant and appropriate QoS parameters from Table 1 and Table 1A, they shall take utmost account of the Guidelines and of the definitions and the measurement methods listed in Table 1 and Table 1A.

Table 1 QoS Parameters as set out in Annex X of the EECC

QoS Definition Measurement method Parameters Annex X

Supply time ETSI ES 202 057-1 (clause 5.1) ETSI ES 202 057-1 (clause 5.1.3) for initial The duration from the instant of a It is measured by: connection valid service order being received a) the times by which the fastest by a direct service provider to the 50%, 95% and 99% of orders are instant a working service is made completed; available for use. This should exclude cancelled orders. b) the percentage of orders completed by the date agreed Applicable to both fixed and mobile with the customer and, where the services. percentage of orders completed by the date agreed with the customer is below 80%, the average number of days, for the late orders, by which the agreed date is exceeded. Statistics for both fixed and mobile access networks.

10

QoS Definition Measurement method Parameters Annex X

Fault rate per ETSI ES 202 057-1 (clause 5.4) ETSI ES 202 057-1 (clause 5.4.3) access line The number of reported faults per Statistics for all fixed access lines. fixed access line per year.

Fault repair ETSI ES 202 057-1 (clause 5.5) ETSI ES 202 057-1 (clause 5.5.3) time The duration from the instant a fault It is measured by: report has been made to the instant a) the time by which the fastest 80% when the service element or and 95% of valid faults on access service has been restored to lines are repaired (expressed in normal working order. clock hours); Applicable to fixed services only. b) the percentage of faults cleared any time stated as an objective by the service provider; c) the provision of information on the hours during which faults may be reported. Statistics for all access fixed networks.

Call setup ETSI EG 202 057-2 (clause 5.2) ETSI EG 202 057-2 (clause 5.2.3) time The call set up time is the period It is measured by: starting when the address a) the mean value in seconds for information required for setting up a national calls; call is received by the network and finishing when the called party busy b) the time in seconds within which tone or ringing tone or answer the fastest 95% of national calls signal is received by the calling are set-up; party. c) the mean value in seconds for Where overlap signalling is used international calls; the measurement starts when d) the time in seconds within which sufficient address information has the fastest 95% of international been received to all the network to calls are set-up; begin routeing the call. e) the number of observations

11

QoS Definition Measurement method Parameters Annex X

performed for national and international calls. Applicable to both fixed and mobile calls. Statistics for both fixed and mobile voice services.

3GPP TS 32.454 clause 5.1.2 3GPP TS 32.454 clause 5.1.2 Session setup time It is measured by the mean value Applicable for IMS (VoLTE KPI) The mean setup time of the sessions

Bill ETSI ES 202 057-1 (clause 5.11) ETSI ES 202 057-1 (clause 5.11.3) correctness The proportion of bills resulting in a It is measured by a percentage. complaints customer complaint about the correctness of a given bill per service. Applicable to both fixed and mobile services.

Voice ETSI EG 202 057-2 (clause 5.3) ETSI EG 202 057-2 (clause 5.3.2) connection ETSI TR 102 506 Statistics for: quality Evaluation of speech quality per - Fixed to fixed calls call. - Fixed to mobile calls The end-user perceived voice - Mobile to fixed calls quality. - Mobile to mobile calls

Applicable to fixed and mobile voice ITU-T G.1020: Performance services. parameter definitions for quality of speech and other voice band applications utilizing IP networks.

ITU-T G.1028: End-to-end quality of service for voice over 4G mobile networks.

12

QoS Definition Measurement method Parameters Annex X

ITU-T P.863: Perceptual objective listening quality prediction.

Dropped call ETSI EG 202 057-3 (clause 6.4.2) ETSI EG 202 057-3 (clause 6.4.2.2) ratio The proportion of incoming and When using the measurements outgoing calls which, once they based on network element counters, have been correctly established the following statistics should be and therefore have an assigned provided: the percentage of dropped traffic channel, are dropped or calls, calculated from all the calls in interrupted prior to their normal the period. completion by the user, the cause When using test calls, the following of the early termination being within statistics should be provided: the the operator's network. percentage of dropped calls, Applicable to mobile networks. together with the number of observations used and the absolute accuracy limits for 95% confidence calculated from this number.

3GPP TS 32.454 clause 5.2.1 3GPP TS 32.454 clause 5.2.1 Call drop for IMS session It is measured by a percentage. Applicable for IMS (VoLTE KPI) The number of dropped sessions divided by the number of successful session establishments.

Unsuccessful ETSI EG 202 057-2 (clause 5.1) ETSI EG 202 057-2 (clause 5.1.3) call ratio Unsuccessful call ratio is defined as It is measured by: the ratio of unsuccessful calls to the a) the percentage of unsuccessful total number of call attempts in a calls for national calls; specified time period. b) the percentage of unsuccessful Applicable for both fixed and mobile calls for international calls; networks. c) the number of observations used for national and international calls together with absolute accuracy.

Call set up ETSI TS 102 024-9 (clause 4.1.1) ETSI TS 102 024-9 (clause 4.1.1) failure

13

QoS Definition Measurement method Parameters Annex X

probability The ratio of total call setup attempts that result in call setup failure to the total call setup attempts in a population of interest. Applicable to both fixed and mobile calls.

Call signalling ETSI TS 102 024-9 (clause 4.2) ETSI TS 102 024-9 (clause 4.2) delays It involves three different scenarios: call setup, call answer and call release delays. The Call Setup Delay (CSD) is the time between the calling terminal providing sufficient address information to set up the call, and the calling party receiving a confirmation from the called terminal that the called party is being alerted. The Call Answer Signal Delay (CASD) is the time between the called terminal indicating that it is ready to initiate the call and receipt of that indication by the calling terminal. The Call Release Delay (CRD) is the time between the clearing terminal initiating a call cleardown, and its receipt of clearing confirmation by the called terminal. Applicable to both fixed and mobile calls.

Table 1A - IAS QoS Parameters as set out in Annex X of the EECC

14

QoS Parameters Definition Measurement method Annex X

Latency Ref. IETF RFC 2681 Ref. BEREC Net Neutrality Regulatory (delay) Assessment Methodology in the respective The time between the valid version (currently (BoR (22) 72, Sec. 3.2) first bit of a packet of a source entering a network, being received by the destination, which immediately sent a bit back to the source, and then the last bit of the packet arriving at the source across the network (round trip delay).

Delay Ref. IETF RFC 3393 Ref. BEREC Net Neutrality Regulatory variation Assessment Methodology in the respective The difference between (jitter) valid version (currently (BoR (22) 72, Sec. 3.2) the delays of the selected packets.

Packet Loss Ref. ITU-T Y.2617 Ref. BEREC Net Neutrality Regulatory Ratio Assessment Methodology in the respective The total number of valid version (currently (BoR (22) 72, Sec. 3.3) packets failing to deliver through the network divided by the total number of transmitted packets.

15

QoS Parameters Definition Measurement method Annex X

Ref. ITU-T Y. 1540 IP packet loss ratio (IPLR) is the ratio of total lost IP packet outcomes to total transmitted IP packets in a population of interest.

36. Additionally, NRAs in coordination with other competent authorities who choose to specify other parameters, to be measured by the providers, namely for customer services (ICS and IAS), and for end-users with disabilities that are not included in Annex X of the EECC, should take utmost account of the QoS parameters listed in Table 2 below and Section 4, “QoS Parameters relevant for end-users with disabilities” (Table 3). To note, NRAs may require providers to publish additional measurements at national level which are not referred to or listed in Tables 1-3 and therefore do not form part of these Guidelines. However, where guidance is provided for specified QoS parameters listed in Tables 1-3 of these Guidelines, the guidance herein must be taken utmost account of.

Table 2 QoS Parameters not set out in Annex X of the EECC

Additional Definition Measurement method QoS Parameters

(not in Annex X)

Response time ETSI ES 202 057-1 (clause 5.6.1) ETSI EG 202 057-1 (clause for operator 5.6.3) 29 Time elapsed between the end of services dialling to the instant the human

16

Additional Definition Measurement method QoS Parameters

(not in Annex X)

(Customer operator answers the calling user to It is measured by: Care Services provide the service requested. a) mean time to answers; – Help Desk) Applicable to both fixed and mobile b) percentage of calls answered services. within 20 seconds.

Customer ETSI ES 202 057-1 (clause 5.10.1) ETSI ES 202 057-1 (clause complaints 5.10.3) The duration from the instant a resolution time customer complaint is notified to the It is measured by: published point of contact of a service a) the time by which the fastest provider and is not found to be invalid 80% and 95% of complaints to the instant the cause for the have been resolved (expressed complaint has been resolved. in clock hours);

b)the percentage of complaints Applicable to both fixed and mobile resolved any time stated as an services. objective by the service provider.

Number of ETSI ES 202 057-1 (clause 5.9.1) ETSI ES 202 057-1 (clause customer 5.9.4) The number of complaints logged per complaints per customer per data collection period. The number of complaints data collection logged per customer per data period collection period should be provided. Statistics should include all complaints received in the data collection period, regardless of the validity and subject of the complaint.

17

Additional Definition Measurement method QoS Parameters

(not in Annex X)

Data Definitions according to Article 4(1) (d) Guidance to NRAs on IAS speed transmission of the Open Internet Regulation, are measurements is provided in speed (upload included in BoR (22) 81, par. 140-157. section 3.1 of BEREC’s Net and download) Neutrality Regulatory Assessment Methodology doc BoR (22) 72

3 QoS Parameters relevant for end-users with disabilities

3.1 Legal basis

37. According to Article 3(2)(d) of the EECC, NRAs in coordination with other competent authorities should promote the interests of the citizens of the Union inter alia by ensuring a high and common level of protection for end-users through the necessary sector-specific rules and by addressing the needs, such as affordable prices, of specific social groups, in particular end-users with disabilities, elderly endusers and end-users with special social needs, choice and equivalent access for end-users with disabilities.

38. NRAs should note that accessibility requirements for products and services, including accessibly of electronic communication services, are harmonised in the European Accessibility Act (EAA) ,as stated in Article 85(4) of the EECC: “Member States shall ensure, in light of national conditions, that support is provided, as appropriate, to consumers with disabilities, and that other specific measures are taken, where appropriate, with a view to ensuring that related terminal equipment, and specific equipment and specific services that enhance equivalent access, including where necessary total conversation services and relay services, are available and affordable”.

39. The EAA defines persons with disabilities in line with the United Nations Convention on the Rights of Persons with Disabilities, adopted on 13 December 2006 (UN CRPD). For the purpose of the EAA and the EECC, persons with disabilities shall be understood as persons who have long-term physical, mental, intellectual or sensory impairments, which in interaction with various barriers may hinder their full and effective participation in society on an equal basis with others (Article 3(1) EAA). 18

3.2 QoS Parameters and Measurement Methods

40. The EECC and the EAA define specific services targeted to address the needs of persons with disabilities that should be of concern to NRAs when considering QoS parameters in coordination with other competent authorities to be measured by the providers. Relay services refer to services which enable two-way communication between remote end-users of different modes of communication (for example text, sign, speech) by providing conversion between those modes of communication, normally by a human operator.

41. Real time text is defined in Article 3(14) of the EAA and refers to a form of text conversation in point to point situations or in multipoint conferencing where the text being entered is sent in such a way that the communication is perceived by the user as being continuous on a character-by-character basis.

42. According to Article 2(35) of the EECC total conversation service means a multimedia real time conversation service that provides bidirectional symmetric real time transfer of motion video, real time text and voice between users in two or more locations.

43. NRAs should note that other transparency measures concerning equivalent access for persons with disabilities are set out in Article 102(1) , and Article 103(1) of the EECC, in particular, the competent authority in coordination, where relevant, with the national regulatory authority can oblige service providers to publish details of products and services, including any functions, practices, policies and procedures and alterations in the operation of the service, specifically designed for end-users with disabilities, in accordance with European Union law harmonising accessibility requirements for products and services.

44. NRAs in coordination with other competent authorities could accompany the QoS parameters to be measured by the providers listed in Table 1, Table 1A and Table 2 by appropriate QoS parameters concerning equivalent access for persons with disabilities, choosing among those listed in Table 3, bearing in mind that all QoS indicators set out in the Guidelines, in particular, those related to IAS, should address the specific needs of end-users with disabilities.

45. It should be noted that obligations from the EAA are in place since 2022 and the measures will be applicable from 2025 .

46. The standard specifying the quality of service parameters and the relevant measurement methods is TSI EN 301 549 V3.2.1 (2021-03). It should be noted that this standard is currently under revision following the publication of ETSI TR 103 19 708 V1.1.1 (2022-08). In the tables below a reference to the changes proposed by ETSI TR 103 708 V1.1.1 (2022-08) is made.

Table 3 – QoS Parameters relevant for end-users with disabilities

Service QoS Definition Measurement Parameters method

Voice Audio bandwidth ETSI EN 301 549 V3.2.1 (2021- ETSI EN 301 communication for speech 03) (clause 6.1) 549 V3.2.1 (2021-03) Where ICT provides two-way voice (clause C.6.1) communication, in order to provide good audio quality, that ICT shall be able to encode and decode twoway voice communication with a frequency range with an upper limit of at least 7 000 Hz.

The above standard is under ETSI TR 103 708 revision according to the ETSI TR V1.1.1 (2022-08) 103 708 V1.1.1 (2022-08) (Annex (Annex A-A2) A-A2) Draft standard proposes to change the wording to: Where ICT provides conversational real-time voice communication, ICT shall be able to encode and decode conversational real-time voice communication with a frequency range with an upper limit of at least 7 000 Hz.

Real-Time Distinguishable ETSI EN 301 549 V3.2.1 (2021- ETSI EN 301 Text (RTT) display 03) (clause 6.2.2.1) 549 V3.2.1 (2021-03) Where ICT has RTT send and (clause receive capabilities, displayed sent C.6.2.2.1) text shall be visually differentiated from and separated from received

20

Service QoS Definition Measurement Parameters method

text.

The above standard is under ETSI TR 103 708 revision according to the ETSI TR V1.1.1 (2022-08) 103 708 V1.1.1 (2022-08) (clause (clause 13.4.4) 13.4.4) Draft standard proposes to change the wording to: Where ICT has RTT presentation capabilities, displayed received text from different sources and sent text shall, by default, be separated with their sources indicated and differentiated.

Programmatically ETSI EN 301 549 V3.2.1 (2021- ETSI EN 301 determinable 03) (clause 6.2.2.2) 549 V3.2.1 send and receive (2021-03) Where ICT has RTT send and direction (clause receive capabilities, the C.6.2.2.2) send/receive direction of transmitted/received text shall be programmatically determinable, unless the RTT is implemented as closed functionality .

21

Service QoS Definition Measurement Parameters method

The above standard is under ETSI TR 103 708 revision according to the ETSI TR V1.1.1 (2022-08) 103 708 V1.1.1 (2022-08) (clause (clause 13.4.5) 13.4.5) Draft standard proposes to change to: "Where ICT has RTT send and receive capabilities, the origin of text shall be programmatically determinable, unless the RTT is implemented as closed functionality” .

Interoperability ETSI EN 301 549 V3.2.1 (2021- ETSI EN 301 03) (clause 6.2.3) 549 V3.2.1 (2021-03) Where ICT with RTT functionality (clause C.6.2.3) interoperates with other ICT with RTT functionality (as required by clause 6.2.1.1) they shall support the applicable RTT interoperability mechanisms described below: a) ICT interoperating with other ICT directly connected to the Public Switched Telephone Network (PSTN), using Recommendation ITU-T V.18 [i.23] or any of its annexes for text telephony signals at the PSTN interface; b) ICT interoperating with other ICT using VOIP with Session Initiation Protocol (SIP) and using RTT that conforms to IETF RFC 4103 [i.13]. For ICT interoperating with other ICT using the IP Multimedia Sub- System (IMS) to implement VOIP, the set of protocols specified in ETSI TS 126 114

22

Service QoS Definition Measurement Parameters method

[i.10], ETSI TS 122 173 [i.11] and ETSI TS 134 229 [i.12] describe how IETF RFC 4103 [i.13] would apply; c) ICT interoperating with other ICT using technologies other than a or b, above, using a relevant and applicable common specification for RTT exchange that is published and available for the environments in which they will be operating. This common specification shall include a method for indicating loss or corruption of characters; d) ICT interoperating with other ICT using a standard for RTT that has been introduced for use in any of the above environments, and is supported by all of the other active ICT that support voice and RTT in that environment .

23

Service QoS Definition Measurement Parameters method

The above standard is under ETSI TR 103 708 revision according to the ETSI TR V1.1.1 (2022-08) 103 708 V1.1.1 (2022-08) (clause (clause 13.4.9) 13.4.9) Draft standard proposes to change the heading to interoperability and networking and modify the above clause of ETSI EN 301 549 V3.2.1.

RTT Responsive- ETSI EN 301 549 V3.2.1 (2021- ETSI EN 301 ness 03) (clause 6.2.4) 549 V3.2.1 (2021-03) Where ICT utilises RTT input, that (clause C.6.2.4) RTT input shall be transmitted to the ICT network, or platform on which the ICT runs within 500 ms of the time that the smallest reliably composed unit of text entry is available to the ICT for transmission. Delays due to platform or network performance shall not be included in the 500 ms limit.

The above standard is under ETSI TR 103 708 revision according to the ETSI TR V1.1.1 (2022-08) 103 708 V1.1.1 (2022-08) (clause 13.4.10) Draft standard proposed changes are modification of Note 1 and addition of Note 4 .

24

Service QoS Definition Measurement Parameters method

Video Resolution ETSI EN 301 549 V3.2.1 (2021- ETSI EN 301 communication 03) (clause 6.5.2) 549 V3.2.1 (2021-03) Where ICT, that provides two-way (clause C.6.5.2) voice communication, includes real-time video functionality, the ICT: a) shall support at least QVGA resolution; b) should preferably support at least VGA resolution.

Frame Rate ETSI EN 301 549 V3.2.1 (2021- ETSI EN 301 03) (clause 6.5.3) 549 V3.2.1 (2021-03) Where ICT, that provides two-way (clause C.6.5.3) voice communication, includes real-time video functionality, the ICT: a) shall support a frame rate of at least 20 frames per second (FPS); b) should preferably support a frame rate of at least 30 frames per second (FPS) with or without sign language in the video stream.

Synchronisation ETSI EN 301 549 V3.2.1 (2021- ETSI EN 301 between audio 03) (clause 6.5.4) 549 V3.2.1 and video (2021-03) v3.1.1 Where ICT that provides two-way (clause C.6.5.4) voice communication, includes real-time video functionality, the ICT shall ensure a maximum time difference of 100 ms between the speech and video presented to the

25

Service QoS Definition Measurement Parameters method

user.

4 Publication of information

4.1 General guidelines

47. NRAs should note that the publication requirements that the NRAs may set in accordance with Article 104(1) are in addition to the transparency measures provided for in Articles 102 and 103 of the EECC and the transparency obligations set out in Article 4(1) of Regulation (EU) 2015/2120.

48. According to Recital 271 of the EECC, NRAs should be able to require publication of information described in Article 104(1) of the EECC, where it is demonstrated that such information is not effectively available to the public.

49. Detailed guidelines for the transparency measures for ensuring open internet access can also be found in BEREC Guidelines on the Implementation by National Regulators of European Net Neutrality Rules.

50. According to Article 104(1) of the EECC, the information on QoS required by NRAs should be comprehensive, comparable, reliable, user-friendly and up-to-date.

51. The requirement that information is “comprehensive” and “user-friendly” means that it should be representative as well as understood by members of the intended audience. NRAs in coordination with other competent authorities should look to ensure that service providers adhere to the following practices in order to ensure that information is user friendly: • it should promote the use of relevant standards; • it should be presented, preferably, using clear and plain language, in as simple a manner as possible, avoiding complex sentence and language structures. • the information should be concrete and definitive; • it should not be phrased in abstract or ambivalent terms; • it should avoid unduly technical terminology; and • it should not include excessively detailed information.

52. Information is “comparable” if the same relevant information is presented, by different providers or by the same providers for different offers, for comparison in such a way that it can show differences and similarities. Information should be 26 comparable at least between different offers, and between different service providers. The NRAs should be empowered to guide the providers on the consistent measurement methodology (qualitative or quantitative parameters) among the providers to ensure comparability.

53. The “reliable” element means that information should be correct and cannot be misleading for end-users. Information shall comply with standards and measurement methodology indicated by NRAs in coordination with other competent authorities, preferably using certified mechanisms if such mechanisms have been introduced in a given Member State.

54. End-users may like to check information on the QoS of the service, and in that regard, the information should be easy to find, easy to understand, up to date and presented in an accessible way.

55. Where relevant, information in respect to the provision of services through special equipment, for example, information in respect to assistive technologies and the provision of augmentative and alternative communication devices, should be available to end-users with disabilities who require it to access the ICS. Information should be available in respect to types of text relay services available.

56. All published information shall be up-to-date. NRAs in coordination with other competent authorities shall ensure that service providers are obliged to insert the date of the updated publications and reference the period of update. As well as information concerning QoS parameters, service providers can be obliged to publish information showing the most recent update of data at a minimum frequency on an annual basis.

57. Information should be accessible for the broadest possible group of end-users in particular end-users with disabilities, elderly end-users and end-users with special social needs. To achieve that aim NRAs in coordination with other competent authorities could oblige service providers to publish information: • in a machine-readable manner and in an accessible format for end-users with disabilities taking into account general accessibility requirements set in Section III of Annex I of the EAA and European standards aiming to address the needs of persons with disabilities and older persons, dealing with accessibility by applying the Design for All approach –on the provider websites (no more than one click from the /homepage) and via mobile applications that are viewable, operable, understandable and robust and meets harmonised published standards.

58. NRAs in coordination with other competent authorities could oblige service providers (where warranted) to directly publish information via their own communication channels (direct approach), or oblige service providers to publish 27 information through third parties and provide information to NRAs to publish simultaneously on NRAs websites.

59. According to Recital 271 of the EECC, NRAs in coordination with other competent authorities should be empowered to monitor the QoS and to collect systematically information on the QoS offered by providers on the basis of criteria which allow comparability between service providers and between Member States. To achieve these objectives NRAs in coordination with other competent authorities could require service providers in accordance to Article 104(1) of the EECC to publish information having regard to different levels of aggregation (regional, national) or different groups of end-users (business clients, consumers), depending on the level of availability of information to the public, QoS parameter or service.

60. To that end, and to enhance overall publication, NRAs shall consider the inclusion of QoE (quality of experience) indicators if appropriate.

5 Quality Certification mechanisms

5.1 General guidelines

61. Where NRAs require publication as provided for in 104(1), the following applies to quality certification mechanisms (Article 104(2) EECC), “NRAs in coordination with other competent authorities shall specify the quality of service parameters to be measured, the applicable measurement methods, and the content, form and manner of the information to be published, including possible quality certification mechanisms”. Moreover, Article 4(4) of the Regulation (EU) 2015/2120 refers to the quality monitoring mechanism certified by an NRA.

62. The EECC does not require Member States or an NRA to establish or certify a monitoring mechanism. Regarding IAS, Article 104(2) of the EECC does not have any impact on monitoring mechanisms in relation to Article 4(4) of Regulation (EU) 2015/2120, which references a monitoring mechanism certified by the NRA. Indeed, the BEREC Guidelines on the Implementation of Open Internet Regulation stipulate that if the NRA provides a monitoring mechanism implemented for this purpose, it should be considered as a certified monitoring mechanism in relation to Article 4(4) of Regulation (EU) 2015/2120.

63. Plural “quality certification mechanisms” used in Article 104 of the EECC anticipates the possibility of functioning of more than one certification mechanism, e.g., for IAS and publicly available ICS.

64. EECC provisions do not prescribe who may be a provider of a quality certification mechanism.

65. NRAs in coordination with other competent authorities must take into account the requirement of independence of the provider of the quality certification mechanism from IAS and publicly available ICS providers. In this context, the NRA may take into account not only circumstances pointing to capital or personal links with 28 telecommunications service providers operating in the market, but also the business model of the quality certification mechanism provider.

66. The approach taken by the NRA in coordination with other competent authorities to specify the quality certification mechanism may take many various forms. Provisions of the EECC do not impose requirements on the certification procedure. The level of formalization of the procedure as well as additional requirements, such as the requirement for a specific form of the certification act (e.g., an administrative decision, ordinance) may be determined in national law.

67. The EECC does not set out requirements about the certification period, the conditions for the certification withdrawal, or extending the certification.

68. The NRA in coordination with other competent authorities should determine what factors are to be taken into account when choosing a quality certification mechanism. The certification should ensure that the quality monitoring fulfils requirements, such as: 1. Accuracy - The results of measurements should be accurate as far as it is possible in accordance with the state-of-the-art knowledge and with the reservation that the end-user or consumer should not be loaded with disproportionate obligations associated with performance of measurements, in particular, if these requirements do not have a significant impact on the result. Achieving this objective cannot limit the availability of the mechanism for quality monitoring for all end-users. When assessing the factors that can affect the accuracy and reliability of measuring the quality of the IAS and other publicly available ICS, it may be appropriate for NRAs to specify the requirements that should be met by the end-user environment . 2. Enables comparison of measurements - The quality monitoring mechanism should make it possible to compare the results of the QoS measurements carried out with those QoS parameters which are included in the contract as well as enabling a comparison between different service providers. 3. Openness - The measurement methodology and implementation should be publicly available, and the NRA or other competent entity should consider publishing information on factors which can affect the reliability of results, if such factors have been identified. The publication of its source code contributes to the openness of the quality monitoring mechanism; however, a provider of a quality monitoring mechanism cannot be obliged to publish the source code. 4. Safety – The quality monitoring mechanism should be adequately safeguarded against attacks, and its integrity and the confidentiality of processed personal data against unauthorized access should be guaranteed. 29

5. Future-proofness - Quality monitoring mechanisms should be based on the current state of technical knowledge, and its design, taking into account the development and evolution of the telecommunications market. 6. Accessibility - The use of the quality monitoring mechanisms should be accessible to people with disabilities.

6 Review clause

69. To ensure consistency and efficiency in the application of the Guidelines, considering past practice and stakeholders' expectations regarding possible changes and potential new QoS parameters to consider in the future, the process of undertaking a review of the Guidelines will commence 2 years from the adaption and publication of these Guidelines in 2024. 30

Annex 1 EECC Article 104 and Annex X

L 321/178 EN Official Journal of the European Union 17.12.2018

Article 104 - QoS related to IAS and publicly available interpersonal communications services 1. National regulatory authorities in coordination with other competent authorities may require providers of IAS and of publicly available interpersonal communications services to publish comprehensive, comparable, reliable, user-friendly and up-to-date information for end-users on the quality of their services, to the extent that they control at least some elements of the network either directly or by virtue of a service level agreement to that effect, and on measures taken to ensure equivalence in access for end-users with disabilities. National regulatory authorities in coordination with other competent authorities may also require providers of publicly available interpersonal communication services to inform consumers if the quality of the services they provide depends on any external factors, such as control of signal transmission or network connectivity. That information shall, on request, be supplied to the national regulatory and, where relevant, to other competent authorities before its publication. The measures to ensure QoS shall comply with Regulation (EU) 2015/2120. 2. National regulatory authorities in coordination with other competent authorities shall specify, taking utmost account of BEREC guidelines, the QoS parameters to be measured, the applicable measurement methods, and the content, form and manner of the information to be published, including possible quality certification mechanisms. Where appropriate, the parameters, definitions and measurement methods set out in Annex X shall be used. By 21 June 2020, in order to contribute to a consistent application of this paragraph and of Annex X, BEREC shall, after consulting stakeholders and in close cooperation with the Commission, adopt guidelines detailing the relevant QoS parameters, including parameters relevant for end-users with disabilities, the applicable measurement methods, the content and format of publication of the information, and quality certification mechanisms.

ANNEX X- QUALITY OF SERVICE PARAMETERS

Quality-of-Service Parameters, Definitions and Measurement Methods referred to in Article 104 For providers of access to a public electronic communications network:

PARAMETER DEFINITION MEASUREMENT METHOD (Note 1)

Supply time for initial ETSI EG 202 057 ETSI EG 202 057 connection Fault rate per access line ETSI EG 202 057 ETSI EG 202 057 Fault repair time ETSI EG 202 057 ETSI EG 202 057 For providers of interpersonal communications services who exert control over at least some elements of the network or have a service level agreement to that effect with undertakings providing access to the network:

PARAMETER DEFINITION MEASUREMENT METHOD

31

(Note 2)

Call set up time ETSI EG 202 057 ETSI EG 202 057 Bill correctness complaints ETSI EG 202 057 ETSI EG 202 057 Voice connection quality ETSI EG 202 057 ETSI EG 202 057 Dropped call ratio ETSI EG 202 057 ETSI EG 202 057 Unsuccessful call ratio ETSI EG 202 057 ETSI EG 202 057 (Note 2) Failure probability Call signalling delays

PARAMETER DEFINITION MEASUREMENT METHOD

Latency (delay) ITU-T Y.2617 ITU-T Y.2617 Jitter ITU-T Y.2617 ITU-T Y.2617 Packet loss ITU-T Y.2617 ITU-T Y.2617

Note 1 Parameters shall allow for performance to be analysed at a regional level (namely, no less than level 2 in the Nomenclature of Territorial Units for Statistics (NUTS) established by Eurostat). Note 2 Member States may decide not to require up-to-date information concerning the performance for those two parameters to be kept if evidence is available to show that performance in those two areas is satisfactory.

32

Recitals:

(271) National regulatory authorities in coordination with other competent authorities, or where relevant, other competent authorities in co-ordination with national regulatory authorities should be empowered to monitor the quality of services and to collect systematically information on the quality of services offered by providers of internet access services and of publicly available interpersonal communications services, to the extent that the latter are able to offer minimum levels of service quality either through control of at least some elements of the network or by virtue of a service level agreement to that end, including the quality related to the provision of services to end-users with disabilities. That information should be collected on the basis of criteria which allow comparability between service providers and between Member States. Providers of such electronic communications services, operating in a competitive environment, are likely to make adequate and up-to-date information on their services publicly available for reasons of commercial advantage. National regulatory authorities in coordination with other competent authorities, or where relevant, other competent authorities in co-ordination with national regulatory authorities should nonetheless be able to require publication of such information where it is demonstrated that such information is not effectively available to the public. Where the quality of services of publicly available interpersonal communication services depends on any external factors, such as control of signal transmission or network connectivity, national regulatory authorities in coordination with other competent authorities should be able to require providers of such services to inform their consumers accordingly. (272) National regulatory authorities in coordination with other competent authorities should also set out the measurement methods to be applied by the service providers in order to improve the comparability of the data provided. In order to facilitate comparability across the Union and to reduce compliance cost, BEREC should adopt guidelines on relevant quality of service parameters which national regulatory authorities in coordination with other competent authorities should take into utmost account.

33

Annex 2 Definitions

This Annex contains a selection of terms and definitions used in the Guidelines to support the consistent and harmonised application of the provisions of Article 104 of the EECC. Internet access services: a publicly available electronic communications service that provides access to the internet, and thereby connectivity to virtually all end points of the internet, irrespective of the network technology and terminal equipment used (Regulation (EU) 2015/2120). Interpersonal communications services: a service normally provided for remuneration that enables direct interpersonal and interactive exchange of information via electronic communications networks between a finite number of persons, whereby the persons initiating or participating in the communication determine its recipient(s) and does not include services which enable interpersonal and interactive communication merely as a minor ancillary feature that is intrinsically linked to another service (Directive (EU) 2018/1972). Number-based interpersonal communications service: an interpersonal communications service which connects with publicly assigned numbering resources, namely, a number or numbers in national or international numbering plans, or which enables communication with a number or numbers in national or international numbering plans (Directive (EU) 2018/1972). Number-independent interpersonal communications service: an interpersonal communications service which does not connect with publicly assigned numbering resources, namely, a number or numbers in national or international numbering plans, or which does not enable communication with a number or numbers in national or international numbering plans (Directive (EU) 2018/1972).

34

Annex 3 Other Benchmarking

2016 Study prepared for the EC – Fixed and Mobile Convergence in Europe – Quality Measurements for 5G and Network Densification Source EC Fixed and Mobile Convergence in Europe Quality Measurements for 5G and Network Densification Table 2.2 Table 2.2 Most widely mandated existing QoS indicators across the MS

Category Indicators

Internet Data transfer speed (maximum, minimum, typical); Web page loading time; Latency; Jitter; Packet loss rate Voice Call set-up time; Unsuccessful call rate; Speech transmission quality; Response time for calls to the operator, customer service and directory assistance Mobile Network availability; Probability of successful connection in an area covered by the network; Dropped call ratio Customer service Time between request for service and start of service; Fault frequency; Time to troubleshoot & eliminate faults; Frequency of complaints about billing Emergency calls Total number of 112 calls per year; 112 calls as a percentage of total emergency calls; Percentage of false calls; Average time to answer; Percentage of calls answered within 10 seconds; Call abandon rate; Average time needed for operator to receive the caller’s location

Source: Regulations published by NRAs.

35