BEREC Guidelines detailing Quality of Service Parameters

BEREC BoR (19) 189 Final V 1.0

BEREC Guidelines detailing Quality of Service Parameters

03 OCTOBER 2019

BEREC BoR (19) 189

1. Introduction

1. According to Article 4 of the BEREC Regulation , BEREC shall issue guidelines on the implementation of the European Union regulatory framework for electronic communications, as referred to in Directive (EU) 2018/1972 (hereinafter referred to as “the EECC”), on, among other things, relevant quality of service (‘QoS’) parameters which National Regulatory Authorities (NRAs), in coordination with other competent authorities, should take utmost account of. Annex 1 to the Guidelines sets out the wording of Article 104 and Annex X of the EECC as well as the related EECC recitals. For the avoidance of doubt, definitions as set out in Article 2 EECC shall be used in this document unless otherwise stated (see Annex 2 to the Guidelines). 2. In accordance with Article 104(2) of the EECC, the reason for issuing the present guidelines (hereinafter also referred to as “the Guidelines”) is to provide guidance to NRAs in respect to Article 104 of the EECC and to contribute to the consistent application of Article 104(2) and Annex X, with the aim of defining: a) the relevant QoS parameters, including the parameters relevant for end-users with disabilities; b) the applicable measurement methods for these QoS parameters, including, where appropriate, the ETSI and ITU standards set out in Annex X of the EECC in relation to interpersonal communications services (“ICS”) and Internet access services (“IAS”), respectively; c) the content and format of publication of the QoS information, and d) the quality certification mechanisms.

3. Annex 3 to the Guidelines sets out the input received from NRAs in respect to any specified QoS parameters, measurement methods, and the content, form and manner of the information published, under the relevant provisions of Article 22 of the Universal Service Directive (2002/22/EC). Annex 4 to the Guidelines sets out other benchmarking undertaken in respect to QoS indicators across member states.

4. Article 104 of the EECC contains a specific reference to Regulation (EU) 2015/2120: “the measures to ensure quality of service shall comply with Regulation (EU) 2015/2120” . As a consequence, a close interdependency between the Open Internet and End-Users Working Groups has been acknowledged and, in order to ensure the consistency of BEREC’s documents, the Guidelines do not focus on defining IAS QoS parameters related to the network performance and measurement methods that are analysed within the Open Internet Working Group (OI WG) – and in that context make an explicit reference to several relevant BEREC reports 2

BEREC BoR (19) 189

5. In addition there are other QoS related and relevant ongoing BEREC work streams which are currently being developed by the following BEREC WGs – Statistics and Indicators WG, OI WG and Roaming WG. The output from these BEREC work streams should be considered and taken utmost account of by NRAs and by other competent authorities when defining the IAS QoS parameters and their measurement methods.

6. In accordance with Article 104 of the EECC, the Guidelines shall be issued by 21 June 2020, after consulting stakeholders and in close cooperation with the Commission.

2. Policy principle, legal basis and scope of the BEREC Guidelines

2.1 Policy principle

7. In the ever-connected, globalized, digital environment that is developing faster and faster electronic communication services play a key role in citizens’ everyday activities. As set out in the QoS regulation manual ITU 2017 pgs. 5 & 110 – ‘The profusion of ever-evolving technologies, networks, services and devices with different QoS capabilities further adds to the complexity of regulation in this area. Quality can be impacted by many factors at the network level and along the value chain. In this regard, a common approach to regulating QoS can enable greater quality prospects irrespective of the locations of the consumer and service provider’. Indeed, the correlation between quality and pricing of services suggests: ‘If the demand for services that require high QoS is very low (compared to demand for services not requiring QoS), then the willingness to pay for high QoS will be also very low. In such a case, telecommunication operators (which are in fact the ISPs nowadays) will have lower interest in QoS. When the demand for services that require high QoS is comparable with demand for services not requiring QoS, then the willingness to pay for QoS is higher’.

8. The QoS, as perceived by the end-user, is a crucial factor for both customers and service providers and, with the profusion of ever evolving technologies, networks and services with different levels of QoS, it is becoming increasingly more complex to manage, measure and regulate QoS. Indeed, quality can be impacted by many 3

BEREC BoR (19) 189

factors at the network level and along the value chain, including the device, hardware, infrastructure, service and applications.

9. Regulatory development in the European electronic communications sector is intended to help improve the end-user experience, to lead to greater competition and investment, and to benefit all the different players in the digital ecosystem. This development has resulted in the EECC and BEREC has undertaken a complex work aimed, inter alia, at achieving one of the very clear objectives of the EECC, i.e., empowering and protecting end-users.

10. The European harmonisation of QoS parameters and data collection and publication practices would result in substantive benefits, such as enabling comparability among Member States and providing better information on the European electronic communications market, while at the same time promoting the consistent application of regulatory obligations and improving transparency for endusers and public authorities in relation to quality of service.

2.2 Legal basis

11. The rationale for issuing Guidelines detailing QoS parameters is, therefore, to contribute to a consistent and harmonised application of the provisions of the EECC, in particular with respect to Article 104 that is broadly a continuation of Article 22 of the Universal Service Directive (2002/22/EC), on the publication of information for end-users on the QoS.

12. More specifically, Article 104(1) of the EECC provides that NRAs in coordination with other competent authorities may require providers of IAS and of publicly available ICS to publish comprehensive, comparable, reliable, user-friendly and upto-date information for end-users on the quality of their services and on measures taken to ensure equivalence in access for end-users with disabilities.

13. According to Recital 260 of the EECC, end-users should be informed, inter alia, of the different levels of the QoS, conditions for promotions and termination of contracts, applicable tariff plans and tariffs for services subject to particular pricing conditions.

14. At the same time, Recital 271 of the EECC provides that NRAs – in coordination with other competent authorities – should be empowered to monitor the QoS and to systematically collect information on the QoS offered by providers of IAS and of publicly available ICS, to the extent that the latter are able to offer minimum levels of service quality either through control of at least some elements of the network or by virtue of a service level agreement (SLA) to that end, including the quality related to the provision of services to end-users with disabilities. That information should be collected on the basis of criteria which allows comparability between service providers and between Member States. Providers of such electronic communications services, operating in a competitive environment, are likely to make adequate and up-to-date information on their services publicly available for reasons of commercial advantage. NRAs in coordination with other competent authorities, or where relevant, other competent authorities in co-ordination with national regulatory authorities should nonetheless be able to require publication of 4

BEREC BoR (19) 189

such information where it is demonstrated that such information is not effectively available to the public.

15. Moreover, according to Article 104(1) of the EECC, NRAs in coordination with other competent authorities may require providers of publicly available ICS to inform consumers if the quality of the services they provide depends on any external factors, such as control of signal transmission or network connectivity.

16. It follows from Article 104(1) of the EECC that the information obligations which an NRA may require from a given provider depend on two criteria: • firstly, the relevant service (IAS and/or publicly available ICS); • secondly, whether the provider controls at least some elements of the network either directly or by virtue of an SLA to that effect.

17. In order to facilitate comparability across the European Union and to reduce compliance cost, according to Recital 272, BEREC should adopt guidelines on relevant QoS parameters which NRAs in coordination with other competent authorities should take utmost account of.

18. Moreover, Article 104(1), provides that “The measures to ensure quality of service shall comply with Regulation (EU) 2015/2120.” Article 4(1) of Regulation (EU) 2015/2120 contains specific transparency obligations for providers of IAS. In the BEREC Guidelines on Net Neutrality adopted in 2016, BEREC has further explained the transparency obligations contained in Article 4(1), first sentence, (a)- (e).

19. In accordance with Article 104(2) of the EECC, NRAs in coordination with other competent authorities shall specify, taking utmost account of the Guidelines, the QoS parameters to be measured, the applicable measurement methods, and the content, form and manner of the information to be published, including possible quality certification mechanisms, using where appropriate, the parameters, definitions and measurement methods set out in Annex X of the EECC.

2.3 Scope of the BEREC Guidelines

20. In light of the above, the Guidelines, in pursuing the goal of providing transparency to consumers on the QoS, provide assistance to NRAs on the QoS parameters that NRAs could decide to be measured and the applicable methods, as well as on the information to be published and the possible quality certification mechanisms.

21. Furthermore, QoS can be distinguished from Quality of Experience (QoE) as QoS concerns the network and terminal equipment up to the user interface while QoE focuses on the entire service experience and includes the whole path from user to user including the end-user expectation, perception and context of use. For more details on QoE see ITU-T Rec G.1011 . Network performance (NP) is more limited in scope because it excludes terminal performance. Figure 1 shows the relationship between these terms. For the purpose of the Guidelines only QoS is taken into consideration. 5

BEREC BoR (19) 189

Figure 1: QoS, QoE, NP (source BEREC, 2011)

CPE: Customer Premise Equipment, UNI: User-to-Network Interface

22. Moreover, the Guidelines focus solely on QoS parameters related to ICS as well as the corresponding measurement methods and certification mechanisms. QoS of IAS is dealt within the BEREC OI WG publications.

23. ICS can be provided by different technological means. Firstly, in the event where the provider has control over network elements (e.g. because he owns the network) or has an SLA with a network operator: it is possible for the provider of the ICS to give information on QoS parameters. For example, in the case of telephony services which are provided as “traditional” (i.e. non-Over The Top – OTT) telephony services or as specialised services (i.e. managed services), it is possible for the provider to indicate the corresponding QoS parameters in the case where voice connections are originated and terminated within the providers network while endusers use specific terminal equipment. In this scenario, a provider is obliged to fulfil the information requirements set out in Article 104(1) of the EECC, if an NRA requires such.

24. Secondly, in the event where the provider neither has control over network elements, nor has an SLA to that effect: this may arise if the interpersonal communication services are provided over the internet, i.e. number-independent ICS (NIICS). In this event, the quality of the ICS depends on the quality of the IAS and terminal equipment used. For example, a provider of a messaging service which also has a voice service functionality cannot indicate the QoS of the voice call because the quality of the voice call is influenced by the underlying IAS and terminal equipment used. According to Article 104(1) of the EECC, an NRA may require the provider of the NIICS to inform consumers if the quality of the services they provide depends on any external factors, such as control of signal transmission, network connectivity and terminal equipment. If the NRA requires so, a NIICS provider is obliged to inform consumers that the voice quality depends e.g. on the quality of the 6

BEREC BoR (19) 189

underlying IAS and terminal equipment. However, the NIICS provider cannot himself make a statement on the QoS as this is outside the area of his influence.

25. Different standards have been defined to detail methodologies to measure QoS of ICS and IAS. The measurement methods specified by NRAs should be based, where appropriate, on standards as set out in Tables 1, 2 and 3 of the Guidelines. NRAs should take account of and consider guidance on IAS QoS indicators and related definitions, methodologies developed by BEREC OI WG .

26. In addition there are other QoS related and relevant ongoing BEREC work streams which are currently being developed by the following BEREC WGs – Statistics and Indicators WG, OI WG and Roaming WG. The output from these BEREC work streams could be considered and taken utmost account of by NRAs and by other competent authorities when defining the IAS QoS parameters and their measurement methods.

27. Several techniques can be used to measure different QoS parameters: measurements based on the actual occurrences , self-certification, survey, drive tests, probes on selected locations, theoretical values, crowdsourcing, etc. Techniques recommended by the standards considered hereafter should be followed when relevant.

28. Any techniques used for conducting measurements should be made transparent and available for third-party verification and, if feasible, to end-users at no additional cost, including end-users with disabilities. Recommended approaches within these standards to guarantee accuracy of measurement should also be taken into consideration.

29. IAS and ICS providers should assess all factors that may impact the QoS levels available to end-users, for example, user environment or the bias brought by the location of test servers or interconnection issues, etc. Whenever possible, providers should take into consideration those factors during the measurement process.

3. Internet access service (network performance QoS parameters)

30. Article 104 contains a specific reference to Regulation (EU) 2015/2120: “the measures to ensure quality of service shall comply with Regulation (EU) 2015/2120” . As a consequence, a close interdependency between the Open Internet and End-Users Working Groups has been acknowledged and, in order to ensure the consistency of BEREC’s documents, the Guidelines do not focus on defining IAS QoS parameters related to the network performance and measurement methods that are analysed within the OI BEREC WG and, in that context, make an explicit reference to several relevant BEREC reports. Therefore, this section only 7

BEREC BoR (19) 189

summarises some aspects relative to those QoS parameters. NRAs who require Providers to publish QoS indicators related to the network performance and measurements for IAS should take utmost account of the provisions developed by BEREC OI WG together with the indicators set out in Article 104 EECC Annex X, i.e. latency, jitter and packet loss as defined with measurement methods in ITU-T Y.2617 where applicable.

31. Annex X of the EECC contains some basic QoS parameters and measurement methods for IAS. Concerning network performance, besides speed, the most important parameters, which influence QoS of IAS, are delay, delay variation (jitter) and packet loss, see BEREC Guidelines Net Neutrality 2016 , paragraph 137.

32. BEREC OI WG has also defined network performance indicators to assess the quality of IAS. The experience of this group resulted in various documents. A guidance on how to measure these parameters is given in the BEREC Net Neutrality Regulatory Assessment Methodology BoR (17) 178. These include QoS parameter of IAS, especially: • IAS speed in both downlink and uplink direction; • Round-trip delay; • Delay variation and • Packet loss.

33. A practical implementation of this methodology is currently being developed by the BEREC NN tool, which is based on the BEREC Net Neutrality Tool Specification BoR (17) 179 and the Annex 1 of BoR (18) 32.

34. In cases where NRAs establish monitoring mechanisms according to Article 4 (4) of Regulation (EU) 2120/2015 these mechanisms shall also be deemed acceptable for monitoring insofar as they provide measurements of the parameters defined by the BEREC OI WG.

4. Interpersonal Communication Services and non-network performance QoS parameters for IAS services

35. According to Figure 1 as set out in para 20 of this document and referenced by the EC in its Final Report “Fixed and Mobile Convergence in Europe” QoS refers to the effectiveness of performance of a system in support of end-user needs or that contributes positively to another system’s performance.

36. In addition, as set out in Annex 4 to the Guidelines, an extract of the most widely mandated existing QoS indicators across Member States include the following: • Voice - Call set-up time; Unsuccessful call rate; Speech transmission quality; Response time for calls to the operator, customer service and directory assistance • Mobile - Network availability; Probability of successful connection in an area covered by the network; Dropped call ratio 8

BEREC BoR (19) 189

• Customer service - Time between request for service and start of service; Fault frequency; Time to troubleshoot & eliminate faults; Frequency of complaints about billing

4.1 QoS Parameters and Measurement Methods

37. Depending on the nature of the content to be exchanged (e.g. audio, video, text, data), different quality parameters need to be specified. ETSI EG 202 057 multi-part deliverable standards (see Annex X of the EECC) provides guidance on the basic approach to be applied in order to assess the different aspects of quality.

38. Providers of NI ICS and NB ICS cannot know and influence the technical characteristics of interconnected networks and terminal equipment used at the endpoints of the communication. Thus, providers can only specify estimates of the resulting communication quality of actual end-to-end communications. However, such providers are only subject to Article 104 of the EECC in so far as they control parts of the network or have an SLA with a network operator.

39. Typically, NI ICS are designed to compensate for the varying transport quality of packet switched networks and the best effort packet forwarding principle. They do not have stringent requirements for network quality and just require an adequate overall performance level (e.g. a maximum delay value not to be exceeded for realtime communication).

40. Table 1 below lists QoS parameters, definitions and measurement methods from Annex X of the EECC which ‘shall’ be used, where appropriate.

41. For completeness BEREC are proposing ETSI definitions and measurement methods for two QoS parameters set out in Annex X of the EECC (the failure probability parameter and the call signalling delays parameter) which currently do not have definitions and measurements methods provided, and which shall be used by NRAs, where appropriate, see Table 1 below.

42. It is important to note here that there is a degree of flexibility allowed when deciding which QoS parameters should be specified by NRAs. To this end NRAs are free to choose among the QoS parameters listed in Table 1, those that are appropriate, taking into account national circumstances and other factors, such as underlying costs, time needed to implement the measurement and possible monitoring systems, changes required to adapt and modify current methodologies and providing for the possibility of comparing new results with previous records. NRAs are therefore not obliged to specify the full list of parameters contained in Table 1, but can choose the ones that are particularly relevant for the needs of their country. Where NRAs choose to impose relevant and appropriate QoS parameters from Table 1, they shall take utmost account of the Guidelines and of the definitions and the measurement methods listed in Table 1. 9

BEREC BoR (19) 189

Table 1 QoS Parameters as set out in Annex X of the EECC

QoS Definition Measurement method Parameters Annex X Supply time ETSI EG 202 057-1 (clause 5.1) ETSI EG 202 057-1 (clause 5.1.3) for initial The duration from the instant of It is measured by: connection a valid service order being a) the times by which the fastest received by a direct service 50%, 95% and 99% of orders are provider to the instant a working completed; service is made available for b) the percentage of orders use. This should exclude completed by the date agreed with cancelled orders. the customer and, where the Applicable to both fixed and percentage of orders completed by mobile services. the date agreed with the customer is below 80%, the average number of days, for the late orders, by which the agreed date is exceeded. Statistics for both fixed and mobile access networks. Fault rate ETSI EG 202 057-1 (clause 5.4) ETSI EG 202 057-1 (clause 5.4.3) per access The number of reported faults Statistics for all fixed access lines. line per fixed access line per year. Fault repair ETSI EG 202 057-1 (clause 5.5) ETSI EG 202 057-1 (clause 5.5.3) time The duration from the instant a It is measured by: fault report has been made to a) the time by which the fastest the instant when the service 80% and 95% of valid faults on element or service has been access lines are repaired restored to normal working (expressed in clock hours); order. b) the percentage of faults cleared any time stated as an objective by the service provider; c) the provision of information on the hours during which faults may be reported. Statistics for all access fixed networks. Call setup ETSI EG 202 057-2 (clause 5.2) ETSI EG 202 057-1 (clause 5.2.3) time The call set up time is the It is measured by:

10

BEREC BoR (19) 189 QoS Definition Measurement method Parameters Annex X period starting when the a) the mean value in seconds for address information required national calls; for setting up a call is received b) the time in seconds within which by the network and finishing the fastest 95% of national calls when the called party busy tone are set-up; or ringing tone or answer signal c) the mean value in seconds for is received by the calling party. international calls; Where overlap signalling is d) the time in seconds within which used the measurement starts the fastest 95% of international when sufficient address calls are set-up; information has been received e) the number of observations to all the network to begin performed for national and routeing the call. international calls. Applicable to both fixed and Statistics for both fixed and mobile mobile calls. voice services. 3GPP TS 32.454 clause 5.1.2 It is measured by the mean value 3GPP TS 32.454 clause 5.1.2 Session setup time Applicable for IMS (VoLTE KPI) Bill ETSI EG 202 057-1 (clause ETSI EG 202 057-1 (clause correctness 5.11) 5.11.3) complaints The proportion of bills resulting It is measured by a percentage. in a customer complaint about the correctness of a given bill per service. Applicable to both fixed and mobile services. Voice ETSI EG 202 057-2 (clause 5.3) ETSI EG 202 057-2 (clause 5.3.2) connection ETSI TR 102 506 Statistics for: quality Evaluation of speech quality - Fixed to fixed calls per call. - Fixed to mobile calls The end-user perceived voice - Mobile to fixed calls quality. - Mobile to mobile calls Applicable to fixed and mobile ITU-T G.1020: Performance voice services. parameter definitions for quality of speech and other voice band applications utilizing IP networks; ITU-T G.1028: End-to-end quality of service for voice over 4G mobile networks;

11

BEREC BoR (19) 189 QoS Definition Measurement method Parameters Annex X ITU-T P.863: Perceptual objective listening quality prediction. Dropped call ETSI EG 202 057-3 (clause ETSI EG 202 057-3 (clause ratio 6.4.2) 6.4.2.2) The proportion of incoming and When using the measurements outgoing calls which, once they based on network element have been correctly established counters, the following statistics and therefore have an assigned should be provided: the traffic channel, are dropped or percentage of dropped calls, interrupted prior to their normal calculated from all the calls in the completion by the user, the period. cause of the early termination When using test calls, the following being within the operator's statistics should be provided: the network. percentage of dropped calls, Applicable to mobile networks. together with the number of observations used and the absolute accuracy limits for 95% 3GPP TS 32.454 clause 5.2.1 confidence calculated from this Call drop for IMS session number. Applicable for IMS (VoLTE KPI) 3GPP TS 32.454 clause 5.2.1 It is measured by a percentage. Unsuccessfu ETSI EG 202 057-2 (clause 5.1) ETSI EG 202 057-2 (clause 5.1.3) l call ratio ETSI EG 201 769-1 (clause 5.4) ETSI EG 201 769-1 (clause 5.4.2) Unsuccessful call ratio is It is measured by: defined as the ratio of a) the percentage of unsuccessful unsuccessful calls to the total calls for national calls; number of call attempts in a b) the percentage of unsuccessful specified time period. calls for international calls; Applicable for both fixed and c) the number of observations mobile networks. used for national and international calls together with absolute accuracy. Call set up ETSI TS 102 024-9 (clause ETSI TS 102 024-9 (clause 4.1.1) failure 4.1.1) probability The ratio of total call setup attempts that result in call setup failure to the total call setup attempts in a population of interest.

12

BEREC BoR (19) 189 QoS Definition Measurement method Parameters Annex X Applicable to both fixed and mobile calls. Call ETSI TS 102 024-9 (clause 4.2) ETSI TS 102 024-9 (clause 4.1.1) signalling It involves three different delays scenarios: call setup, call answer and call release. The call set up signalling delay is the time between the calling terminal providing sufficient address information to set up the call, and the calling party receiving a confirmation from the called terminal that the called party is being alerted. Applicable to both fixed and mobile calls.

43. Additionally, NRAs who choose to specify other parameters, namely for customer services (ICS and IAS), mobile SMS and end-users with disabilities that are not included in Annex X of the EECC, shall take utmost account of the QoS parameters listed in Table 2 below and Section 5, “QoS relevant for end-users with disabilities” (Table 3).

Table 2 QoS Parameters not set out in Annex X of the EECC

Additional Definition Measurement method QoS Parameters (not in Annex X) Response time ETSI EG 202 057-1 (clause ETSI EG 202 057-1 (clause 5.6.3) for operator 5.6.1) It is measured by: services Time elapsed between the a) mean time to answers; (Customer end of dialling to the instant b) percentage of calls answered Care Services the human operator within 20 seconds. – Help Desk) answers the calling user to provide the service requested. Applicable to both fixed and mobile services.

13

BEREC BoR (19) 189 Additional Definition Measurement method QoS Parameters (not in Annex X) Frequency of ETSI EG 202 057-1 (clause ETSI EG 202 057-1 (clause 5.9.3) customer 5.9.1) ETSI EG 202 843 complaints ETSI EG 202 843 Statistics: The number of complaints Number of customer requests to logged per customer per - technical support data collection period. - commercial support Number of customer complaints Applicable to both fixed and related to mobile services. - repair services - network/service management by the customer - cessation Number of customer complaints of any kind Customer ETSI EG 202 057-1 (clause ETSI EG 202 057-1 (clause 5.10.3) complaints 5.10.1) It is measured by: resolution time The duration from the a) the time by which the fastest 80% instant a customer and 95% of complaints have been complaint is notified to the resolved (expressed in clock hours); published point of contact b) the percentage of complaints of a service provider and is resolved any time stated as an not found to be invalid to the objective by the service provider. instant the cause for the complaint has been resolved. Applicable to both fixed and mobile services. Successful ETSI EG 202 057-2 (clause ETSI EG 202 057-2 (clause 5.6.1.3) SMS Ratio 5.6.1) The percentage of successfully sent Probability that a user can short messages, together with the send a Short Message number of observations used and successfully from a terminal the absolute accuracy limits for 95% equipment to a Short confidence calculated from this Message Center. number. Applicable to mobile networks.

14

BEREC BoR (19) 189 Additional Definition Measurement method QoS Parameters (not in Annex X) SMS delivery ETSI EG 202 057-2 (clause ETSI EG 202 057-2 (clause 5.6.3.3) time 5.6.3) It is measured by: The end-to-end delivery a) the mean value in seconds for time for SMS is the period sending and receiving short starting when sending a messages; SMS from a terminal b) the time in seconds within which equipment to a Short the fastest 95 % of short messages Message Center and are sent and received; finishing when receiving the c) the number of observations very same SMS on another performed. terminal equipment. ETSI EG 102 250-2 (clause 7.4.5) ETSI TR 102 529 Applicable to mobile networks. Consultation Question 1

1. According to Article 104 of the EECC information required from providers on the quality of their services should be comparable, reliable, user-friendly and up-to- date. Do you believe the parameters and measurement methods in Table 2 are suitable for this purpose? If not, please explain why and the possible changes that could be made to improve the information.

5. QoS relevant for end-users with disabilities

5.1 Legal basis

44. According to Article 3(2)(e) of EECC NRAs should promote the interests of the citizens of the Union inter alia by ensuring a high and common level of protection for end-users through the necessary sector-specific rules and by addressing the needs, such as affordable prices, of specific social groups, in particular end-users with disabilities, elderly end-users and end-users with special social needs, and choice and equivalent access for end-users with disabilities.

45. NRAs should note that accessibility requirements for products and services including accessibly of electronic communication services are harmonised in the 15

BEREC BoR (19) 189

European Accessibility Act (EAA)23 and are also stated in Article 85 (4) of the EAA: “Member States shall ensure, in light of national conditions, that support is provided, as appropriate, to consumers with disabilities, and that other specific measures are taken, where appropriate, with a view to ensuring that related terminal equipment, and specific equipment and specific services that enhance equivalent access, including where necessary total conversation services and relay services, are available and affordable.”

46. The EAA defines persons with disabilities in line with the United Nations Convention on the Rights of Persons with Disabilities, adopted on 13 December 2006 (UN CRPD) and for the purpose of the EAA and the EECC means persons who have long-term physical, mental, intellectual or sensory impairments which in interaction with various barriers may hinder their full and effective participation in society on an equal basis with others (Article 3(1) EAA).

5.2. QoS Parameters and Measurement Methods

47. The EECC and the EAA define specific services targeted to address the needs of persons with disabilities that should be of concern to NRAs when deciding about QoS parameters. Relay services refer to services which enable two-way communication between remote end-users of different modes of communication (for example text, sign, speech) by providing conversion between those modes of communication, normally by a human operator.

48. Real time text is defined in Article 3(14) of the EAA and refers to a form of text conversation in point to point situations or in multipoint conferencing where the text being entered is sent in such a way that the communication is perceived by the user as being continuous on a character-by-character basis.

49. According to Article 2(35) of the EECC total conversation service means a multimedia real time conversation service that provides bidirectional symmetric real time transfer of motion video, real time text and voice between users in two or more locations.

50. NRAs should note that other transparency measures concerning equivalent access for persons with disabilities are set out in Articles 102(1) and 103(1) of the EECC in particular the competent authority in coordination, where relevant, with the national regulatory authority can oblige service providers to publish details of products and services, including any functions, practices, policies and procedures and alterations in the operation of the service, specifically designed for end-users with disabilities, in accordance with European Union law harmonising accessibility requirements for products and services.

51. NRAs could accompany the QoS parameters listed in Table 1 and Table 2 by appropriate QoS parameters concerning equivalent access for persons with disabilities choosing among those listed in Table 3, bearing in mind that all QoS indicators set out in the Guidelines should, in particular, those related to IAS,

BEREC BoR (19) 189

address the specific needs of end-users with disabilities. The QoS parameters to address the specific needs of end-users with disabilities listed in Table 3 are therefore considered additional to the QoS parameters in Tables 1 and 2.

Table 3 – QoS Parameters for end-users with disabilities

Service QoS Parameters Definition Measurement method

Voice Audio bandwidth ETSI EN 301 549 v2.1.2 (clause ETSI EN 301 549 v2.1.2 communication for speech 6.1) (clause C.6.1) Where ICT provides two-way voice communication, in order to provide good audio quality, that ICT shall be able to encode and decode two-way voice communication with a frequency range with an upper limit of at least 7 000 Hz.

Real-Time Text Distinguishable ETSI EN 301 549 v2.1.2 ETSI EN 301 549 v2.1.2 (RTT) display (clause C.6.2.2.) Where ICT has RTT send and receive capabilities, displayed sent text shall be visually differentiated from and separated from received text.

Real-Time Text Programmatically ETSI EN 301 549 v2.1.2 (clause ETSI EN 301 549 v2.1.2 determinable 6.) (clause C.6.2.2.) send and receive Where ICT has RTT send and direction receive capabilities, the send/receive direction of transmitted text shall be programmatically determinable, unless the RTT has closed functionality.

Real-Time Text Interoperability ETSI EN 301 549 v2.1.2 (clause ETSI EN 301 549 v2.1.2 6.2.3) (clause C.6.2.3.)

Where ICT with RTT functionality interoperates with other ICT with RTT functionality they shall support at least one of the four RTT interoperability mechanisms described in clause 6.2.3.

17

BEREC BoR (19) 189

Service QoS Parameters Definition Measurement method

Real-Time Text RTT ETSI EN 301 549 v2.1.2 (clause ETSI EN 301 549 v2.1.2 Responsiveness 6.2.4) (clause C.6.2.4.)

Where ICT utilises RTT input, that RTT input shall be transmitted to the ICT network supporting RTT within 1 second of the input entry.

Video Resolution ETSI EN 301 549 v2.1.2 (clause ETSI EN 301 549 v2.1.2 communication 6.5.2) (clause C.6.5.2.)

Where ICT that provides two-way voice communication includes real-time video functionality, the ICT a) shall support at least QCIF resolution; b) should preferably support at least CIF resolution.

Video Frame Rate ETSI EN 301 549 v2.1.2 (clause ETSI EN 301 549 v2.1.2 communication 6.5.3) (clause C.6.5.3)

Where ICT that provides two-way voice communication includes real-time video functionality, the ICT: a) shall support a frame rate of at least 12 frames per second (FPS); b) should preferably support a frame rate of at least 20 frames per second (FPS) with or without sign language in the video stream.

Video Synchronization ETSI EN 301 549 v2.1.2 (clause ETSI EN 301 549 v2.1.2 communication between audio 6.5.4) (clause C.6.5.4) and video Where ICT that provides two-way voice communication includes real-time video functionality, the ICT should ensure a maximum time difference of 100 ms between the speech and video presented to the user.

18

BEREC BoR (19) 189 Consultation Question 2

2. According to Article 104 of the EECC information required from providers on the quality of their services and on the measures taken to ensure equivalence in access for end-users with disabilities should be comparable, reliable, user-friendly and up-to- date. Do you believe the parameters and measurement methods in Table 3 are suitable for this purpose? If not, please explain why and the possible changes that could be made to improve the information.

6. Publication of information

52. NRAs should note that the publication requirements set in accordance to Article 104(1) are in addition to the transparency measures provided in Articles 102 and 103 of the EECC and the transparency obligations set in Article 4(1) of Regulation (EU) 2015/2120.

53. According to Recital 271 of the EECC NRAs should be able to require publication of information described in Article 104(1) of the EECC where it is demonstrated that such information is not effectively available to the public.

54. Detailed guidelines for the transparency measures for ensuring open internet access can also be found in BEREC Guidelines on the Implementation by National Regulators of European Net Neutrality Rules, , which are currently being updated.

55. According to Article 104(1) of the EECC, the information on QoS required by NRAs should be comprehensive, comparable, reliable, user-friendly and up-to-date.

56. The requirement that information is “comprehensive” and “user-friendly” means that it should be complete/statistically representative as well as understood by members of the intended audience. NRAs should look to ensure that service providers adhere to the following practices in order to ensure that information is user friendly: • promote the use of relevant standards; • preferably using clear and plain language, in as simple a manner as possible, avoiding complex sentence and language structures. The information should be concrete and definitive; • it should not be phrased in abstract or ambivalent terms; • avoid unduly technical terminology; and • it should not include too detailed information.

57. Information is “comparable” if the same relevant information is presented, by different providers or by the same providers for different offers, for comparison in such a way that it can show differences and similarities. Information should be comparable at least between different offers, and between different service providers. 19

BEREC BoR (19) 189

58. The “reliable” element means that information should be correct and cannot be misleading for end-users. Information shall comply with standards and measurement methodology indicated by NRAs, preferably using certified mechanisms if such mechanisms were introduced in a given Member State.

59. End-users should be able to check the information related to their current situation and to do so published information shall be up-to-date. NRAs shall ensure that service providers are obliged to regularly update publications by indicating the period of update. As well as information concerning QoS parameters, service providers can be obliged to publish information showing the most recent update of data at a minimum frequency on an annual basis.

60. Information should be accessible for broadest possible group of end-users including in particular end-users with disabilities, elderly end-users and end-users with special social needs. To achieve that aim NRAs could oblige service providers to publish information: - in machine-readable manner and in an accessible format for end-users with disabilities taking into account general accessibility requirements set in Section III of Annex I of the EAA and European standards aiming to address the needs of persons with disabilities and older persons, dealing with accessibility by applying the Design for all approach - ETSI EG 202 952 ; - on the websites (no more than one click from the /homepage) and via mobile applications that are viewable, operable, understandable and robust and meets harmonised standards published in accordance with regulation (EU) No 1025/2012 and directive 2016/2120 .

61. NRAs could oblige service providers to directly publish information via their own communication channels (direct approach) or to oblige service providers to publish information through third party and provide information to NRAs to publish simultaneously on NRAs websites.

62. According to Recital 271 of the EECC, NRAs should be empowered to monitor the QoS and to collect systematically information on the QoS offered by providers on the basis of criteria which allow comparability between service providers and between Member States. To achieve these objectives NRAs could require service providers in accordance to Article 104(1) of the EECC to publish information having regard to different levels of aggregation (regional, national) or different groups of end-users (business clients, consumers), depending on the level of availability of information to the public, QoS parameter or service.

63. To that end, and to enhance overall publication, some consideration of QoE (quality of experience) indicators shall be included whenever possible.

Consultation Question 3

3. Do you agree with the Guidelines outlined above covering Publication of Information? Please provide comments if any. 20

BEREC BoR (19) 189

7. Quality Certification mechanisms

64. Article 104(2) EECC refers to “quality certification mechanisms”; NRAs shall specify the quality of service parameters to be measured, the applicable measurement methods, and the content, form and manner of the information to be published, including possible quality certification mechanisms. Moreover, Article 4(4) of the TSM Regulation (Regulation (EU) 2015/2120) refers to the quality monitoring mechanism certified by an NRA.

65. The EECC does not require Member States or an NRA to establish or certify a monitoring mechanism.

66. Plural “quality certification mechanisms” used in Article 104 of the EECC anticipates the possibility of functioning of more than one certification mechanism, e.g., for internet access services and publicly available interpersonal communications services.). With regard to IAS, Article 4(4) of Regulation (EU) 2015/2120 stipulates that if the NRA provides a monitoring mechanism for IAS implemented for this purpose, it should be considered as a certified monitoring mechanism.

67. EECC provisions do not prescribe who may be a provider of a quality certification mechanism.

68. NRAs or other competent entities must take into account the requirement of independence of the provider of the quality certification mechanism from IAS and publicly available ICS providers. In this context, the NRA or entity may take into account not only circumstances pointing to capital or personal links with telecommunications service providers operating in the market, but also the business model of the quality certification mechanism provider.

69. The approach taken by the NRA or other competent entity to choose or award the certification of the quality monitoring mechanism may take many various forms. Provisions of the EECC do not impose requirements on the certification procedure. The level of formalization of the procedure as well as additional requirements, such as the requirement for a specific form of the certification act (e.g. an administrative decision, ordinance) may be determined in national law.

70. The EECC regulation does not set out requirements about the certification period, the conditions for the certification withdrawal, or extending the certification.

71. The NRA or other competent entity should determine what factors are to be taken into account when choosing a quality certification mechanism. The certification shall ensure that the quality monitoring fulfils requirements, such as: 1. Accuracy - The results of measurements should be accurate as far as it is possible in accordance with the state-of-art knowledge and with the reservation that the end-user or consumer should not be loaded with disproportionate obligations associated with performance of measurements, in particular, if these requirements do not have a significant impact on the result. Achieving this objective cannot limit the availability of the mechanism for quality monitoring for all end-users. When assessing the factors that can affect the accuracy and reliability of measuring the quality of 21

BEREC BoR (19) 189

the IAS and other publicly available ICS, it may be appropriate for NRAs to specify the requirements that should be met by the end-user environment . 2. Enables comparison of measurements - The quality monitoring mechanism should make it possible to compare the results of the QoS measurements carried out with the level of service quality guaranteed in the contract. Feedback received by the end-user or consumer as a result of the measurements carried out should be sufficient for him or her to draw an independent conclusion regarding the quality of the service under examination. The quality monitoring mechanism may provide the possibility of transferring the values of contractual parameters (e.g. through the provision by the end-user or consumer) and the listing together with the results of the measurements conducted. 3. Openness - The measurement methodology and implementation should be publicly available, and the NRA or other competent entity should consider publishing information on factors which can affect the reliability of results, if such factors have been identified. The openness of the quality monitoring mechanism can be achieved by publication of its source code. 4. Safety - The quality monitoring mechanism should be adequately safeguarded against attacks, its integrity and the confidentiality of processed personal data against unauthorized access. 5. Future-proofness - Quality monitoring mechanisms should be based on the current state of technical knowledge, and its design, taking into account the development and evolution of the telecommunications market. 6. Accessibility - The use of the quality monitoring mechanisms should be accessible to people with disabilities.

Consultation Question 4

4. Do you agree with the Guidelines on Quality Certification mechanisms? Please provide comments if any.

8. Review of Guidelines

8.1 Review Period

72. The process of undertaking a review of the Guidelines will commence 2 years from the publication of the Guidelines. Subsequent reviews will be determined by BEREC and will be agreed and set out in future BEREC work programmes. 22

BEREC BoR (19) 189

Annex 1 EECC Article 104 and Annex X

L 321/178 EN Official Journal of the European Union 17.12.2018

Article 104 - QoS related to IAS and publicly available interpersonal communications services

1. National regulatory authorities in coordination with other competent authorities may require providers of IAS and of publicly available interpersonal communications services to publish comprehensive, comparable, reliable, user-friendly and up-to-date information for end-users on the quality of their services, to the extent that they control at least some elements of the network either directly or by virtue of a service level agreement to that effect, and on measures taken to ensure equivalence in access for end-users with disabilities. National regulatory authorities in coordination with other competent authorities may also require providers of publicly available interpersonal communication services to inform consumers if the quality of the services they provide depends on any external factors, such as control of signal transmission or network connectivity. That information shall, on request, be supplied to the national regulatory and, where relevant, to other competent authorities before its publication. The measures to ensure QoS shall comply with Regulation (EU) 2015/2120.

2. National regulatory authorities in coordination with other competent authorities shall specify, taking utmost account of BEREC guidelines, the QoS parameters to be measured, the applicable measurement methods, and the content, form and manner of the information to be published, including possible quality certification mechanisms. Where appropriate, the parameters, definitions and measurement methods set out in Annex X shall be used. By 21 June 2020, in order to contribute to a consistent application of this paragraph and of Annex X, BEREC shall, after consulting stakeholders and in close cooperation with the Commission, adopt guidelines detailing the relevant QoS parameters, including parameters relevant for end-users with disabilities, the applicable measurement methods, the content and format of publication of the information, and quality certification mechanisms. ANNEX X- QUALITY OF SERVICE PARAMETERS Quality-of-Service Parameters, Definitions and Measurement Methods referred to in Article 104 For providers of access to a public electronic communications network

PARAMETER DEFINITION MEASUREMENT METHOD (Note 1) Supply time for initial ETSI EG 202 057 ETSI EG 202 057 connection

23

BEREC BoR (19) 189 Fault rate per access line ETSI EG 202 057 ETSI EG 202 057 Fault repair time ETSI EG 202 057 ETSI EG 202 057

For providers of interpersonal communications services who exert control over at least some elements of the network or have a service level agreement to that effect with undertakings providing access to the network PARAMETER DEFINITION MEASUREMENT METHOD (Note 2) Call set up time ETSI EG 202 057 ETSI EG 202 057 Bill correctness ETSI EG 202 057 ETSI EG 202 057 complaints Voice connection quality ETSI EG 202 057 ETSI EG 202 057 Dropped call ratio ETSI EG 202 057 ETSI EG 202 057 Unsuccessful call ratio ETSI EG 202 057 ETSI EG 202 057 (Note 2) Failure probability Call signalling delays

Note 1 Parameters shall allow for performance to be analysed at a regional level (namely, no less than level 2 in the Nomenclature of Territorial Units for Statistics (NUTS) established by Eurostat). Note 2 Member States may decide not to require up-to-date information concerning the performance for those two parameters to be kept if evidence is available to show that performance in those two areas is satisfactory 24

BEREC BoR (19) 189

Recitals:

(260) The specificities of the electronic communications sector require, beyond horizontal contract rules, a limited number of additional end-user protection provisions. End-users should be informed, inter alia, of any quality of service levels offered, conditions for promotions and termination of contracts, applicable tariff plans and tariffs for services subject to particular pricing conditions.

(271) National regulatory authorities in coordination with other competent authorities, or where relevant, other competent authorities in co-ordination with national regulatory authorities should be empowered to monitor the quality of services and to collect systematically information on the quality of services offered by providers of internet access services and of publicly available interpersonal communications services, to the extent that the latter are able to offer minimum levels of service quality either through control of at least some elements of the network or by virtue of a service level agreement to that end, including the quality related to the provision of services to end-users with disabilities. That information should be collected on the basis of criteria which allow comparability between service providers and between Member States. Providers of such electronic communications services, operating in a competitive environment, are likely to make adequate and up-to-date information on their services publicly available for reasons of commercial advantage. National regulatory authorities in coordination with other competent authorities, or where relevant, other competent authorities in co-ordination with national regulatory authorities should nonetheless be able to require publication of such information where it is demonstrated that such information is not effectively available to the public. Where the quality of services of publicly available interpersonal communication services depends on any external factors, such as control of signal transmission or network connectivity, national regulatory authorities in coordination with other competent authorities should be able to require providers of such services to inform their consumers accordingly.

(272) National regulatory authorities in coordination with other competent authorities should also set out the measurement methods to be applied by the service providers in order to improve the comparability of the data provided. In order to facilitate comparability across the Union and to reduce compliance cost, BEREC should adopt guidelines on relevant quality of service parameters which national regulatory authorities in coordination with other competent authorities should take into utmost account.

25

BEREC BoR (19) 189

Annex 2 Definitions

This Annex contains a selection of terms and definitions used in the Guidelines and that could be useful to support the consistent and harmonised application of the provisions of Article 104 of the EECC. Internet access services: a publicly available electronic communications service that provides access to the internet, and thereby connectivity to virtually all end points of the internet, irrespective of the network technology and terminal equipment used (Regulation (EU) 2015/2120). Interpersonal communications services: a service normally provided for remuneration that enables direct interpersonal and interactive exchange of information via electronic communications networks between a finite number of persons, whereby the persons initiating or participating in the communication determine its recipient(s) and does not include services which enable interpersonal and interactive communication merely as a minor ancillary feature that is intrinsically linked to another service (Directive (EU) 2018/1972). Number-based interpersonal communications service: an interpersonal communications service which connects with publicly assigned numbering resources, namely, a number or numbers in national or international numbering plans, or which enables communication with a number or numbers in national or international numbering plans (Directive (EU) 2018/1972). Number-independent interpersonal communications service: an interpersonal communications service which does not connect with publicly assigned numbering resources, namely, a number or numbers in national or international numbering plans, or which does not enable communication with a number or numbers in national or international numbering plans (Directive (EU) 2018/1972). Quality of service: totality of characteristics of a telecommunications service that bear on its ability to satisfy stated and implied needs of the user of the service (ITU-T Rec. E.800).

Definitions of parameters referred to in Article 104 of Directive (EU) 2018/1972.

Supply time: the duration from the instant of a valid service order being received by a direct service provider to the instant a working service is made available for use. This should include cases where: a new access line is installed; an existing access line is taken over by another customer; an additional access line is provided to a customer who already has service. It is measured by: a) the times by which the fastest 50%, 95% and 99% of orders are completed; b) the percentage of orders completed by the date agreed with the customer and, where the percentage of orders completed by the date agreed with the customer is below 80%, the average number of days, for the late orders, by which the agreed date is exceeded (ETSI EG 202 057-2 V1.3.1). Fault rate per access lines: number of valid fault reports per access line per year (ETSI EG 202 057-1 V1.3.1). Fault repair time for fixed access lines: the duration from the instant a fault has been notified by the customer to the published point of contact of the service provider to the instant when the service element or service has been restored to normal working order. It is measured by: a) the time by which the fastest 80% and 95% of valid faults on access lines are repaired (expressed in clock hours); b) the percentage of faults cleared any time

26

BEREC BoR (19) 189

stated as an objective by the service provider; c) the provision of information on the hours during which faults may be reported (ETSI EG 202 057-2 V1.3.1). Call set up time: the period starting when the address information required for setting up a call is received by the network and finishing when the called party busy tone or ringing tone or answer signal is received by the calling. It is measured by: a) the mean value in seconds for national calls; b) the time in seconds within which the fastest 95% of national calls are set-up; c) the mean value in seconds for international calls; d) the time in seconds within which the fastest 95% of international calls are set-up; e) the number of observations performed for national and international calls (ETSI EG 202 057-2 V1.3.1 and ETSI EG 202 057-2). Bill correctness complaints: the proportion of bills resulting in a customer complaint about the correctness of a given bill (ETSI EG 202 057-1 V1.3.1). Speech connection quality: the end-user perceived speech quality. It depends on: a) quality category according to ITU-T Recommendation G.109 [i.14]; b) characteristics of terminals; c) reference connections (ETSI EG 202 057-2). Call drop rate: the probability of a call terminating without the user’s action (ITU-T E.807). Unsuccessful call ratio: the ratio of unsuccessful calls to the total number of call attempts in a specified time period. It is measured by: a) the percentage of unsuccessful calls for national calls; b) the percentage of unsuccessful calls for international calls; c) the number of observations used for national and international calls together with absolute accuracy (ETSI EG 202 057-2 V1.3.1). Call setup failure probability: the ratio of total call setup attempts that result in call setup failure to the total call setup attempts in a population of interest (ETSI TS 102 024-9 V4.1.1). Call signalling delays: it involves three different scenarios: call setup, call answer and call release. The call set up signalling delay is the time between the calling terminal providing sufficient address information to set up the call, and the calling party receiving a confirmation from the called terminal that the called party is being alerted (ETSI TS 102 024-9 V4.1.1). Delay: the time between the first bit entering the network and the first bit arriving at the user across the network. It includes three factors: transmission delay (caused by the data rate of the link), propagation delay (the amount of time spent for the traffic to travel from the sender to the receiver) and node processing delay (the time spent for in-node processing, such as output link selection, bit errors check and queuing delay) (ITU-T Y.2617).

27

BEREC BoR (19) 189

Annex 3 Benchmarking

NRAs Questionnaire

NRAs were asked to respond to a questionnaire that contained nine questions. The purpose of the questionnaire was to assess the current level of harmonisation of the quality of service standards/guidelines available in MS for IAS and publicly available ICS under the relevant provisions of article 22 of the Universal Service Directive. This section contains the aggregated results for each question and analysis of the reported parameters and processes which are currently available in each of the MS. More specifically the questions covered the following key areas in relation to Quality of Service (QoS):

• The QoS parameters in place • Guidelines/measures detailing the relevant QoS parameters • QoS parameters applicable for end-users with disabilities • Applicable measurement methods for these QoS parameters • Content, form and manner of the QoS information to be published • Quality certification mechanisms • Customer satisfaction surveys • Measurement of call waiting times for customer support • Compliance cases with respect to QoS

27 responses were received from NRAs.

Quality of Service measures in place

NRAs were asked to indicate what QoS measures are in place in their country. A wide range of QoS measures are listed in the responses for ICS and IAS. However, for fixed ICS, network performance measures stand out as the key measures in place, i.e., supply time for connection, fault rate per access line and fault repair times. Indeed some of the responses detail the full range of measures listed in Annex III of Directive 2002/22/EC to include measures relating to directory inquiry services, bill correctness, working order of public payphones and call set up time. Other measures include access to Text Relay Services, provision of information to consumers regarding the indicators measured, publication of performance in relation to measures, requirement to incorporate QoS parameters in users’ contracts, frequency of clients’ complaints, resolution time for complaints, complaints on pre-paid lines and measures taken by providers to ensure equal access for end-users with a physical disability. A number of QoS measures are also in place for mobile networks to include quality of voice and data services such as network coverage, drop call rate, rate of successful SMS/MMS transfer, bit rate error, frame rate error based on ITU technical standards, throughput, packet loss, delay and jitter. In one specific case a QoS audit is performed on the mobile services to assess the QoS that mobile operators provide to users for benchmarking purposes so that users experience is reflected in various scenarios (in the city, rural areas, different forms of transport, etc) and to also include the services most used (calling, texting, web browsing,

28

BEREC BoR (19) 189

video streaming, file downloads, etc). The NRA in question also requires the MNO to publish daily a list of antenna that are out of order/malfunctioning. QoS measures are also in place for IAS to include minimum guaranteed IAS speed values for fixed and mobile networks. Operators are required to measure the achievable speeds on their networks and publish the results periodically/annually on their websites. For fixed broadband measures to include data transmission speeds, availability for internet access, latency, packet loss. Drive tests/ railways/train journeys are used to measure speeds which are then published on the website and may be used in certain cases to evaluate complaints received in this regard.

Guidelines or measures detailing the relevant quality of service parameters

NRAs were asked to indicate if they had guidelines or measures detailing the relevant QoS parameters in their country. In many cases Regulations/Guidelines are in place specifying the quality levels/targets for ECS, methodologies used to collect and analyse performance data, inspection criteria, the submission of performance reports and the format and publication of same. In other cases the terms of the USO sets guidance regarding QoS targets and technical requirements. A number of NRAs have issued rules in relation to the publication of internet speeds by IAS providers in line with Article 4 of the NN Regulations. The procedure established in ETSI standards are also relevant.

Parameters applicable for end-users with disabilities

NRAs were asked to indicate what QoS parameters are available for end-users with disabilities. Of the respondents to this question a majority of NRAs stated that there were no specific QoS parameters for end-users with disabilities. However, twelve respondents provided details of available measures relevant to endusers with disabilities; a number of which, as listed below, identified unique quality of service associated with the services provided. A key theme emerging from the respondents was that service providers are obliged to ensure that end-users with disabilities obtain equivalent levels of access to services and choice of undertakings as those enjoyed by the majority of end-users. Also in many instances NRAs oblige the USP to provide specific services under the USO for end-users with disabilities to include some of the following: - Access to the emergency services via text messages - Directory enquiry services and directories - Text relay services and sign language services - Publish availability of adapted terminal equipment to meet users’ needs - Special pricing plans for deaf and blind users - Telephone cabins with voice assistance - Priority fault repair - Accessibility, safety, information, comfort and convenience - Accessible bill formats - A visually impaired user shall have unrestricted access to the services required for the customer relationship

29

BEREC BoR (19) 189

- providing appropriate devices in entities serving end-users - provide relevant formats of information regarding facilities offered and model contracts - provide appropriate terminal equipment and assistance in configuration of device in premises The following examples provided in the responses outline specific examples of the available measures relevant to end-users with disabilities and their associated quality of service metrics to include the following services: 1. Fault Repair Time of <or equal to 48hours 2. Providers are obliged to offer ECS for deaf, hearing-impaired, blind-deaf or aphasic end-users with a simultaneous written and visual transcription, free of charge. Conditions of quality associated with the offer include; - QoS target >70% of users accessing an interpreter within 3 minutes - rate of users accessing an interpreter within 30 seconds; - dropout rate; - QoS target >99% for availability; - quality of experience ranging from 1 (poor) to 5 (excellent). 3. For hearing or speech impaired users the subscription; - must allow internet connection for the use of video call and remote interpreting services with a speed of at least 512 kilobit/s for incoming and outgoing traffic; - a one-way delay in an IAS provide as a universal service to persons with hearing and speech impairments for the purpose of video conference and remote interpreting services must not exceed 150ms in the access network of a USP. 4. Accessible obligation including the following: - ensure the website and the information available for end-users with disabilities is compliant with the WCAG, AA Level. One respondent stated that the General Conditions for service providers specifically require that providers must ensure the following: - Access to directory information, - Relay services, - Mobiles SMS access to emergency organisations, - Priority fault repair, - Third party bill management and - Bills and contracts in accessible formats as appropriate. More generally one NRA stated that Operators are obliged to publish and submit an annual report to the NRA outlining the measures provided for accessibility of end-users with disabilities and state whether they voluntarily include indicators for their actions.

Applicable measurement methods for Quality of Service parameters

NRAs were asked to indicate measurement methods applied. One NRA stated that there were no applicable measurement methods in place and that very few complaints were received. However, they planned to implement the BEREC tool for IAS measurement. Based on the remaining NRA responses, many of the measurement

30

BEREC BoR (19) 189

methods for QoS of ICS parameters are based on various iterations of the ETSI standards and where the service providers are required to submit performance data to the NRAs. With regard to IAS measurements there are a range of methodologies used to include interactive real time performance test tools developed or provided by the NRA, a testing methodology based on Standard ITU-T Y/1564, by means of controlled probes that are installed in each region, by users in a crowdsourced approach (based on the BEREC NN Reg Assessment Methodology), which provides the user with certified results of the QoS of the connection at the time of the test, passive data collection method within the IAS network, drive tests, surveys based on data collected from internet speed test providers.

Content and format of publication of the information

NRAs were asked to indicate the content and format of the information. It is clear from all the responses received that there are various requirements for content of information with less emphasis on the format of publication of information. Many NRA responses listed the many parameters that are published periodically (some quarterly, every 6 months, or annually) on their websites or the operators websites, with some specifying that information must be provided at regional and national level. One NRA stated that content, format, time limits or manner of publishing information regarding QoS is not specified but the information has to be clear, legible and simple. Other NRAs listed requirements only for IAS parameters to include upload/download speed, delay, delay variation, limitations regarding terminal equipment usage and response times. 2 NRAs indicated that online interactive mapping tools are provided which allow users to view all QoS data collected, with filtering settings for various parameters, and track progress that operators have made on their networks to improve QoS. Some responses provided very detailed requirements about providing terms and conditions of services to be made available in writing with clear and comprehensive information made available at stores for viewing or available online for consultation prior to entering into contracts. One NRA provides a detailed report comparing service quality for providers of fixed, mobile and broadband services so that consumers are informed should they wish to choose a new provider.

Quality certification mechanisms

NRAs were asked to indicate if there are quality certification mechanisms in place in their countries. 4 NRAs stated that there are certified quality measurement systems in place for the measurement of IAS. 1 NRA reported that there is a requirement only for voice call systems (number based communications) to have the metrological certificate. The vast majority of NRAs stated that there are no quality certification mechanisms in place at the time of responding to the questionnaire. Moreover, a number of NRAs have an online tool in place that users can check internet quality parameters.

31

BEREC BoR (19) 189 Consumer satisfaction measures/ Indicators for measurement of call waiting times for customer support

NRAs were asked to indicate if consumer satisfaction measures and call waiting times for customer support were measured in their country. Approximately half of the responses received from NRAs indicated that they do not have consumer satisfaction measures in place. However, where measures are in place to ascertain consumer satisfaction, these are conducted in some cases by the NRA, in other cases the operator is obliged to carry out the research and in certain instances market research companies carry out the surveys on behalf of the NRA. Consumer perception/satisfaction surveys are conducted biennially in some instances, annually in other cases and every two years in one case. A number of responses indicated that, apart from questions about general consumer satisfaction, questions are tailored to obtain discrete information and concerns that consumers are experiencing in their country at the time of the survey including topics such as: - Satisfaction with mobile coverage at home location and travelling locations - The quality of helpdesks- are the correct answers provided and waiting times - Impact of fibre optic access on behaviours and activities on the Internet Other responses stated that some measures are in place to gauge consumer satisfaction such as: - indirectly measure consumer satisfaction by assessing the no of users’ complaints - performance against QoS parameters for fixed telephone and internet service access: bill correctness complaints ratio/target value <1% Nine NRAs responded reported various indicators that are in place for the measurement of call waiting times for customer support to include the following: - Duration of the voice response system main menu <or equal to 45sec - Time waiting until before option to connect to the person in the call centre < or equal to 20sec - Non-binding direction on customer service which requires operators to answer consumer calls within 5 minutes – no measurement in place - Comparison of average waiting times reported in previous calendar year for major landline, mobile and broadband providers - Measurements are in place for connections time for USO - Average response time by helpdesk (seconds) and the % of calls responded to within 2 minutes - Response time for operator service - Average response time < 20 sec – ETSI EG 201 769-1 - Customer service response time – operators customer support service may not >15 sec annually and % of calls answered by operators customer support service in 20 sec or less may not <80% annually – ETSI EG 202 057-1

32

BEREC BoR (19) 189 Compliance cases with respect to Quality of Service

NRAs were asked to provide details of compliance cases with respect to QoS. A majority of NRAs responses indicated one of the following: NRA did not respond to the question, NRA stated that it was not applicable, NRA does not deal with QoS cases or the NRA stated that there were no specific QoS compliance cases. However, a number of respondents provided various levels of information regarding compliance cases; many of which related to QoS requirements under the USO. Also a number of established processes observed where non-compliance issues are detected and forms of resolution are described in the responses to include the following: 1. Monitoring of the market to establish non-compliance. The NRA and provider collaborate /engage to reach a resolution without the need for court action. 2. The NRA monitors end-users complaints regarding ICS and IAS and conducts a comprehensive evaluation which is published each quarter. 3. Where the USP fails to comply with targets, the NRA notifies the USP and in certain cases fines may be imposed. 4. Where end-users complaints are detected, the NRA may request the operator to pay compensation to the end-user or allow the end-user withdraw from the contract without a penalty. 5. One NRA stated that <10% of complaints related to QoS in 2018. 6. Many of the complaints reported to an NRA regarding QoS of IAS were attributable in the main to mobile networks as opposed to fixed networks.

33

BEREC BoR (19) 189

Annex 4 Other Benchmarking

2016 Study prepared for the EC – Fixed and Mobile Convergence in Europe – Quality Measurements for 5G and Network Densification Source EC Fixed and Mobile Convergence in Europe Quality Measurements for 5G and Network Densification Table 2.2

34

BEREC BoR (19) 189

Annex 5 Consultation Questions

Consultation Question 1

1. According to Article 104 of the EECC information required from providers on the quality of their services should be comparable, reliable, user-friendly and up-to- date. Do you believe the parameters and measurement methods in Table 2 are suitable for this purpose? If not, please explain why and the possible changes that could be made to improve the information.

Consultation Question 2

2. According to Article 104 of the EECC information required from providers on the quality of their services and on the measures taken to ensure equivalence in access for end-users with disabilities should be comparable, reliable, user-friendly and up-to- date. Do you believe the parameters and measurement methods in Table 3 are suitable for this purpose? If not, please explain why and the possible changes that could be made to improve the information.

Consultation Question 3

3. Do you agree with the Guidelines outlined above covering Publication of Information? Please provide comments if any?

Consultation Question 4

4. Do you agree with the Guidelines on Quality Certification mechanisms? Please provide comments if any? 35