Paris, 4 November 2011
In 2011, ARCEP performed its thirteenth consecutive annual assessment of the quality of service provided on the second and third generation mobile networks operated by Bouygues Telecom, Orange France and SFR in mainland France.
The purpose of the survey is to check that mobile network operators are complying with the obligations listed in the licences issued by ARCEP. It is part of the Authority's broader efforts to improve the information on electronic communications services that is available to consumers.
The goal of the survey is to assess the quality of voice call, SMS, MMS and WAP browsing services provided to consumers, and the speeds that can be reached on mobile networks. The quality of Web browsing services was also measured for the first time this year.
Through technical measurements taken in the field, the survey aims to provide a statistical account of the quality of the services delivered by operators in the areas covered by their networks. Its purpose is not to obtain subscribers' views of the end-to-end quality of these services - through a customer survey, for instance.
The quality of the service that is actually delivered to a user depends not only on the quality of the carrier's network but also on other parameters that are specific to the user, such as how their device performs, the contract they have with their carrier, their circumstances (outside, inside a building, in a car, in a fixed position, moving at a more or less high speed, etc.), their location with respect to the network base station, the number of users on the network, and so on - many of which may vary from moment to moment.
The methodology that the Authority employs to measure QoS therefore aims to reflect a broad range of circumstances which are representatives of consumers' various behaviour patterns.
- Quality of voice call services still high
The survey performed in 2011 confirms that the quality of voice calls over these telephone networks continues to be as high as it has for the past several years.
The service was tested in 52 towns and cities with a population of more than 10,000 - both indoors and outdoors and in a moving vehicle. The results of the tests revealed that the success rates for setting up and holding a call for two minutes and five minutes continue to very high - 97.3% and 96.2%, respectively - as they have been in previous years. The tests also revealed a perfect audio quality of the successfully completed calls that was virtually consistent.
The quality of the phone service when travelling was also measured on the main TGV (high-speed train) lines, in commuter trains for the four largest cities in France and on the most heavily used motorways.
The success rate for setting up and holding a call for two minutes on the TGV ha decreased by 3% compared to the previous survey, while the results also reveal sizeable disparities in QoS levels between the carriers.
Testing the successful completion and maintenance of calls on motorways revealed a roughly 2% decrease from the previous survey, although the percentage of calls of perfect quality has increased by 2%.
Lastly, the rate of acceptable and perfect quality calls made on commuter trains and trams has increased by 2% and 4% respectively, compared to previous survey.
- Connection speeds on mobile networks continue to rise, on average, but gaps between carriers are widening
File transfer tests were performed in the 12 largest metropolitan areas in France and in 20 towns and cities with a population of between 50,000 and 400,000, using 3G USB keys plugged into a laptop computer.
The results of these tests reveal file downloads speeds on 3G networks of up to 9.2 Mbit/s for the fastest services and, for sending files, upload speeds of more than 3 Mbit/s for the fastest services.
The average download speed of 3 Mbit/s is 40% faster than the average speed measured during the previous survey, while for file uploads an average upload speed of 1.3 Mbit/s was measured - which is slightly faster than the average 1.2 Mbit/s of the previous survey.
We also found an increased disparity in the connection speeds being supplied by the different carriers, both for downloading and uploading files. Median speeds stood at 4.8 Mbit/s for Orange France, 2.8 Mbit/s for SFR and 1.2 Mbit/s for Bouygues Telecom.
- Quality of the SMS service still very satisfactory, and MMS service quality improving
The quality of SMS and MMS services was tested in the 12 largest metropolitan areas in France, and in 20 towns and cities with a population of between 50,000 and 400,000.
The results for SMS (i.e. texting) services reveal the same high level of overall quality provided by these services for several years, with a success rate of over 99%. There has also been a significant improvement in the quality of services for sending a photo via multimedia message (MMS): the success rate for receiving an MMS in less than three minutes has increased by 2.4% in cities with a population of more than 50,000.
- Quality of internet access services measured for the first time
The quality of services for accessing the Web using a 3G key was measured for the first time this year. The two indicators that were tested were the success rate for connecting to the internet in less than 50 seconds - which was achieved in 97.5% of cases, on average - and the rate of success for achieving and holding a connection for five minutes, which stands at roughly 96%.
- First experimental findings for smartphones
Experimental measurements were also taken with smartphones for the first time this year, in the 12 largest cities in Metropolitan France. Although fewer in number, these tests made it possible to check that the measurement protocols defined for file transfers and Web browsing on these devices could be fully implemented. As with other indicators, however, they do not enable individual results for each carrier, but do nonetheless provide some useful information.
The average download speed reached on a smartphone is 2.4 Mbit/s, which is slower than what is achieved using a 3G key in the 12 biggest cities. For file transfers, the average speed of 1.4 Mbit/s is slightly faster than the average with a 3G key. Differences in download speeds compared to 3G USB keys may be attributable in part to smartphones' maximum performance levels which are below those of 3G keys, and do not appear to be due to carriers' networks.
The survey's complete findings, which include a description of the testing protocol and detailed results for each operator, are available (in French) on the ARCEP website.
The next quality of service survey for mobile networks in Metropolitan France, which will be conducted in 2012, will be based on the same methodology as the one employed for this survey but will be expanded to include new forms of consumption and a broader range of circumstances. Efforts are also being made to extend this survey to the French overseas territories and collectivities, using a protocol that could be adapted from the one being used in mainland France. The QoS survey will also eventually extend to fourth-generation systems - for which ARCEP issued the first licences on 11 October of this year, and whose performances should help further improve the quality of the services provided to users.
Alongside these efforts, ARCEP is also working on defining and measuring mobile coverage levels - the aim being to continue to add to the information that mobile carriers already publish on the matter (1).
__________________________ (1) Mobile carriers publish coverage maps that provide details on the different classes of service available across the country, and so provide information on the nature and maximum performances of the services on offer. These maps are available online (in French) at the following urls: - couverture-reseau.orange.fr; - www.sfr.fr/assistance/reseau-sfr-france; - www.couverture.bouyguestelecom.fr
Linked documents
The Survey for 2011 (pdf - 1.46MB) (pdf - 
) 