On 8 December 2020, Arcep published the results of its 2020 quality of service measurement campaign in Metropolitan France. Today, the QoS findings for France’s overseas departments and territories are being presented and published on the “Mon réseau mobile” website.
These publications will give consumers in the overseas markets the ability to compare operators’ performances, and for decision-makers to have a detailed diagnosis of the status of mobile connectivity in their area. All of this information is available as open data.
Calling quality is improving significantly, but more mixed results for mobile internet
Quality of service for calling and texting is improving, in every territory and for every operator. The results are more mixed when it comes to data services, however: some indicators, such as average speeds, continue to improve, but web browsing and streaming quality appear to be stagnating, and in some cases has deteriorated compared to 2019. This can be explained by the period during which the measurements were performed (September to December in 2020, versus July and August in 2019) and by the effects of the Covid-19 crisis which put added pressure on the networks.
The results of this campaign should be assessed territory by territory:
- In Reunion: a high quality of service overall, and especially high calling quality for which operators have similar scores. Orange and SFR are neck and neck on all of the indicators, with Orange having a slight advantage when it comes to connection speeds. For ZEOP, measurements are being taken more and more of its “own” network, which the operator continues to deploy. Meanwhile, Free has improved its quality of service since last year, and is now more or less tied with ZEOP.
- In Mayotte: SRR, Orange and Free each stand out on different indicators: if SRR provides a better web browsing experience, Orange provides better quality voice services and faster connections. Free continues to deliver high quality of service for voice calling and texting, and has made real strides on its data services: it tops the rankings on several indicators including streaming quality. Quality of service is still lagging for Maore Mobile (largely due to its still limited coverage) but it has improved since last year.
- In Guadeloupe and Martinique: the quality of voice and data services continues to improve, and has reached levels close to those found in Metropolitan France, particularly in Martinique. Orange leads the way on virtually every criterion, and maintains its lead over SFR, which is itself ahead of Digicel.
- In Guiana: on average, the quality of every operator’s data services has improved but continues to lag behind QoS in the other overseas territories. Orange and SFR have both made significant progress, and Orange has kept its lead on most indicators. Digicel is in third place with a quality of service that remained unchanged between 2019 and 2020. The gap between operators is very slight when it comes to voice services: depending on the indicators tested, it is either Orange or Digicel delivering the strongest performance.
In addition, measurements taken in the field revealed weak performances from Outremer Telecom in remote parts of Haut-Maroni, where it is the only operator reporting 4G coverage. It is indeed providing LTE coverage but using a satellite backhaul solution, with speeds topping out at 1.5 Mbit/s in these areas during the measurement campaign, with an average speed of 600 kbit/s.
- In Saint-Barthélemy, results for all the operators remain unchanged between 2019 and 2020, and in some cases are inferior to those obtained last year on several indicators. Orange maintains its lead over Digicel and Dauphin Telecom.
- In Saint-Martin, Orange is the only operator to have made significant progress since 2019, and has consolidated its lead, whether it comes to voice services or web browsing experience. With regard to web browsing, next in the rankings are Digicel then Dauphin Telecom and UTS Caraïbe. With regard to voice services, depending on the indicators, Orange is followed by Digicel, UTS or Dauphin Telecom.
There can be significant disparities in quality between operators and between the regions: Arcep invites everyone to compare them on monreseaumobile.fr, and to view the findings in accordance with their needs and their area.
New addition to the 2020 campaign: tests that better reflect the user experience
Once again this year, Arcep has made some of its tests more demanding, particularly for mobile calling services. Up until now, the tests had been performed using two mobile phones with a SIM card from the same operator. In 2020, to more closely reflect users’ actual experience, calls were made using every possible combination of operators: this testing, which therefore more closely matches users’ behaviour, reflects not only the quality of service available on a given operator’s network, but also the quality of its interconnection with the other operators.
As it did in Metropolitan France, Arcep has also introduced a new indicator: the percentage of connection speeds above 3 Mbit/s. In most cases, a connection of more than 3 Mbit/s is enough to sustain “standard” mobile internet use, such as web browsing, reading e-mails and watching most videos in 720p without any significant slowing.
Verification of theoretical coverage maps
One aspect of Arcep’s measurement campaign in the overseas territories involved verifying operators’ theoretical coverage maps: these maps, which are produced based on digital simulations, cover voice calling, SMS and mobile internet services. Arcep verified that the maps supplied by operators comply with a high level of reliability, throughout the territory. Arcep thus verified the maps in Q2 2020, which were required to have a 95% accuracy level. In 2020, Arcep increased this threshold to 98% : this new level came into effect in late 2020, and will therefore apply to Arcep’s next audit of operators’ maps.