Aussie Broadband hit the #1 Australian ISP according to Steam (and why we don’t care about it)

This week, Aussie Broadband topped the Steam charts for the highest average download rate in Australia (we’re currently at 35.6 Mbps as of this article).*

Being “#1 in Australia” sounds great, but unlike some we’re not going to be plastering it all over our marketing because we don’t believe the ranking is relevant when comparing providers.

Here’s why.

What is Steam actually measuring?

Steam does not actually measure the maximum speed of your internet connection. Instead, they only monitor how fast each user is downloading from the Steam server. This measurement is then averaged out equally amongst all users for a particular Internet Service Provider (ISP), giving the ‘Average Download Rate’ that is used on the Steam website.

So what does this mean when it comes to comparing ISP performance?

The problem with averaging the speeds equally in this way is that it assumes everyone’s connection speeds are the same, which isn’t the case.

For example, let’s assume two ISPs are both uncongested, and all of their users can achieve the full speed of their connection at all times of day. ISP A has nine users with a 100mbps plan and one user with a 12mbps plan, whereas ISP B has one user with a 100mbps plan and 9 users with a 12mbps plan.

Even if the performance of both ISPs is identical, and everyone downloads at their full speeds, the Steam statistics would be far higher for ISP A (91.2mbps) than ISP B (20.8mbps). This makes ISP B look far worse than ISP A when actually they are the same.

Can the statistics be changed to be used as a comparison?

If each ISP on Steam’s Top 10 ISP list had an equal proportion of users on each speed tier, and each of those clients downloaded from Steam in a consistent manner, then the statistics might be able to be used to compare the actual performance of each ISP.

Unfortunately it doesn’t work like that in reality: some ISPs have a large proportion of high speed tier customers, while others have more low speed tier customers. The NBN Wholesale Market Indicators report (March 2018), which is published by the ACCC, explains this in better detail:

Looking at the above data, an ISP like TPG is never likely to achieve a high result on Steam Australia as they have a large proportion of 12 Mbps customers. This is not even taking any legacy non-nbn services like ADSL into consideration (which some ISPs still have many of). In contrast, an ISP like MyRepublic should perform quite well as they have almost exclusively sold 100 Mbps services to their on-net nbn customers.

What about the Netflix ISP Speed Index?

The Netflix index is just as bad for comparing ISP performance.

In their case, Netflix isn’t measuring the full speed of your connection like Steam does. Instead, they only measure the average download rate of the content you are streaming, which is usually a lot less than your connection speed.

As a result, if an ISP’s customer base mainly watches Ultra HD content, their Netflix results will be higher than an ISP with customers who prefer SD or HD content, even though the performance of both ISPs might be the same. This is because more bandwidth is required to stream higher definition content, which raises the average download rate for that ISP.

It’s worth pointing out Netflix themselves say their ranking is not a measure of the overall performance of an ISP’s network, but of course that doesn’t stop people from doing exactly that.

So, how can you measure the performance of an ISP?

While both the Steam and Netflix leader boards are interesting, unfortunately neither provide an accurate representation of the performance of an ISP.

The most reliable (but still not perfect) comparison is probably the ACCC’s Measuring Broadband Australia program, which uses purpose-built devices located in a customer’s house to measure the speed of their internet connection. There are only four ISPs listed at the moment but we understand another two will be added in the next report (due in early August).

Ultimately, however, one of the best indicators is feedback from current customers – you can check out reviews of different providers online using sites like ProductReview, Facebook or Google and see what they say about the performance of their ISP. If you’re not happy with your current ISP then remember you can always switch!

*Due to the Steam data, we recently noticed that our legacy name Wideband Networks was still being used in some geolocation databases, so we updated it.