I was recently setting up a new 1.5MB DSL connection at home. After connecting my computers (wirelessly) to my DSL modem I fired off a broadband speed tester to see if both were delivering the expected throughput. My WinXP box was delivering about 0.7MB while my WinVista box was delivered about 1.2MB. Clearly I had an issue localized to the WinXP box so I tweaked its network settings with TCP Optimizer. That made its speed comparable to my other machine, both at about 1.2MB.
Reflecting upon this I asked myself: how do I really know what my speed is? I decided to test further, applying 3 different speed testers using 2 different browsers on each of 2 machines and, for good measure, I repeated each test twice, yielding 24 data points (well, OK, 48 since there was upstream and downstream). The raw data appears below. Note that the data is sorted by provider then by speed (in Kbps) as the most useful order to see trends--they are not shown in the order of trials performed. Also note that I shutdown all other foreground apps and windows but did not meticulously check service activity so that could possibly account for some of the glitches.
Observations on downstream activity
- There are 3 outliers (2 from Speed.IO on the XP box only, and one on from AuditMyPC on Vista) and all 3 are on Firefox rather than IE8.
- The standard deviation is 244 when I include all data, but just 56 when I remove the 3 outliers.
- SpeedTest was remarkably consistent across browsers and OSes.
- Ignoring outliers, the averages per service are:
- SpeedTest = 1334
- Speed.IO = 1243
- AuditMyPc = 1217
Observations on upstream activity
- There are no true outliers but the scatter is much larger than downstream.
- The standard deviation is 264 including all data.
- SpeedTest and Speed.IO were remarkably consistent across browsers and OSes, while AuditMyPc had a wild ride (though faster than SpeedTest or Speed.IO in all cases).
- The averages per service are:
- SpeedTest = 428
- Speed.IO = 468
- AuditMyPc = 930
I have some estimate of my connection speed (1270 Kpbs downstream, 600 Kpbs upstream). The downstream estimate is much more reliable based on standard deviations given above. My question is how does one get truer or more accurate estimates of connection speed? (Note I ask for truer rather than true--I am a pragmatist:-) Or to put it another way, how does one judge which numbers to believe?
The raw data
(2 machines x 2 browsers x 3 services x 2 trials x 2 directions)
Machine Browser Service Direction Speed XP IE8 SpeedTest Downstream 1340 XP IE8 SpeedTest Downstream 1340 Vista IE8 SpeedTest Downstream 1340 XP FireFox SpeedTest Downstream 1330 Vista FireFox SpeedTest Downstream 1330 Vista IE8 SpeedTest Downstream 1330 XP FireFox SpeedTest Downstream 1330 Vista FireFox SpeedTest Downstream 1330 Vista IE8 Speed.IO Downstream 1264 Vista IE8 Speed.IO Downstream 1263 XP IE8 Speed.IO Downstream 1240 XP IE8 Speed.IO Downstream 1235 Vista FireFox Speed.IO Downstream 1231 Vista FireFox Speed.IO Downstream 1226 XP FireFox Speed.IO Downstream 630 XP FireFox Speed.IO Downstream 353 XP FireFox AuditMyPc Downstream 1242 XP FireFox AuditMyPc Downstream 1242 XP IE8 AuditMyPc Downstream 1240 Vista IE8 AuditMyPc Downstream 1208 Vista IE8 AuditMyPc Downstream 1202 Vista FireFox AuditMyPc Downstream 1200 XP IE8 AuditMyPc Downstream 1182 Vista FireFox AuditMyPc Downstream 780 XP FireFox SpeedTest Upstream 430 XP IE8 SpeedTest Upstream 430 Vista FireFox SpeedTest Upstream 430 Vista IE8 SpeedTest Upstream 430 XP FireFox SpeedTest Upstream 430 XP IE8 SpeedTest Upstream 430 Vista IE8 SpeedTest Upstream 430 Vista FireFox SpeedTest Upstream 410 Vista FireFox Speed.IO Upstream 473 Vista FireFox Speed.IO Upstream 472 Vista IE8 Speed.IO Upstream 471 XP FireFox Speed.IO Upstream 468 Vista IE8 Speed.IO Upstream 466 XP FireFox Speed.IO Upstream 465 XP IE8 Speed.IO Upstream 464 XP IE8 Speed.IO Upstream 461 Vista IE8 AuditMyPc Upstream 1191 XP FireFox AuditMyPc Upstream 1165 XP IE8 AuditMyPc Upstream 1074 XP IE8 AuditMyPc Upstream 1022 XP FireFox AuditMyPc Upstream 884 Vista IE8 AuditMyPc Upstream 851 Vista FireFox AuditMyPc Upstream 677 Vista FireFox AuditMyPc Upstream 574