Internet Performance Testing: Speed Vs. Quality

When measuring internet performance, there are two basic metrics, speed and quality. Speed is pretty straightforward: How much throughput can I get between two devices? Throughput is typically measured using TCP and hopefully with very little disk interaction, since your hard drive may be the highest source of latency. Quality measures packet loss, roundtrip time, and jitter which impacts latency sensitive applications.

Applications that transfer large quantities of data would be affected by throughput. Some examples would be downloading a file with your web browser (HTTP, HTTPS), file transfer client (FTP, SFTP), Email (POP, SMTP, IMAP), and File and Print Sharing (SMB). There are UDP applications that can be used for file transfers like TFTP and QUIC, but they're less common.

Throughput is fairly easy to test: Transfer a big file and measure the utilization, bytes transferred, or transfer time and compare against your expected values. For example, you might use FTP to download a 1 GB  file and find that it took five minutes. Some things to be aware of when transferring data when testing:

• As mentioned earlier, try to avoid the disk for transfers when possible. For example, iperf runs in memory by default. The disk impact on performance is more of a consideration when performing local tests.
• When transferring files, perform separate upload and download tests.
• Some network devices treat files types differently when compression devices are used. Use different file types in your tests if you’re not sure.
• Different protocols and applications will have different overhead/priority resulting in varying throughput or times. Since various protocols might be shaped or routed differently, test with multiple applications.
• Run more than one test. I typically perform five tests, discard the best and worst results, and average the remaining three to get a good sample.

Now on to quality testing. Some applications that are sensitive to latency include video and audio based applications. The protocols can’t tolerate huge variances in jitter and packet loss. Others examples would be VPN, Citrix, remote desktop, QUIC, and some security/authentication protocols. Most of these latency-sensitive applications are UDP-based and noticeably affected by packet loss or jitter. Unfortunately, UDP does not have the ability to detect or recover from packet loss and must rely on the application layer. This protocol behavior is referred to as, “Connection (TCP) and Connectionless (UDP).” Packet loss and jitter cannot be measured using the same methodology as throughput.

This is where your tools come in. When using iperf, the –u option  measures packet loss and jitter values. Keep in mind that ping measures round trip time, which is not the same as jitter. Unfortunately, iperf sites on the internet are few and far between; moreover, your firewall might block that port number. The majority of us will go to an internet speed test site to make life easier.

Before using Speedtest.net or any internet performance testing site, take these considerations into account:

• Are you trying to test throughput or quality? Or are you trying to test TCP or UDP?
• How long do you want to test for?
• Does time and date matter? Consider business vs. after-hours, quarter end vs, tax season, and backups running
• Is web traffic shaped?
• When dealing with any internet site, there are many variables that will affect your results:
  
  • Approximately how far away is the server?
  • Is the client on WiFi?
  • How many hops are between you and the server?
  • Do you know if there are "busier" times for the server or network connection?

A good rule of thumb is to pay attention to consistency. For example if you performed three speed tests and the results were 22 Mbps, 20 Mbps and 21 Mbps, consider that test pretty consistent, or good. You might notice that the further away the server is, the less consistent your results are.

Specifically with Speedtest.net, pay attention to the selected server and see if other choices are available. In this screenshot, you can see that three servers were available for testing.

I ran throughput tests to each server and got different ping and throughput results.

In this case, I would retry with just one server to see if the numbers stay more consistent. I re-tested the last CenturyLink server and got pretty consistent numbers; 20, 22, 24, 23 Mbps for download speeds and 11, 10, 13, 10 Mbps upload results. Note that I also moved the computer to a wired connection to eliminate any possible WiFi issues.