When Testing, Get Active

I’m sure we have all received that call, “VOIP is choppy or application XXX is slow”.

This simple statement sends the support desk down its typical investigative path of test scripts, procedures, and reporting protocols. This common approach often fails to detect the source of the problem. In many cases, what’s needed is active testing to complement these tried and true methods.

I will use web performance as an example even though any application would apply to the following scenario. A typical test would include pinging the web server or DNS server to check if the host is down, has packet loss, or experiences abnormal response times. In most cases, I would bet your pings will be just fine. The analyst will then report that all is well with the web server.

So what is ping missing or why is this not be the best way to troubleshoot this issue?

I will use Microsoft ping to explain since it is the most commonly used by Microsoft clients. Here are some tests to use to get to the bottom of an application performance problem.

Payload: The default ping payload is 32 Bytes which you should increase to match your application payload. For example, if your web application uses 1,000 Byte payload, you should use the –l (dash lowercase L) 1000 option with your ping command. For example ping www.thetechfirm.com –l 1000.

Generally speaking, smaller packets are more likely to get through or return quicker response times.

In the example above, the first set of smaller pings all got through with an average time of 8ms while the second set of larger packets had an average time of 13ms with 25 percent packet loss.

I try to use a tool that allows me to perform a ping sweep or use various payload sizes, like hrping (https://www.cfos.de/en/ping/ping.htm).

Staying on the theme of payload, fragmentation should be taken into consideration. For example, by default Ping’s ICMP and UDP allows fragmentation where TCP does not. The hrping option to set the do not fragment bit on is –f. Now your pings will fail instead of being fragmented.

Interval: The default ping interval for Microsoft is once ping per second and it cannot be modified. In contrast, your HTTP application will generate hundreds, if not thousands, of packets per second. Sending one packet per second lowers the probability that you will encounter an issue or skew your measurements.  Chances are that one packet every second will get through. Another reason I use hrping is that it allows me to change the ping interval.

Protocols: Ping uses ICMP, where HTTP uses TCP, and DNS is uses UDP. This means different network equipment may block, spoof, or reroute ICMP packets depending on their configuration. There are additional protocols such as Google QUIC over UDP and others that you may not be aware of.

A HTTP, DNS, or SQL server will treat an ICMP ping differently than an actual application command. Cryping (www.cryer.co.uk) has options to use TCP port numbers, HTTP, and other protocols.