We've talked about raw performance--but though the numbers are useful, they rarely tell the whole story. Benchmarking a single Web server tells you the performance of that server, but that doesn't mean you can add two servers and double performance. There has to be a device that balances traffic between the two--a device that has its own performance limitations. Fail to take that into account and you've pooched the test.
As you develop the goals of your test, consider the traffic scenario you wish to examine. Obviously, you'll model the traffic you have or expect to have as closely as possible, but there are many variations in how you create that model. For example, real traffic is often made up of a mix of UDP and TCP traffic, and a large portion of it is less than 256 bytes in size and composed of TCP-session setup and tear-down, DNS queries, fragments and ICMP traffic. As mentioned earlier, how detailed you need to be in modeling the traffic depends largely on the SUT.
It's common for performance tests to be conducted at large packet sizes because network devices typically perform better. However, in a review of VPN hardware (see "IPsec VPNs: Progress Slow but Steady") where our performance tests were conducted with a mix of traffic ratios, we were surprised to see the Cisco 7140 perform better with a predominance of small packet sizes. When we asked Cisco to explain, it told us it optimized the packet handling for smaller sizes because small packets are more common than large packets. These distinctions don't come out if you test at a single fixed size.
Lost In ...
Loss can be measured in several ways. Remember that TCP has mechanisms to handle loss, such as retransmission and sliding windows. The effect of loss, in the conventional sense of a lost packet, can be hidden by TCP--a fact that's acceptable, because when transactions are tested, some loss is part of normal network operations. An increase in TCP retransmissions, however, is an early indicator that some part of the SUT is becoming overwhelmed. Retransmissions result in a reduction of TCP throughput, because less data is making it end to end, and/or an increase in latency, because each established TCP connection is waiting for data to arrive.
Discover the benefits of scalable backbone and network services for application and cloud providers.
Network automation is gaining momentum. Here's how you can drive this powerful technology to its full potential.
Subsea cable alternatives can provide a level of comfort for those concerned about disruptions. Realistically, they will continue to play a small, but critical role in the short term.
