Dear Santa: While I wouldn’t be upset to find a Mercedes SLK350 with a big ribbon on it parked in front of my apartment on Christmas morning, the best present you, and the networking industry, could give me is general availability of 10GBase-T across switches and NIC/CNAs. Once I can just order my 10gig network gear with 10Gbase-T, I can stop worrying about how to pay for optical transceivers at $350 to $1,200 a pop, as well as about the never ending compatibility problems with both optics and twin-ax direct connect cables.
Since 10Gbase-T runs over plan old twisted pair cable, there’s no way for a switch or CNA to interrogate the cable and reject it because it wasn’t blessed. Sure, you have to make sure your cable plant is up to the higher data rate. In general, that means Cat6a unshielded twisted pair or Cat 7 shielded cable, though you can use Cat 6, or even Cat 5E, patch cables for short distances (30 meters or less).
Just as I was feeling that I had found a solution to the 10 Gigabit Ethernet cable conundrum, my friend Greg Ferro blogged that he thought twisted pair cable was a mistake for 10 Gigabit Ethernet. He pointed out four problems he had with Cat 6 for 10 Gigabit Ethernet traffic.
Greg’s first complaint and the biggest real down side to 10GBase-T is power consumption. Even with today’s 40nm PHY chips, driving a 10Gbase-T port will take 2 to 5 watts, where a twin-ax DAC cable or 10Gbase-SR transceiver draws less than a watt. Since I’ve spent much of my career working in data centers on the 34th floor of New York office buildings, I’m sensitive to the power limitations many data centers face.
But before you write off 10Gbase-T as a major cause of global climate change since it uses four to five times as much power, remember that we’re talking about 5 watts in the context of a server that’s probably drawing 300 to 500 watts. Unless you’re designing a system for a submarine, or one of my client’s over-populated Manhattan data centers, 10GBase-T isn’t going to bust your power--or your financial--budget.
Assuming a worst-case situation, where a 10GBase-T link would use 10 watts more than a fiber link, that’s only an additional 7.3 KWh per month. We use about as much power each year to cool our lab as the servers and storage inside the lab use and pay about 18 cents per KWh. At those rates, a 10-watt link will cost us about $30 more a year to run while costing $700 less. Since most 10Gbase-T equipment will run in a roughly 2-watt mode for short cables (up to 30M, which should cover most server-switch connections), we’re talking about a few dollars a year to run 10Gbase-T on a server.
Greg then objects to the size of Cat 6A cables. He has a point if we’re comparing Cat 6A to fiber. After all, the 48-strand fiber cable that would serve a full rack of servers is just 0.6 inches in diameter, where a 48-cable bundle of Cat6A cables could be over 3 inches in diameter. From where I sit, it makes more sense to compare Cat 6A not to fiber but to the Cat 5E that it’s probably going to replace. Most Cat 5E cable is between 0.2 and 0.25 inches in diameter, where the Cat 6A standard allows cable to be up to 0.35 inches. However, vendors including Panduit, which Greg references in his post, and Belden are now making Cat6A cables that are less than 0.3 inches, so that a 48- cable bundle will be about 2.5 inches. Yes, bigger, but considering that we’re taking a typical server from 5-9 Cat 5E cables for 1GBase-T to three--two data and one IPMI management--with cables in the 10 Gigabit era, the total space twisted pair cables will take up will be less than we’ve used in the recent past.
