Companies like Google, Microsoft, Arista, and Mellanox are pushing the development of a 25 Gigabit Ethernet standard for top-of-tack server networking. Some may question the need for this technology, but they will soon see the benefits.
As Intel x86 CPUs improve in I/O and speed, 10 Gigabit Ethernet is no longer fast enough. For server to top-of-rack network connections to keep up, you would need to double the number of switches in each rack and use 10 GbE NICs. The hidden costs make this impractical.
Why 25 Gigabit Ethernet?
The next generation of Intel "Grantley" CPUs are due to ship in the third quarter, and servers with these processors will be able to forward data faster than 10 Gbit/s. In the high-density server farms of cloud providers like Google and Microsoft, using multiple 10 GbE would require twice as many Ethernet switches with their associated space, power, and cooling costs.
The 100 GbE Ethernet standard consists of four lanes of 25 Gbit/s Ethernet on four fiber or copper pairs. Small form-factor and C form-factor pluggable transceiver (SFP/CFP) modules have four lasers, each transmitting at 25 Gbit/s. Each lane requires a serializer/deserializer (SerDes) chipset.
The proposed 25 GbE standard will use the same physical silicon from a single 25 Gbit/s lane. This simplifies the process with just minor changes for forward error correction and lane alignment, and it reduces the cost when compared to 40 GbE.
"We expect, in the first generation, 2.5 times the performance at 25 gigabit for the 1.5 times the price of 10G," Anshul Sadana, senior vice president of customer engineering at Arista Networks, told me. "And in the second generation, we expect pricing parity for 2.5 times the performance."
Getting 25 Gbit/s performance for the same price of 10 -- combined with reduced operating costs -- makes this a compelling proposal.
IEEE, consensus, and snails' pace
The 25 Gigabit Ethernet Consortium has formed to accelerate IEEE adoption of standards. The IEEE has a notoriously slow process for voting and consensus building. This has led to splinter organizations like the Ethernet Alliance and the Metro Ethernet Forum, groups that develop standards and technology at a more practical and useful pace before passing them to the IEEE for approval.
This often presents a confusing picture of standards development and undermines the market belief in the IEEE as a meaningful organization. But if influencing the IEEE 802.3 board is the only way to get things done, then clearly the institution is not working in the best interests of the market.
Consortium value
The inclusion of Microsoft and Google in the 25 GbE group is significant, because they are major customers. At the end of 2013, the research firm Dell'Oro estimated last year that 10% of server sales were white-box servers. Other reports have suggested the number could be as high as 30%. Arista is known to supply both Microsoft and Google with its technology, including Mellanox network adapters.
Brocade most recently joined the consortium's ranks. And 10 additional vendors are reportedly in the final stages of signing up.
Physical connectors and cables
The standard does not specify physical cabling, but Sadana confirmed that the manufacturers will decide what to support. Practitioners expect twin-axial cable to be preferred initially over fiber for its lower power and cost.
Right move at the right time
During the IEEE discussions in the mid-2000s, there was much debate about 25 GbE versus 40 GbE. At the time, the committee chose to focus on delivering standards for backbone connections between switches for long-haul wide-area networks. In 2014, the rise of cloud providers has changed the landscape for data center Ethernet. There's now a large market and increased demand for high-speed connectivity in the rack at a moderate cost.
The simplicity and use of the related components have the secondary benefit of increasing growth in 100 GbE. Increased production of VCSELs and other photonic components will drive down cost, allowing for greater adoption. Forward-thinking organizations like Google are sure to be considering this.
It's difficult to predict when 25 GbE will arrive, but it could happen in late 2015. However, don't expect to use 25 GbE in your enterprise data centers until 2016, when mainstream adoption is expected.
Bob Friday, Chief AI Officer for Juniper Networks, explains how the advanced technology is transforming operations.
Contact center leaders from 8x8, Awaken Intelligence, and 360insight discuss the importance of agent experience.
AI may force enterprises to rewire parts of their data centers so they are fully optimized to run such workloads. The question is do you use Ethernet or InfiniBand?
