With a vast ecosystem of thousands of product suppliers committed to compatibility and the IEEE 802 family of networking standards undergoing constant, market-driven innovation, Ethernet has been taken places its inventors never could have imagined 40 years ago. Certainly, its role in the datacenter continues to expand rapidly.
What must enterprises understand about the technology's development with regard to datacenter connectivity, in order to ensure that the upgrades they initiate today position their infrastructures to take full advantage of Ethernet's evolution into the future?
Converging the datacenter
Every day, Ethernet shoulders a wider array of sophisticated datacenter services. A technology initially developed as a way to link printers and other devices seamlessly across a local area network (LAN), Ethernet now supports increasingly demanding applications within and among the world's most advanced datacenters. For example, Internet Small Computer System Interface (iSCSI) and Fibre Channel over Ethernet (FCoE) utilize the higher available bandwidths of Ethernet to consolidate storage traffic across the datacenter on to a unified transport. This is valuable because it eliminates the need to maintain separate block and file infrastructures.
In addition, Network File System and Server Message Block, which have been in place for decades, leverage the ubiquitous Ethernet transport to provide remote file access. In high-performance computing applications, Ethernet services such as Remote Direct Memory Access over Converged Ethernet (RoCE) enable a cost-effective reduction in cabling and administrative needs.
Datacenters have historically deployed various link-layer technologies to run a range of application-specific networks. Fibre Channel was deployed to support storage services, InfiniBand for high-performance computing, and Ethernet for basic LAN applications. Today, however, Ethernet is emerging as the unifying fabric for the full gamut of such applications.
Datacenter bridging (DCB) has provided a breakthrough in this area. The IEEE 802.1 Working Group's Datacenter Bridging Task Group works to satisfy the requirements of protocols and applications in the datacenter. A host of standards is intended to enable a DCB capability that eliminates packet loss due to queue overflows for selected mission-critical applications and assigns quality of service to help ensure that such high-priority traffic is allotted the bandwidth that it requires. The standards include:
- IEEE 802.1Qbb "Priority Flow Control"
- IEEE 802.1Qaz "Enhanced Transmission Selection and Datacenter Bridging Exchange Protocol"
- IEEE 802.1Qau "Congestion Notification"
In this way, DCB is designed to enable lossless Ethernet networks that can cost-effectively collapse FCoE, iSCSI, and RoCE protocols on to the same physical Ethernet infrastructure. Datacenters have long sought convergence and its cost and simplicity benefits. And now the ability to converge more and more datacenter traffic on Ethernet is one of the primary drivers for ongoing innovation of the technology.
Center of the bandwidth storm
Today, we are seeing strong growth in the use of Ethernet to accommodate the exploding bandwidth needs that datacenters are under pressure to meet.
After years of 1 Gbit/s server interconnections as the mainstay in datacenters, conversion to 10G server interconnections and evolution to 40G and 100G Ethernet infrastructures are coming soon. Furthermore, the Ethernet ecosystem is already making longer-term plans for 400G Ethernet links among datacenters.
No less than a bandwidth tsunami has been unleashed with the recent proliferation of innovations such as smartphones, tablets, WiFi, 3G/4G/LTE mobile deployments, 10G servers, Internet-enabled TV, the cloud, social media, video calling, online gaming, and new database technologies. The datacenter, of course, is the epicenter of the storm.
In 2010, an amendment was made to extend the IEEE 802.3 standard for Ethernet to support operating speeds of both 40G and 100G. Thus 802.3ba became the first standard to define two new Ethernet speeds. This paved the way for a new generation of high-rate server connectivity and core switching while maintaining maximum compatibility with the installed base of interfaces.
Innovation continues with datacenters seeking higher-density, lower-cost, and lower-power solutions. IEEE P802.3bm, the draft standard for Ethernet for 40G and 100G over fiber-optic cables, is being written to define:
- 100G physical layer specifications and management parameters, using a four-lane electrical interface for operation on multimode and single-mode fiber optic cables, reducing fibers per link, cost, and power and increasing the density of 100G applications in the datacenter
- Optional Energy Efficient Ethernet, a technology that lowers power consumption for systems based on utilization, for 40G and 100G operation over fiber optic cables
- 40G physical layer specifications and management parameters for operation on single-mode fiber optic cables of extended reach, which would support adoption for metropolitan applications, simplify networks, and entail a low risk of change as datacenters grow beyond their conventional brick-and-mortar structures into regional facilities
Other standards in development include a draft to expand IEEE 802.3 Ethernet to 100G operation on backplanes and copper cables and to specify optional EEE for 40G and 100G.
In April 2013, the IEEE launched a study group to explore the development of a 400G Ethernet standard to support the exponential network bandwidth growth. Again, it's a market-driven effort designed to improve the 802.3 Ethernet standard in order to stay ahead of the industry's needs. Two years of open efforts to assess the Ethernet market's emerging application requirements showed that 400G Ethernet strikes the correct balance among cost, power, density, and bandwidth demand considerations as the next speed step for IEEE-defined Ethernet applications.
The history of Ethernet has shown that standards development, technology innovation, and market growth are closely linked in a constantly spinning cycle. Understanding where the technology is headed will help datacenters make better use of Ethernet for datacenter connectivity today while keeping them well positioned for the future.
Patrick Strick is the technical chair of the Data Center Subcommittee of the Ethernet Alliance and a technical marketing engineer in NetApp's Data Center Platforms group.
