Fibre Channel Backbone 6 Boosts Storage Network Performance

For the past two years, the T11 standards bodies have been working diligently to codify the Fibre Channel Backbone 6 (FC-BB-6) standard. This detailed, challenging, and time-consuming task has been led by some of the brightest technical minds in the storage and networking industries. Their hard work is now providing results that will prove to be fruitful and significant.

Let's take a step back and analyze the objectives of FC-BB-6 at the start of standards development. Its predecessor, FC-BB-5, was ratified in October 2009, and has been widely adopted by the leading switch and adapter vendors. Although FC-BB-5 was ground breaking in many aspects, ongoing development is always required to lead innovations in data center switching technology.

For instance, a principal development in FC-BB-5 was the concept of the Fibre Channel Forwarder (FCF). In a nutshell, the FCF is the heart of a traditional Fibre Channel over Ethernet (FCoE) infrastructure, because it serves as a domain ID of the fabric, encapsulating and decapsulating Fibre Channel from Ethernet frames. While this works well in replicating Fibre Channel topologies and creating hybrid FCoE/FC architectures, new initiatives in the industry were making it clear that even more flexible options were required.

So, the next iteration, FC-BB-6, set out to improve upon these capabilities. The key objectives were:

• To support point-to-point and multipoint storage network designs
• To eliminate unnecessary bottlenecks in the storage network design and provide high performance solutions
• To reduce the administrative burden by reducing the number of domain IDs used in a SAN design, thereby addressing domain ID scalability issues with traditional Fibre Channel SANs.

One key concept FC-BB-6 defines is the distributed FCF, which, in layman terms, is a distributed switch architecture. Essentially, this new switch architecture separates the control and data plane in a switching fabric. Below I explain what the separate planes do:

Control plane: Fibre Channel Forwarders (FCF)

• Controlling FCFs and FCoE Data Forwarders share one or more single virtual domains. A virtual domain simply provides similar functionality to a traditional FC domain (physical switch), albeit with functionality distributed in multiple physical switches
• Controlling FCFs have a principal domain, wherein the FC-BB-6 FCF acts just like a FC-BB-5 FCF connected to traditional FC fabrics
• FCFs provide Fibre Channel services
• BB-5 FCFs could be upgraded to BB-6 with a firmware update (providing investment protection).

Data plane: Fibre Channel Data Forwarders (FDF)

• FDFs obtain routing and zoning information from their controlling FCFs
• No changes at the end nodes (ENodes) are required. ENodes operate the same as FC-BB-5, ensuring investment protection at the end nodes -- unless you want to deploy point-to-point and multipoint storage network designs (such as VN2VN; see below for more detail).
• Routing at the FC layer rather than the MAC layer
• Single FC domain shared by controlling FCF and FDF

The FC-BB-6 standard has been ratified by the T11.3 technical committee and has now been forwarded to the InterNational Committee for Information Technology Standards for approval. It has achieved the goals and objectives that were set by the committee. These include:

VN2VN node support: VN2VN functionality establishes point-to-point and multipoint connectivity between two nodes in a FCoE network -- for example, a converged network adapter (CNA) and a storage array host port supporting FCoE. This eliminates the need for a FCF (a controlling switch) in a smaller network when you need two nodes to talk to each other. The simplest connection is one-to-one between two nodes, but this can be extended to one-to-many relationships and VN2VN connections between multiple nodes. There are no zoning requirements in the network designs, resulting in lower complexity and costs.

Separation of control and data plane: Virtual links have been established between FCFs and FDFs to allow separation of the control and data planes in a switching fabric. The very definition of the FCF states that it is a Fibre Channel switching element that can forward FC and FCoE frames across one or more switch ports. The core problem today with Fibre Channel and Fibre Channel over Ethernet is that all frames in an FC and FCoE SAN have to pass through the controlling FCF, and can become a performance bottleneck, creating management issues and increasing cost.

FC-BB-6 addresses this issue by detailing how virtual links are established between FCFs and FDFs. The FC-SW-6 standard currently under development defines optimal routing mechanisms between switches, so that local switching can be done by the FDFs without going through an FCF. For example, two nodes connected to a FDF would be able to talk to each other, directly bypassing the FCFs. This capability should simplify the architecture and accelerate the movement of data via more optimal traffic flows.

Domain ID scalability: In traditional Fibre Channel switches (including current FC-BB-5-based FCoE switches), a SAN cannot architecturally have more than 239 switches (because each switch consumes a single domain ID). While in traditional SANs there are some good workarounds to overcome this limitation, in a converged SAN, this could present problems. With FC-BB-6, FCoE fabrics can use domain IDs more efficiently, which enables larger converged network SANs.

FC-BB-6 fully defines how to support the deployment of the distributed switch architecture (currently in the final stages of development in the T11 FC-SW-6 working group) over Ethernet. See the diagram for a functional model of a BB6 virtual domain. In this new style of SAN, the controlling FCF is the brain of the distributed switch, while the FDF simply does what it is instructed to do by the FCF. The FCFs are redundant in the SAN (primary, secondary and alternate) in order to retain no single point of failure in the FC SAN.

The table below shows the difference between switch types in BB-5 versus BB-6.

­­­Key points to note:

1. In FC-BB-6, there is a virtual domain that consists of multiple switches, while in BB-5, each FCF consumes a domain ID. This new switching standard helps with scaling of FC and FCoE storage area networks.
2. FDFs do not have to implement all the fabric services and do not consume a domain ID. For example, there is no name server implementation in FC-BB-6 FDFs.
3. Path selection inside the virtual domain is enforced by the controlling FCF. The FDFs simply follows Layer 3 routing mechanisms.
4. In this new architecture, FCFs no longer become bottlenecks and hot spots for traffic flow between end nodes.

As you can see, FC-BB-6 and FC-SW-6 incorporate significant architectural changes from the previous generation of the standard. These enhancements are a natural and necessary progression in the evolution of converged networks. From a customer viewpoint, FC-BB-6-based converged storage area networks will result in more efficient network designs for converged Fibre Channel and Ethernet networks, thereby resulting in higher performance and lower-cost networks.

Author's note:
The author wishes to thank Babu Puttagunta and Siamack Ayendeh from Hewlett-Packard for reviewing this document and providing some very valuable technical feedback and editorial commentary.