Virtualization causes significant changes in network traffic, forcing IT to reconsider the traditional three-tier network approach. Tiered networks were designed to route traffic flows from the edge of the network through the core and back, which introduces choke points and delays while providing only rudimentary redundancy.
Enter the flat network. Also called a fabric, this approach allows for more paths through the network and is better suited to the data center's requirements, including the need to support virtualized networking and virtual machine mobility. A flat network aims to minimize delay and maximize available bandwidth while providing the multitude of network paths demanded in a virtual world.
But a flat network also requires some trade-offs, including the need to rearchitect your data center LAN and adopt either new standards such as TRILL (Transparent Interconnection of Lots of Links) and SPB (Shortest Path Bridging) or proprietary, vendor-specific approaches. We'll look at how a flat network differs from a traditional tiered infrastructure and examine potential shortcomings. (For a look at flat network security, see "How To Secure Your Flat Network".)
How We Got Here
Ethernet won the battle for the LAN more than a decade ago, but it still suffers significant limitations. One prominent problem is the forwarding mechanism. When an Ethernet switch doesn't have a MAC address and interface pair in its forwarding table, or if it receives a broadcast Ethernet frame, the switch makes a copy of the frame and forwards the copy to all interfaces. Because Ethernet has no Time To Live (TTL) header field to keep a frame from being forwarded indefinitely, if there's a physical loop in the network these frames will be copied and propagated repeatedly throughout the network until it crashes.
Radia Perlman, a renowned network engineer and an Intel Fellow at Intel Labs, created the Spanning Tree algorithm, which became part of the Spanning Tree Protocol (STP), to solve this issue (among others). And it works. In at least 40% of the networks I see, Spanning Tree has never been changed from its default settings, but it keeps the network up, while providing some redundancy.
However, while STP solves significant problems, it also forces a network design that isn't optimized for many of today's data center requirements. For instance, STP paths are determined in a north-south tree, which forces traffic to flow from a top-of-rack switch out to a distribution switch and then back in again to another top-of-rack switch. By contrast, an east-west path directly between the two top-of-rack switches would be more efficient, but STP doesn't allow that path.