Although Carlson didn't save any up-front money with IP, Johnson says it's cheaper in the long run because it lets Carlson leverage its in-house IP expertise and infrastructure as well as its network-management tools. Carlson figures it saved about $500,000 steering clear of a Fibre Channel backbone, which would have required hiring and training Fibre Channel experts. Eliminating storage-server and tape-drive equipment in its remote sites saves the company about $100,000 per site, Johnson says. Carlson has thousands of remote sites, but not all sites will require an IP SAN.
The SAN, which operates in Carlson's Minneapolis data center, runs IP as the core, but it uses Fibre Channel between the servers and storage switches and between the storage array and storage switches because that's the most widely available interface for those devices. The 12 Nishan Systems 4300 IP Storage Switches perform the conversion between Fibre Channel and IP. All switching and transport within the SAN and across the WAN are run over IP.
But even with Carlson's heavy IP expertise, the company didn't get the initial configuration of the IP SAN quite right. To save a little money and simplify the installation, Carlson decided to forgo a pair of core Cisco Systems Catalyst 6509 switches that were in the original SAN plan. The SAN ran smoothly without the switches, but it was living on borrowed time: Because the large number of servers running on the IP SAN had to use the lower-level UDP (User Datagram Protocol) transport protocol instead of TCP, there was no way to ensure the servers didn't get flooded. The shortcut could have been disastrous for the SAN, with storage devices becoming overloaded or one server hogging all the bandwidth, for instance.
So Johnson and his team went back and installed the Cisco switches so they could tap into the TCP-based features in Carlson's Nishan storage switches and, eventually, the QoS (Quality of Service) parameters in the Cisco switches. The TCP-based "knobs" on the Nishan switches let Carlson adjust the amount of data on the pipe.
Managing the SAN required a bit of a learning curve, too. Although Carlson is able to use its existing Hewlett-Packard OpenView network-management system to monitor the SAN, there are some limitations to what Carlson can manage on the Nishan storage switches. OpenView can report only whether the switches are up or down, so Nishan is now working with HP to expand the level of detail the management tool can glean from the storage switches.