First, you break it. Stage a mock outage in your lab before deployment (or, if that's not possible, on your production network during off hours) and observe how your backup equipment responds. Note how long the network stays down, and which parts of the network are affected. You can accomplish this by pinging continuously from one side of the network to the other. Also, unplug cards and power supplies to see if your network management software gets the SNMP trap messages. If you have full spare chassis, you can monitor them as well.
Document your findings, reviewing step by step what worked and what didn't, and develop approaches to improve the process. Then, if time allows, run through the drill again, and be sure you're satisfied before putting the product on your live network and filing the configuration information somewhere safe and accessible. (The outage at my friend's organization would have lasted much longer had there been no backup of the original software configuration stored on the network--the switch's complex array of VLANs would have been torture to reconfigure from scratch.)
Stage yet another planned outage during off hours shortly after the equipment is up and running on your live network. This will give you the opportunity to test your restore procedures in a real-world setting. It'll also give you one last chance to be sure all the spare gear you need is readily available.
Additionally, this is a good time to check for security access. If you want to be able to load software or configurations quickly from a TFTP server, for instance, be sure you have access across firewalls and router ACLs (access-control lists). Also be sure there aren't any filters on the servers that may stand in your way.
But the fun's not over yet. All too often, network problems stem from configuration changes made on the fly. Once you take the product live, you must monitor the network closely and chart every change you make, so you can retrace your steps if necessary. You don't want to be testing your memory when time is of the essence.
Discover the benefits of scalable backbone and network services for application and cloud providers.
Network automation is gaining momentum. Here's how you can drive this powerful technology to its full potential.
Subsea cable alternatives can provide a level of comfort for those concerned about disruptions. Realistically, they will continue to play a small, but critical role in the short term.
