We didn't realize that by doing the cutover at night, when traffic was light, we weren't immediately experiencing the new system's full impact on network performance. And we hadn't expected the system to add significant additional traffic. But the next morning, when employees started to log in and fire up their client- server apps, we had a problem. Our network performance took a nosedive, traffic spiked, and user productivity came to a near standstill. Users who arrived later could not log in at all.
Working with the authentication vendor's tech support, we resolved the problem--or so we thought. But every few days, the traffic spiked again and we had to pull the vendor back in. Dirk wanted to pull the plug on the technology, but Bucky was pigheaded about it. I gave them a few more days to work things out--but after that, we would have to shut it down.
On a subsequent Monday morning, the network crew spotted traffic spikes again, so Dirk pressed me to let him take down the authentication server application, and I gave him the go-ahead. When Bucky arrived, he was piping hot and spouting threats of internal audit rule noncompliance. I told Bucky we could live without this for now. Somehow, we had found ourselves solving a problem that didn't exist yet. Sure, token-based authentication would improve our security, but at such a high cost to our network performance, it simply wasn't worth it.
Problem Re-Solved?
Projects like this are not good for IT's image. We work hard to maintain good service levels, and when a supposed "improvement" hurts productivity, it damages our reputation, too. Bucky and Dirk are working with the security vendor to solve the problem--we may end up returning the hardware and going with another vendor.