Earlier this month, Cisco warned of a critical vulnerability in its Smart Install Client tool that enables attacks on network switches, potentially hundreds of thousands of them. Switches and routers affected by these assaults became inaccessible and inoperable after hackers wiped their configurations, rendering networks completely unavailable. Cisco was by no means caught unware by this vulnerability; in fact, it had created patches to prevent the exploit, and issued an advisory to draw attention to this critical issue. However, most network administrators were slow to implement the Cisco-provided update, leaving a majority of these network devices vulnerable.
What’s going on here is a shakeup to the status quo, where the introduction of technology for managing large-scale networking deployments can also invite large-scale attacks. As the approach to data center infrastructure evolves, with a shift towards white-box deployments and non-proprietary systems, there will be a similar shift in network security thinking and practices.
Until recently, network engineers could largely rely on “security by obscurity,” since the proprietary architecture customizations used by each network device vendor meant that no single vulnerability presented a target that could affect all that many devices. Network vendors would also customize the operating system used on their network appliances to include only those components and services necessary to fulfill the device’s specific purpose, thus minimizing its attack surface. As a result, any hacking attempts large enough to mention were typically aimed at consumer computing devices utilizing more common versions of operating systems, hardware, or application frameworks – not at enterprise networking devices.
However, the continually escalating need for additional network capacity has driven a decisive shift toward network devices that are easier to manage at scale, and that use more mainstream hardware and software components. Case in point: The Cisco Smart Install Client tool exploited in the recent attacks is intended to speed the deployment of Cisco hardware. Common tools can now be used to manage the entirety of large network infrastructure, from networking to compute to storage systems.
Advances including SDN, NFV, and white-box networking are further adding to the convenience with which networking resources can be deployed. Unfortunately, this convenience also extends to hackers, who now recognize targets that are large enough (and approachable enough) to be worth their attention. Thus, we’re beginning to experience attacks on networking, such as the Cisco hack, that feature a scale and severity like we haven’t seen before.
Under the previous status quo, the deliberate pace with which network engineers installed firmware updates to network devices was actually intentional and prudent. Patching a network device such as a Cisco switch isn’t a carefree operation like clicking “update” on a computer or phone. Rather, human error and configuration change are the most common causes of network downtime, the devastating business costs and consequences of which are a network admin’s job to avoid. In the absence of vendor-neutral automated configuration management and provisioning systems, network engineers have traditionally been very careful to minimize potential disruptions by only adopting firmware updates when absolutely necessary.
But network engineers must now adapt to the new reality in the industry. Security advisories must be taken to heart and patches must be deployed rapidly, or major attacks on networking infrastructure will continue to succeed. To do this safely, network engineers will need to adopt new tools and processes, including automated configuration management and provisioning systems that can eliminate human errors, prevent downtime, and roll back changes to automatically recover if or when a disruption does occur.
The shift in networking toward the use of more mainstream components and large-scale deployments is a necessary one, but also something that can really be more opportunity than crisis for those operations teams ready to evolve their practices to mirror these trends and counter the threats. By adapting to this shift and adopting the right tools, engineers can oversee networks that are larger, more manageable, and more secure than ever.
Marcio Saito is Opengear's chief technology officer, where is responsible for the company's product and technology strategy. He previously held executive-level positions in global technology companies. At Cyclades, he was a pioneer in the open source software movement and helped to establish the concept of out-of-band management for data center infrastructure. Later, as the VP of strategy for Avocent, he managed product and engineering teams and led the development of one of the first DCIM solutions in the market. He holds a BSEE degree from University of São Paulo
NetOps teams, like Spider Man, have great powers and great responsibilities. Increasingly, they are adopting a Zero Trust approach to network operations to help fight their battles.
Senior stakeholders who want to hold on to their CISOs must ensure that they have sufficient incentives and, more importantly, support to cope with the burden of risk that they are carrying.
The most alarming takeaway from Cisco’s new Cybersecurity Readiness Index report is that even after decades of having the importance of cybersecurity driven home, cyber threats are still taken too lightly.
