Stateless Security In The Data Center: A Hybrid Approach

Recently, I was talking to a customer about their new data center architecture. The conversation naturally drifted toward the types of security controls they required and what types of devices would fulfill that need.

The discussion took a familiar turn, as they shared their requirements for stateful inspection capability inside the proposed architecture. The customer listed the standard security devices: firewalls, intrusion prevention systems, and other inline inspection tools common in the industry. The security teams wanted visibility and access control, and there was an implicit assumption that stateful security devices sprinkled liberally at different layers in the data center would provide a comprehensive solution.

In years past I would have agreed wholeheartedly with this assertion.

Lately, however, my opinion has begun to change regarding this idea. I believe that in today's data center environment that stateless access controls should not only be considered a complementary option, but in some cases may actually provide a better solution than their stateful counterpart.

Let me explain.

Around 20 years ago, access to the World Wide Web was controlled by stateless devices called packet filters. They were stateless in that they treated each packet individually with no regard to the relationship with other packets. These devices provided access control and were commonly found at network boundaries. A common example of this stateless control is an access list in a router or a switch.

A major limitation to stateless packet filters is that they lack the ability to guard against spoofed addresses, fragmentation exploits, and other types of potentially malicious IP communication. To counter this weakness, the packet filters evolved over time into state-checking devices that could monitor the TCP three-way handshake and make access decisions based upon the state of existing flows.

A stateful inspection device actually builds a state table in memory to manage these flows and allow them to be inspected. This is the basis for modern stateful inspection security controls today, and is generally accepted as far superior to its stateless counterpart for enforcing security policy.

So where's the issue?

For all of their strengths, stateful inspection devices have some weaknesses when deployed in a typical data center environment. Let's examine three of the more common ones.

Where to deploy stateful security devices in the data center?
Only in recent years has adding security controls and visibility in the data center become mainstream. The concern was that stateful security devices weren't as robust as the data center devices and technologies they would support, and any security outage would lead to a data center outage.

Fortunately we've evolved our thinking, and today security services in the data center are a must -- especially when considered in the context of compliance.

The question of where to deploy stateful security devices in the data center becomes paramount. It was much easier connecting to the Internet when the boundaries between trusted and untrusted were obvious. A data center tends to have more complex traffic flows, given that the points of intersection are less defined. Does one deploy hypervisor-based solutions, physical security appliances, or a combination of both? Each has caveats and limited visibility, depending on where they are positioned.

The rise of asymmetric flows
Asymmetric flows occur when there are multiple network paths for ingress and egress. A decade ago it was considered poor network design to have any asymmetry in the network, but now it's very uncommon to find a network that doesn't have a significant number of these flows.

A simple example of asymmetry is where a company has two service providers, A and B. Flows leave the company via Provider A and return via Provider B. The Internet is built on this type of interaction, as clients and servers couldn't care less what path is taken to get from here to there.

Asymmetric flows are pervasive in data centers today. Much of this is intentional so workloads can take advantage of secondary paths that might be underutilized.

The major problem is that stateful inspection devices do not like asymmetric flows and typically will drop the "broken" flows in the interest of security. This has become such a problem for the security industry that most vendors now have options to disable state-checking in order to be deployable in an asymmetric environment.

When stateful inspection becomes a target
It's never good when one's stateful security device becomes the target itself. This is nothing new; devices on the Internet have long been exploited via denial-of-service (DoS) attacks. 

Data centers frequently have large bandwidth demands. 10 Gigabit Ethernet is the standard today and will soon be usurped by 100 GbE. The sheer volume of traffic and throughput in this landscape means that stateful inspection devices are being pushed to their limits. I've seen pwned servers with 10 GbE fabric connections devastate stateful devices by filling up their state tables with garbage data. I've seen poorly written applications and dying NIC cards produce similar negative effects.

The hybrid approach
The issues I've outlined with stateful security devices in the data center are the main reason I began to seriously consider a hybrid approach combining stateful and stateless access control. Stateless controls are easier to insert and don't suffer the same challenges with asymmetric traffic. They also aren't susceptible to DoS attacks, because they hold no state information.

Stateful security devices should be deployed in areas where visibility and defensive capability are required. Stateless controls, on the other hand, can be deployed in the data center periphery. Think of them as complementary in that the wider, more easily deployed stateless controls are pre-filtering flows before they reach the deeper, more powerful stateful devices.

Stateful security controls aren't going away anytime soon and are still a necessary part of a comprehensive data center security framework. Stateless security controls provide an easier means for making security services more pervasive in the data center without the costs and challenges of their stateful inspection cousins. Considering the advantages and disadvantages of both types has led me to believe that a hybrid approach might be best.