Let's take a minute to discuss key storage. Never store your server's private keys on disk on the same server that uses those keys to encrypt your data. Instead, use smartcards or an HSM (hardware security module) for key storage and management. To steal the keys, these solutions require physical access to the card reader. They provide better security because access to the keys is managed via the smartcards. Rainbow Technologies, nCipher Corp. and Ingrian sell products for securely managing your keys. Don't simply use a removable storage device for these purposes because most operating systems view these devices as mounted file systems or drives, so they are remotely accessible.
Load-Balancers and Connection Control
Just as you must ensure that your firewall is configured correctly and lets only traffic on specific ports pass into your infrastructure, it's equally important to configure back-end devices, where possible, to accept connections only from specified clients.
Controlling who and what can connect to your servers provides for tighter access control and, thus, better security. If you have a load-balancer that can change the source IP address (one that can fully terminate TCP/SSL), your back-end Web servers sitting behind that load-balancer should not accept Web requests from a device other than the load-balancer. In other words, use the TCP-based access control available or create a "firewall sandwich" that allows connections only from specified, trusted sources. Again, start with a deny-all policy and open the machine only as far as necessary to let it perform its tasks.
This is particularly important for your databases because that's where you keep the crown jewels. If you aren't encrypting the data (and you should be), you need to take particular care that access is allowed only from those systems that need it.