EMC's recent announcement of the culmination of the code-named Project Lightning resulted in the new VFCache solution, a server-based flash cache, which may be used as a complement or alternative to flash storage that appears as if it were a disk drive. This lightning strikes twice, though not in the same spot. The first is dramatically improved I/O performance for customers and the second is the challenge that VFCache brings to competitors trying to distinguish their own flash storage solutions.
In old radio serials, episodes would start with a short recapitulation of "what has taken place so far" so that the listener would have the context to understand the latest episode. Let's apply that to flash storage.
The solid state disk (SSD) market, notably flash storage, has been a gold rush for startups as well as large vendors for some time now. The driver behind the increased use of SSD is what is called the I/O performance gap or bottleneck. As EMC pointed out in a recent analyst briefing, CPU performance improves 100 times each decade while HDD performance has remained flat (as the rotational speed for the fastest drives hasn't changed in years and is not likely to change).
But consider that while CPU performance from 2000 to 2010 increased by 100 times, by 2020 chips will deliver 10,000 times the performance of their 2000 counterparts. A storage device's inability to process CPU-generated I/Os fast enough (i.e. the I/O bottleneck) can be a significant problem in many cases today, but obviously is on its way to becoming more or less universally intolerable.
Ta da! (Sound the trumpets). Enter flash memory stage right with the potential of improving storage I/O performance by at least two orders of magnitude. How so? In large part because flash has none of the mechanical parts that inherently limit HDD performance. Is it any wonder that there is an SSD vendor gold rush on?
EMC was the first enterprise vendor to introduce flash in enterprise storage arrays in 2008 with SSDs that appeared to the OS and application on the server and the controller on the storage array as if they were simply disk drives. What was missing at that point was that enterprises needed the ability to use flash as a tier of storage (tier 0) where only the data that was most active (i.e., hottest) would be kept in flash, and less active data would be kept in another tier (such as FC/SAS tier 1 storage hard disks).
In 2009, EMC introduced software to do just this: FAST (First Automated Storage Tiering). This enables more effective use of the SSD tier and other tiers of storage not only from a performance perspective, but also from an economic perspective (as the relatively more expensive SSD storage only holds performance-sensitive data, which is typically a small subset of all data stored).
