More on Advanced Erasure Codes: Page 2 of 2

Then there's the decoding overhead. Conventional RAID systems store the original data plus additional parity data. Under normal conditions, the system satisfies read requests by reading the original data from the disks that hold it, ignoring the additional parity data.  The compute overhead in the RAID controller for reads is minimal.  

With advanced erasure codes, the data and forward error correction information is encoded into every data chunk. To recover the data, the system has to retrieve the minimum number of chunks that the coding system it uses requires, and then decode those chunks back to recover the data. So a Cleversafe system has to retrieve 10 chunks, of the 16 it originally stored, and decode them to satisfy a read request.  

The requirement to always retrieve a minimum number of chunks and then decode them substantially increases the compute load on a system using advanced erasure codes. It should also be noted that it increases the overhead on small writes, as the data that isn't being overwritten needs to be decoded, combined with the new data and recoded.

As a result, all the systems we've discussed that use high-level erasure codes also share a scale-out architecture. Having a Xeon processor for every four to 18 disk drives gives these systems the compute horsepower to handle encoding and decoding data using these more sophisticated ECC methods that a conventional midrange array with  four Xeons for 800 or more disk drives wouldn't.