"If you ask my boss what information he'd like to keep and what information he'd remove, he'll tell you none, " says Ratzesberger. "We simply don't know what queries we'll need to run in the future. As such if we don't store the data today it'll takes us 13 months to build up a history to answer that sort of question."
For storage and IT professionals the inclination to store greater amounts of data and retrieve more of that data suggests a growth in the primary data tier as well as a growth in the secondary data tier. SSDs with their faster retrieval times will be needed to provide for access to the very hottest of data with disk providing a "fast" secondary layer for perhaps less frequently, but still very frequently accessed data.
IT will also need to think about it backup and redundancy plans. It's one thing to duplicate or backup and restore a 100 gigabytes of data. It's a very different matter to do so on a terabyte or a Petabyte of information. Retrieval times are necessarily longer; the sheer cost of storing the data so much higher.
And while today the largest organizations may be using data warehousing look for those capabilities to move downstream to the SMEs. From a business standpoint, the benefit to an Ebay is the same to any e-commerce organization so there will be a natural imperative to try and claim as much data as possible. The means will also be become more viable. In discussion with Todd Sylvester, Teradata's director of R&D strategy, it's clear that the company isn't looking downstream to bring its software to the Fortune 10,000. This will likely take the form of a data warehouse service where service providers can deliver data warehousing the cloud. Organizations then will be freed from the challenge of having to build out the necessary storage infrastructure locally.
The introduction won't come soon. Discussions are underway right now with Teramark, the service provider, he notes. Then in 2011 there are a number of structural enhancements that need to happen to the Teradata database for it to support multi-tenancy. But in 2012 look for Teradata to make its entry and bring the power of big data to small companies.
