In the first scenario--correlating concrete numbers, such as IT budgets and infrastructure spending, with interest in Internet/ intranet-related products--we sought a relationship that could show whether people interested in these products came from a company with particular spending patterns.
Indeed, each product generated a report that comprised these variables and concluded that readers from companies with huge IT budgets that spend copious amounts of money on infrastructure are not particularly interested in Internet/intranet products. But those of you who work for technology companies with comparatively small IT budgets and very little spending on infrastructure were extremely interested in these products. We haven't told our marketing people this, though, so if they appear to target you, it's not our fault. Really.
Our second scenario was designed to examine the tools' ability to analyze more nebulous concepts. In this case, we tried to determine whether there is any relationship between a subscriber's job title and the size of his or her company's IT budget, ostensibly to help automate and streamline the subscription consideration process.
Every product pulled together the relevant information and came up with the same answer: No such relationship exists. If our IT staff had been considering such an option at the behest of our marketing department, we could have provided hard data that showed that such an undertaking would not be cost-beneficial. In other words, the result of our analysis was cost-avoidance--something that can benefit every company.
The cliché about the picture and the thousand words holds true for the tools we tested: A business- intelligence solution's means of presenting and distributing information can make or break the product. Most companies have so much data to consider that they often overlook relationships and trends. We spent most of our time evaluating the presentation facet of the products.
We evaluated the reports' formats, as well as how each product delivers those reports. We considered not only the graphical means by which the products present the results of queries--which were impressive for the most part--but also the breadth of formats in which the data could be delivered.