Saturday, March 19, 2011

10 Technologies for 2011 : 10. Data Visualization Tools

Data Visualization Tools

An ever accelerating trend in information technology has been the growth in the volume of data that must be collected and processed within an enterprise. 10 years ago, it would have been rare for a data store to be over a terabyte except perhaps within federal agencies or other instituitions that deal with massive numbers of users or customers such as a Walmart and those that were worked in industries that are by nature data intensive: Oil Exploration, Stock Market analysis etc. Now terabyte data stores are common place and can often  be found in medium sized companies.

With the advent of cheap processing power in modern CPUs and advances in cheap relational database technology such as found in Microsoft SQL Server, PostgreSQL and MySQL at the low end, and DB2, ORACLE, TeraData, Netezza at the high end, suddenly the Terabyte doesnt seem to be a daunting measure of workload any more.

However, the rapid increase in IT's ability to process data has not come with a commensurate increase in our ability to understand this data. The vast majority of intelligence generated from these giant data stores is still distributed to end users through paper based reports. They are produced by scheduled batch jobs that churn through oceans of data and distill their wisdom usually in the middle of the night when there are fewer users to slow the system down. This works well for many kinds of reports.

However there is a new kind of business intelligence platform that is now gathering momentum, upsetting the current equalibrium between the few remaining vendors after waves of consolidation have worked there way through the Industry. MicrosStrategy, SAP Business Objects, Cognos, Oracle, IBM are looking at the new kids on the block: QlikView, TIBCO Spotfire and Tableau as insurgents that bring disruptive technology.

These vendors focus on interpreting data on the fly and presenting various charts and graphs that are updated in real time in response to the users clicks. Some of them like QlikView cache vast amounts of compressed data in memory for very rapid response. These approaches change the fundamental paradigm of business intelligence: Paper Reports dont seem all that neccessary when you can explore your data visually in an instant.

It is very likely that these new approaches to processing data will lead the Business Intelligence majors to either develop extensions to their current stable of products that more directly support Data Visualization or simply decide that its cheaper to acquire these insurgents.

In either case, Data Visualization is here to stay!