Sometimes vendors do get it (mostly) right. Hewlett-Packard put together a brief white paper in February of this year laying out their view of Business Intelligence (BI) for 2009 (and beyond). I think that they got it largely right. Their #7 trend is an increasing corporate focus on rigorous data integration, including data cleansing, Master Data and Metadata Management, all as part of a comprehensive Enterprise Information Management (EIM) Strategy. Below is a summary of the trend, my thoughts on whether HP got it right and what the trend may mean for HIT.
HP Predicts: New pressures are being placed on successful Data Warehousing (DWH) and Business Intelligence (BI) initiatives. Automation of DWH analysis increasingly removes human judgement from report creation and validation; the extension of DWH/BI into the operational realm demands near-real time data, thereby significantly contracting or even completely eliminating the traditional ETL window; recessionary pressures have invigorated cost-control projects, projects which increasingly require cross-functional, and therefore cross-silo, analysis; historically recessions have also spawned mergers & acquisitions (M&A)-driven market consolidations, an attendant sequela is the need to reconcile often wildly divergent datasets; existing DWH and BI applications are being “modernized”, that is new hardware and software architectures and infrastructures which support ever larger data volumes, query complexities and user bases, the net result is the desire, need and ability to combine datamarts, reduce the number of servers, and implement data governance; and the final pressure noted by HP is the steady erosion of domain boundaries between Customer Relationship Management (CRM), Enterprise Resource Planning (ERP), DWH/BI and product lifecycle management. Together, these pressures mandate an enterprise-wide, strategic investment in creating and implementing an EIM. The Verdict: Yes, yes and yes. Before Data Integration (DI), there was Data Governance (DG), before DG there was Master Data Management (MDM), before MDM there was Data Quality (DQ), before DQ there was Data Cleansing (DC) and before DC there was Extract, Transfer and Load (ETL). In the beginning ETL included all of the above (and more) although often not explicitly. Over time DC, DQ, MDM, DG and DI all have split off from ETL, mutating into stand-alone products only to cross-breed and hybridize as various specialty vendors have merged or been acquired. As DI is the most recent branching, it is all the buzz – white papers abound, analyst opinions are legion and many organizations have embarked on or renewed their emphasis on the DI journey. That being said, the DI branch arose in response to the very pressures mentioned in the HP report.