In the last few years, enterprises have added several tools to their IT data integration toolbox to improve the return on their CRM investments. Yet, most of these companies have not achieved a simple goal: to create reliable, unified views of their customers – aggregated across data silos – and deliver these to all customer-facing applications in a timely fashion. Recently, companies have turned to three common technologies to create solutions for customer data integration. These are data movement tools such as Extract-Transform-Load (ETL), data query and aggregation tools such as Enterprise Information Integration (EII) and Data Quality (DQ) tools. However, what the tool vendors aren’t telling you is that these tools are woefully inadequate for developing a reliable Customer Data Integration (CDI) platform.

Customer Hubs Emerging

Industry Market Research firm Gartner Inc. defines CDI as "the combination of the technology, processes and services needed to create and maintain an accurate, timely and complete view of the customer across multiple channels, business lines and, enterprises, where there are multiple sources of customer data in multiple application systems and databases”. There are several implementation styles of CDI solutions but the most effective is where an enterprise commits to building and managing a customer hub that serves as a central repository of customer data reconciled from multiple data sources. This hub may contain some or all of the critical customer data needed to provide multiple customer views to downstream applications. While there are significant differences among the various customer hubs available, such as what type of data to persist and how much to aggregate dynamically, there is little doubt that a large, enterprise-class CDI solution needs a central customer hub.

Data Tools Ill-suited

In the past decade, many companies that tried to build an in-house version of a customer data integration hub using ETL, EII and DQ tools are now struggling with the aftermath of a custom solution. There are several reasons for the failure of CDI solutions built with these tools.

First, all three technologies originated for narrow purposes ill-suited for CDI: ETL to move large volumes of data in batch-mode; EII to run distributed queries across disparate sources in real-time; and DQ tools to “scrub” incorrect names and addresses in a single source at a time. Each of these technologies effectively supports only a single data-modality; batch or real-time. Since customer data is inextricably tied to both operational and strategic business processes of a company, such as order-to-cash process or profitability segmentation analyses, it needs to be delivered in time for each business process. Therefore, any customer data integration solution needs to support a range of modalities of data movement: from a large-volume batch process that loads a new source into a customer hub; to scheduled intra-day batches; to a publish-subscribe model for immediate updates of critical data. Tools designed for single-modality can quickly hamper the reliability and scalability of a CDI solution.