The Challenge of Data Models

One of the key reasons custom solutions are inextensible is because of their instantiation of a fixed data model in a physical database repository or data warehouse. This fate is also shared by “packaged” CDI solutions offered by application vendors (such as Siebel, Oracle and SAP). In a large enterprise, rarely does a single vendor have access to all sources of customer data – external and internal. Therefore, standardizing on the application vendor data model means more, not less, work since every data source outside the vendor application has to be transformed to feed into the vendor’s customer data hub. The best approach is to create a template-driven, logical data model specifically for each enterprise reflecting all its specific customer data sources that need to be integrated. Ultimately, the solution provider has to deliver a data model and a solution framework cognizant of the needs of each major industry vertical. None of these data tools attempt to address the challenge of data models for a diverse set of data sources encountered in various verticals.

Meta-data Driven Framework Needed

The most fundamental short-coming of the trio of data tools (ETL/EII/DQ) is the fact that they do not offer a meta-data framework for managing the complete set of data management tasks required of customer data integration solution. Each of these tools, along with the numerous enterprise application integration (EAI) technologies, solves only a narrow integration issue within the IT “stack” – integrating application to application, moving data to single warehouse, cleansing a single source, etc.

A comprehensive CDI framework must include the tools needed for all processes associated with managing different data types. For example, the framework should address the complete lifecycle of master reference data; model, cleanse, match, merge, share, extend and manage. The solution should allow customer and organization hierarchies across data sources to be leveraged instead of tied to a fixed hierarchical view of an implementation. The solution should readily access all relevant customer activity data and accurately unify it with other data types for a complete view (through caching or aggregation).

For the solution to manage data changes without software programming efforts, it must be driven by meta-data that captures the data syntax, semantics and business rules that are relevant to integrating customer data into unified views. It is important to maintain the distinction between managing meta-data through a generalized meta-data tool versus having a meta-data driven framework designed for a specific purpose (such as CDI). A meta-data driven framework captures, stores and uses highly contextual meta-data tied to a business purpose (such as, when was a customer address changed and by whom). By separating meta-data from its business context, a generalized meta-data tool often limits its business value.