Differentiating Sharable and Autonomous Metadata
The following are examples of sharable data:
- The table name and attributes shared among applications and the data warehouse
- Some definitions of data shared among the enterprise data model, the data warehouse, and the data mart
- Physical attributes shared among the applications and the ODS
- Physical attributes shared from one application to another
- Description of how shared data is transformed as it moves through the integration and transformation (I & T) layer
Ultimately, very commonly used metadata needs to be sharable.
An example of autonomous metadata might be the indexes a table has for its use in an application. At the data mart level, the results of a given analysis might be autonomous. At the data warehouse level, the metrics of the table content may well be autonomous data. At the ODS level, the response time achieved in access and analysis of the ODS is a form of autonomous metadata. In short, many varied forms of autonomous metadata exist. In general, much more autonomous metadata can be found than shared metadata.
Defining the System of Record
In order to make the structure workable as suggested in Figure 12.5, there needs to be a clear and formal definition of the system of record (i.e., authoritative source) for shared metadata (see Figure 12.6).
Figure 12.6 The system of record for the metadata.
The system of record for shared metadata implies that each shared metadata element must be owned and maintained by only one component of the corporate information factory (such as a data warehouse, data mart, ODS, etc.). In contrast, this shared metadata can be replicated for use by all components of the corporate information factory.
For example, suppose that the definition of shared metadata is made at the data warehouse level (a very normal assumption). The implication is that the definition can be used throughout the CIF but cannot be altered anywhere except at the data warehouse. In other words, the data mart DSS analyst can use the metadata but cannot alter the metadata.
Establishing the system of record for metadata is a defining factor in the successful implementation and evolution of the CIF. Another important implication of the approach to distributed metadata is that there be an exchange of this meta object among the different architectural components (see Figure 12.7).
Figure 12.7 Metadata needs to be exchanged across the many different components of the CIF.
The sharable meta object needs to be passed efficiently and on an on-demand basis. One of the more important implications of this sharability is that it be shared across multiple technologies, as illustrated in Figure 12.8.
Figure 12.8 One of the main challenges of sharing metadata across the CIF is that many different platforms must be serviced.
Using Metadata
The whole notion of sharing metadata across different components of the architecture along with the autonomy of metadata at each component level results in a larger architecture. The larger picture shown in Figure 12.9 shows that indeed a balance exists between sharability and autonomy. With this architecture, the end user has all the autonomy desired, and the CIF architect has all the sharability and uniformity of definition that is desired. The conflict between sharability and autonomy that is evident in metadata is resolved quite nicely by the architecture outlined in Figure 12.9.
Figure 12.9 A larger metadata architecture that balances sharability and autonomy
Consider the usage of this architecture in Figure 12.10, where a DSS analyst at the data mart level desires to know the lineage of attribute ABC in table XYZ. The data mart analyst recognizes that sharable information about table XYZ is available. The DSS analyst asks the data warehouse what information is available. The data warehouse offers sharable information about the table and the
attribute in question. If the DSS analyst has what is needed, the question is resolved. If the DSS analyst has what is needed, the question is resolved. If the DSS analyst is still unsure about the table and its attributes, he will push further into the network of sharable and go back to the application. The DSS analyst then learns even more about the metadata in question. If still further questions are being asked, the DSS analyst can look inside the I & T layer or even go back to the enterprise model to see more information about the table and attribute.
Another use of sharable metadata is that of impact analysis (see Figure 12.11). In the figure, an application programmer is getting ready to make a change to some part of the application. The CIF architect will ask the following questions:
- When was the change made in the application?
- What elements of data are going to be affected across the CIF?
With the formalized structure of metadata, it is a relatively simple matter to determine what data in what environment will be impacted by a change in the applications environment.
Figure 12.10 Metadata information all the way back to the applications environment is available to the DSS analyst at the data mart, if needed
|
