Overview of the BI Organization
Key Issues:
- Information silos run contrary to the goal of the business intelligence (BI) organization architecture: to ensure enterprisewide informational content to the broadest audience.
- Corporate culture and IT may limit the success in building BI organizations.
- Technology is no longer the limiting factor to the BI organizations.
The question for architects and project planners is not whether the technology exists, but whether they can effectively implement the technology available.
For many organizations, a data warehouse is little more than a passive repository dutifully doling out data to the ever-needy user communities. Data is predictably extracted from source systems and populated into target warehouse structures. The data may even be cleansed with any luck. However, no additional value, no informational content is added to or gleaned from the data during this process. Essentially, the passive warehouse, at best, only provides clean, operational data to user communities. The creation of information and analytical insight is entirely dependent on the users.
Judging whether the warehouse is a success is a subjective business. If we judge success on the ability to efficiently collect, integrate, and cleanse corporate data on a predictable basis, then yes, this warehouse is a success. On the other hand, if we look at the cultivation, nurturing, and exploitation of the information the organization as a whole enjoys, then the warehouse is a failure. A data warehouse that acts only as a passive repository provides little or no information value. Consequently, user communities are forced to fend for themselves, causing the creation of information silos.
This chapter presents a complete vision for rolling out an enterprisewide BI architecture. We start with an overview of BI and then move to discussions on planning and designing for information content, as opposed to simply providing data to user communities. Discussions are then focused on calculating the value of your BI efforts. We end with defining how IBM addresses the architectural requirements of BI for your organization.
Overview of the BI Organization Architecture
Powerful transaction-oriented information systems are now commonplace in every major industry, effectively leveling the playing field for corporations around the world. To remain competitive, however, now requires analytically oriented systems that can revolutionize a company’s ability to rediscover and utilize information they already own. These analytical systems derive insight from the wealth of data available, delivering information that’s conclusive, fact-based, and actionable.
Business intelligence can improve corporate performance in any information- intensive industry. Companies can enhance customer and supplier relationships, improve the profitability of products and services, create worthwhile new offerings, better manage risk, and pare expenses dramatically, among many other gains. Through business intelligence your company can finally begin using customer information as a competitive asset with applications such as target marketing, customer profiling, and product or service usage analysis. Having the right intelligence means having definitive answers to such key questions as:
- Which of our customers are most profitable, and how can we expand relationships with them?
- Which of our customers provide us profit, or cost us money?
- Where do our best customers live in relation to the stores/branches they frequent?
- Which products and services can be cross-sold most effectively, and to whom?
- Which marketing campaigns have been most successful and why?
- Which sales channels are most effective for which products?
- How can we improve our customers’ overall experience?
Most companies have the raw data to answer these questions. Operational systems generate vast quantities of product, customer, and market data from point-of-sale, reservations, customer service, and technical support systems. The challenge is to extract and exploit this information.
Many companies take advantage of only a small fraction of their data for strategic analysis. The remaining untapped data, often combined with data from external sources like government reports, trade associations, analysts, the Internet, and purchased information, is a gold mine waiting to be explored, refined, and shaped into informational content for your organization.
This knowledge can be applied in a number of ways, ranging from charting overall corporate strategy to communicating personally with vendors, suppliers, and customers through call centers, kiosks, billing statements, the Internet, and other touch points that facilitate genuine, one-to-one marketing on an unprecedented scale.
Today’s business environment dictates that the data warehouse (DW) and related BI solutions evolve beyond the implementation of traditional data structures such as normalized atomic-level data and star/cube farms. What is now needed to remain competitive is a fusion of traditional and advanced technologies in an effort to support a broad analytical landscape, naturally serving up a rich blend of real-time and historical analytics. Finally, the overall environment must improve the knowledge of the enterprise as a whole, ensuring that actions taken as a result of analysis conducted are fed back into the environment for all to benefit.
For example, let’s say you classify your customers into categories of high to low risk. Whether this information is generated by a mining model or other means, it must be put into the warehouse and be made accessible to anyone, using any access tool, such as static reports, spreadsheet pivot tables, or online analytical processing (OLAP). However, currently, much of this type of information remains in the data silos of the individuals or departments who generate the analysis and act upon it, essentially creating information silos. The organization, as a whole, has little or no visibility to the insight. Only by blending this type of informational content into your enterprise warehouse can you eliminate information silos and elevate your warehouse environment and BI effort to a level called the business intelligence organization.
There are two major barriers to building a BI organization. First, we have the problem of the organization itself, its corporate culture, its discipline (or lack thereof) to rein in rogue executives, and its dedication to IT as a facilitator of the information asset. Although we cannot help with the political challenges of an organization, we can help you understand the components of a BI organization, its architecture, and how IBM technology facilitates its development. The second barrier to overcome is the lack of integrated technology and a conscious approach that addresses the entire BI space as opposed to just a small component. IBM is meeting the challenge of integrating technology. It is your responsibility to provide the conscious planning. This architecture must be built with technology chosen for seamless integration, or at the very least, with technology that adheres to open standards. Moreover, your company management must ensure that enterprise business intelligence is implemented according to plan and that you do not
allow the development of information silos that result from self-serving agendas, or objectives. That is not to say that the BI environment is not responsive to the individual needs and requirements of user communities; instead, it means that the implementation of those individual needs and requirements is done to the benefit of the entire BI organization. An overview of the BI organization’s architecture can be found on page 9 in Figure 1.1. The architecture demonstrates a rich blend of technologies and techniques. From the traditional view, the architecture includes the following warehouse components :
- Atomic layer. This is the foundation, the cornerstone to the entire data warehouse and therefore strategic reporting. Data stored here will preserve historical integrity, data relationships, and include derived metrics, as well as be cleansed, integrated, static, geocoded,
and scored using mining models. All subsequent usage of this data and related information is derived from this structure. It is an excellent source for data mining and advanced structured query language (SQL) reporting, and it is the wellspring for data to be used in OLAP applications.
- Operational data store (ODS) or reporting database. These are data structures specifically designed for tactical reporting. The data stored and reported on from these structures may ultimately be propagated into the warehouse via the staging area, where it could be used for strategic reporting.
- Staging area. The first stop for most data destined for the warehouse environment is the staging area. Here data is integrated, cleansed, and transformed into useful content that will be populated in target data warehouse structures, specifically the atomic layer of the warehouse.
- Data marts. This part of the architecture represents data structures used specifically for OLAP. The presence of data marts, whether the data is stored in star schemas that superimpose multidimensional data in a relational environment or in proprietary data files used by specific OLAP technology, such as DB2 OLAP Server, is not relevant. The only constraint is that the architecture facilitates the use of multidimensional data.
The architecture also incorporates critical technologies and techniques that are distinctively BI-centric, such as:
- Spatial analysis. Space is an information windfall for the analyst and is critical to thorough decision making. Space can represent information about the people who live at a location, as well as information about where that location physically is in relation to the rest of the world. To perform this analysis, you must start by binding your address information to longitude and latitude coordinates. This is referred to as geocoding and must be part of the extraction, transformation, and loading (ETL) process at the atomic layer of your warehouse.
- Data mining. Data mining permits our companies to profile customers, predict sales trends, and enable customer relationship management (CRM), among other BI initiatives. Mining must therefore be integrated with the warehouse data structures and supported by
warehouse processes to ensure both effective and efficient use of the technology and related techniques. As shown in the BI architecture, the atomic layer of the warehouse as well as data marts are excellent data sources for mining. Those same structures must also be recipients of mining results to ensure availability to the broadest audience.
- Agents. There are various “agents” for examining customer touch points, the company’s operational systems, and the data warehouse itself. These agents may be advanced neural nets trained to spot trends, such as future product demand based on sales promotions,
rules-based engines to react to a given set of circumstances, or even simple agents that report exceptions to top executives. These agent processes generally occur in real time and, therefore, they must be tightly coupled with the movement of the data itself.
All these data structures, technologies, and techniques guarantee that you will not create a BI organization overnight. This endeavor will be built incrementally—in small steps. Each step is an independent project effort and is referred to as an iteration in your overall warehouse or BI initiative. Iterations can include implementing new technologies, initiating new techniques, adding new data structures, loading additional data, or expanding the analysis to your environment. This topic is discussed in greater depth in Chapter 3.
|
