Today’s business executives of large enterprises have a ravenous appetite for information about their business performance but are frustrated by the inability of the IT infrastructure to keep up with their changing demands. Part of the cause of this is the habit of some IT departments to apply a straightjacket of formal processes to all applications, with careful separation of the chain of stages from specification to design to coding to testing to production (so-called “waterfall” methodologies, where each link in the chain of project stages is signed off before proceeding to the next stage). Of course such a cautious approach makes perfect sense for large transaction processing systems, where the generic processes remain constant and upgrading to a new package release is in itself a major project, and the consequences of a failure are dramatic. Such systems are automating well-defined business processes (or else they would not be packages), which by their nature do not change too often (to take finance, for example, where double entry book-keeping was invented by the Venetians).

However such rigidity can be a problem for business intelligence applications, where the business needs are much more fluid, and where this month’s burning issue for management is entirely different from last month’s. The consequence has been frustration on the part of business people who have invested millions in business intelligence applications, or the data warehouses that feed these applications with business information, but find that these systems cannot quickly adapt to their new requirements.

IT departments can hardly be blamed for being cautious, since most data warehouses – databases that store a copy of the company’s data for the purposes of analysis - cannot exactly ‘turn on a dime’ when it comes to major changes in requirements. This is due to their rigid structures whose initial build and development is usually based upon the systematic, yet time-consuming waterfall project methodologies. The need to design and build the data warehouse against a formal set of requirements that are to be carefully defined at the start of the project is an inevitably slow process. Traditional data warehouses are typically built in 12-18 months, yet the business can change its requirements several times within this timeframe.

However the rewards for those customers who boldly embrace an iterative or evolutionary approach to data warehousing can be substantial. This alternate approach is based on a flexible data warehouse structure that is created and updated through data warehouse lifecycle management (DWLM) software. This data warehouse automation software has enabled companies such as Intelsat to deploy the first phase of their data warehouse in just 30 days, then adding further feeds and new reports incrementally. At Intelsat, this application has had huge business success, identifying a significant additional saleable network capacity through greatly improved matching of capacity to forward contracts.