This article is excerpted from Data Warehouse Project Management (Addison-Wesley, 2000). All material is copyright Addison Wesley Longman, Sid Adelman and Larissa Moss. This is the first article in a two-part series on Data Warehouse Indicators of Success.

There has been much heated discussion over the failure rate of data warehouses. Luminaries disagree on the percentage of those that have succeeded. The problem may be with the definition of success and failure. In fact, very few organizations have identified, up front, what for them will be success or failure. That being the case, any industry-wide numbers on failure are meaningless.

We have all seen projects that are grossly over budget, years behind schedule, and delivering a fraction of the promised function. For various reasons some of these projects have been declared a success with promotions, industry and publication awards and bonuses for all involved. There is a clear problem in that we have no accepted definition for success.

So that everyone is working off the same set of goals, and so that you are legitimately able to declare victory, we suggest that you list your measures of success and have that list approved by both IT and by the business sponsor.

Measures of Success

1. Return on Investment (ROI)
   ROI can be achieved a number of ways:
   • Lower cost – Costs could be lowered through better inventory management, fewer dollars spent on unproductive measures, product promotions, etc.
   • Improved productivity – Greater productivity could be expected from both IT and the user. Today user analysts may spend 80% of their time gathering data and only 20% analyzing the data. The data warehouse should reverse those numbers. IT will still be responsible for developing complex reports as well as writing reports for production systems. The data warehouse can provide reporting tools with a well documented, clean, and easily accessible database. This capability should significantly improve IT productivity.
   • Increased revenue – This could be as a result of greater market share and increased sales as marketing is able to more effectively target customers and provide the right products at the right time to the right market.
   The effects of costs and revenues may be difficult to assign to the impact of the data warehouse. As the data warehouse is being implemented, the organization is not standing still. There are both internal and external factors that impact both costs and revenues, and so the actual benefits of the data warehouse may be difficult to determine.
2. The Data Warehouse is used
   One of the easiest categories to understand can be measured by the number of users and the total number of queries and reports generated. If queries and reports are run regularly, it’s a good indication that the users are achieving some benefit.
3. The Data Warehouse is useful
   The data warehouse may be used, but the users may find the benefits to be marginal and illusive. It’s important to ask the users what they see as the benefits of the data warehouse, how it has changed the way they do business, how it may have improved their productivity, and how it may have improved the quality of their decisions.
4. Project delivered on time
   This measure is problematic, as schedules are often set without an understanding of what is involved and how long each project task will take. “On time” is only relevant if a realistic schedule is the base for comparison.
5. Project delivered within budget
   This criterion may be difficult to achieve since the total costs of a data warehouse are difficult to determine. Initially, you may not have known how many users to expect, how many queries and reports they would be generating, and the complexity and resources used by the queries and reports. You did not know how large the data warehouse would be, or how many indexes and summary tables would be required and desired. You may not have anticipated needing a larger CPU. You may not have known that the software was more difficult than the vendors represented, and that teams of software consultants would be required. You may not have anticipated needing to upgrade our network to support the increased line traffic. You may not have anticipated needing to raise the salaries of the data warehouse team nor the increased cost of recruiting the talent required to make the project a success. All these factors will contribute to severely underestimating the budget. “Within budget” is only relevant if a realistic budget is the basis for comparison.
6. Improved user satisfaction
   Users may be internal, external or both. In all cases, the goal is to have users who are happy with the features and capabilities, the performance, the quality of data and the level of support.
7. Additional requests for data warehouse functions and data
   You’ll know you were successful if other user departments are beating down your door with requests for access to the data warehouse, and current users are requesting new data and functions to be added to the existing data warehouse.
8. Business performance-based benchmarks
   This is the most subjective of all the measures and will become the most controversial. Most industries have sets of industry averages as well as benchmark (the best) companies against which they make comparisons. For example, the industry average for quality, represented by the number of defects for a new car, may be three while the best is one. With better information, a car manufacturer in the middle of the pack, may have a goal to manufacture a mid-size sedan using eight worker days. The data warehouse may be able to provide improved, more complete and timelier information and, with this information, the auto manufacturer may be able to achieve their productivity goals.
9. Goals and objectives met
   On the assumption that you have developed goals and objectives, success will be defined by how well these goals and objectives were met. No doubt, not all were met or were only partially met. A scorecard will give you an initial and then an ongoing measure of your project’s success. Use the scorecard at the end of the article to score your results.
10. Business problems solved
    The data warehouse was developed for some specific reason. Perhaps marketing was unable to identify customer demographics for target marketing. If the data warehouse now provides this capability, it should be considered a success.
11. Business opportunity realized
    The identified opportunity might have been the ability to provide information to suppliers through the web so they would be able to respond more quickly to your demands for components you need for your manufacturing process. If the supplier now has fewer stock-outs, the project is successful.
12. The data warehouse has become an agent of change
    The world is changing, and the rate of change is accelerating dramatically. Successful organizations must be able to respond and respond quickly. Decisions must be made more quickly, but this can only happen with better and more timely information. There can be some fundamental changes to the business in the manner and speed in which decisions are made and, the data warehouse can be the vehicle for that change.