Database Management | Why Data Warehouse Projects Fail

67 %
33 %
Information about Database Management | Why Data Warehouse Projects Fail

Published on March 4, 2009

Author: findling67



Using Schema Examination Tools to Ensure Information Quality whitepaper. Data Quality is one of the hottest topics in any IT shop. Although very important, Data Quality is far from being enough because decisions are based on information, not on data. Having quality data does not assure quality information. To have quality information, it is necessary to have quality data, but this is not sufficient on its own. We need more.

Tech Notes Why Data Warehouse Projects Fail Using Schema Examination Tools to Ensure Information Quality, Schema Compliance, and Project Success Embarcadero Technologies January 2008 Corporate Headquarters EMEA Headquarters Asia-Pacific Headquarters 100 California Street, 12th York House L7. 313 La Trobe Street Floor 18 York Road Melbourne VIC 3000 San Francisco, California Maidenhead, Berkshire Australia 94111 SL6 1SF, United Kingdom

Why Data Warehouse Projects Fail According to a 2003 Gartner report, more than 50 percent of data warehouse projects failed, and the ones that survived were delivered very late with extremely high costs. In a 2007 study, Gartner predicted once more that 50 percent of data warehouse projects would have limited acceptance or be outright failures as a result of lack of attention to data quality issues. DATA QUALITY IS NOT ENOUGH Data Quality is one of the hottest topics in any IT shop. Although very important, Data Quality is far from being enough because decisions are based on information, not on data. Having quality data does not assure quality information. To have quality information, it is necessary to have quality data, but this is not sufficient on its own. We need more. IT IS ALL ABOUT THE DATABASE SCHEMA Information is produced by an application program that accesses data in a database, usually a relational database such as Oracle, DB2, Sybase, SQL Server, etc. The core of the database is the database schema, wherein are stored all the data definitions, the relationships between the data, and the business rules. The quality of the information depends on 3 things: (1) the quality of the data itself, (2) the quality of the application programs and (3) the quality of the database schema. Joe Celko (, a very well known expert and consultant in relational technology, states that without a quality database schema, it is very difficult to: • Achieve good program performance and • Deliver quality information When developing any database application, we must always ensure the database schema has integrity and consistency – and no flaws. This must be done when the schema is created, and every time it is changed. If the database schema has flaws, the information will be flawed and the Data Warehouse projects will fail. MODELING TOOLS ARE NOT ENOUGH Database schemas are normally created using a modeling tool such as ERwin®, ER/Studio®, or PowerDesigner®. These tools validate the data model for completeness of the model, but they do not have the intelligence to “debug” the data model. QUALITY OF FEEDER SYSTEMS Data Warehouse projects depend on feeder systems. If the database schemas of the feeder systems have flaws, the information produced by the data warehouse will not have quality. This is the major reason why data warehouse projects fail. The database Embarcadero Technologies -1-

Why Data Warehouse Projects Fail schemas of the feeder systems must be validated for consistency, integrity and compliance to the rules of the relational technology before a data warehouse project is initiated. This is where Embarcadero® Schema Examiner™ comes in. SCHEMA EXAMINER Schema Examiner was created to fill this gap, providing a means to “debug” the schema. Schema Examiner provides over 50 diagnostics to assure the schema adheres to the relational model, is consistent and has integrity. Schema Examiner can validate the data model, a set of SQL/DDL scripts or the database schema directly. Schema Examiner can also compare schemas, indicating the differences. Manual validation is impossible due to the size and complexity of today’s database schemas. A SUCCESSFUL DATA WAREHOUSE PROJECT A corporation in the telecom business contracted with one of the major consulting companies to develop a large data warehouse project. The cost of the project was $10 Embarcadero Technologies -2-

Why Data Warehouse Projects Fail million. After the project was in production, they discovered that the quality of the information was not good; many answers were inconsistent. They considered to re-do the entire project or even scrap it. Their committee suggested the hiring of a consultant. The consultant used Schema Examiner and after a couple of weeks of analyzing the feeder systems, he made several suggestions based upon the findings of Schema Examiner. Once the recommendations were adopted, the results improved dramatically and the project was a total success. The client has stated that the success of the project was due to the use of Schema Examiner. They immediately purchased an enterprise license and made it mandatory to use Schema Examiner in all their IT projects, internal or external. The project used 4 Oracle feeder system that were “debugged” using Schema Examiner; they were also compared to each other to discover inconsistencies. Below is a graphical representation of the project. SUMMARY • About half of all Data Warehouse projects fail due to poor data quality (Gartner Group) • Data Quality is not enough - decisions are based on information quality, not on data quality • A flawed schema impacts negatively on information quality • Database schemas must be validated for compliance with the rules of relational technology • Modeling tools validate data models / schemas for completeness, not for compliance • Data Warehouse feeder systems schemas must be validated for compliance • Schema Examiner validates schemas from feeder systems and compares them to verify inconsistencies Embarcadero Technologies -3-

Embarcadero Technologies, Inc. is a leading provider of award-winning tools for application developers and database professionals so they can design systems right, build them faster and run them better, regardless of their platform or programming language. Ninety of the Fortune 100 and an active community of more than three million users worldwide rely on Embarcadero products to increase productivity, reduce costs, simplify change management and compliance and accelerate innovation. The company’s flagship tools include: Embarcadero® Change Manager™, CodeGear™ RAD Studio, DBArtisan®, Delphi®, ER/Studio®, JBuilder® and Rapid SQL®. Founded in 1993, Embarcadero is headquartered in San Francisco, with offices located around the world. Embarcadero is online at

Add a comment

Related presentations

Related pages

Database - Wikipedia, the free encyclopedia

A database is an organized collection of data. [1] It is the collection of schemas, tables, queries, reports, views and other objects. The data are ...
Read more

5 Database Management - Oracle Help Center

Introduction to Database Management. Database management involves the monitoring, administration, and maintenance of the databases and database groups in ...
Read more

Database Management - Oracle | Integrated Cloud ...

Database Management Focus Area ... Enterprise Manager Database Management. Oracle’s Database management capabilities make DBA lives easier by providing a ...
Read more

Deutsche Post | Database Management | Database Management

Database Management - die Basis jeder Kundenbeziehung. Nur wer seine Kunden kennt und weiß, wie er sie erreicht, kann seine Zielgruppen mit den passenden ...
Read more

What is Database Management System (DBMS)? Webopedia

DBMS Organization. From a technical standpoint, database management systems can differ widely. The terms relational, network, flat, and hierarchical all ...
Read more

Datenbank – Wikipedia

Eine Datenbank, auch Datenbanksystem (DBS) genannt, ist ein System zur elektronischen Datenverwaltung. Die wesentliche Aufgabe eines DBS ist es, große ...
Read more

Definition » Direct Marketing / Database Management ...

Direct Marketing / Database Management: Umfasst alle marktgerichteten Aktivitäten, die sich einstufiger (direkter) Kommunikation und/oder des ...
Read more

Object Database Management Group – Wikipedia

Die Object Database Management Group (kurz ODMG) ist ein Zusammenschluss von Herstellern objektorientierter Datenbanksysteme zur Entwicklung gemeinsamer ...
Read more

Category:Database management systems - Wikipedia, the free ...

A database management system (DBMS) is a computer program (or more typically, a suite of them) designed to manage a database, a large set of structured ...
Read more

What is database management system (DBMS)? - Definition ...

A database management system (DBMS), sometimes just called a database manager, is a program that lets one or more computer users create and access data in ...
Read more