Database Management | Why Data Warehouse Projects Fail

67 %
33 %
Information about Database Management | Why Data Warehouse Projects Fail

Published on March 4, 2009

Author: findling67



Using Schema Examination Tools to Ensure Information Quality whitepaper. Data Quality is one of the hottest topics in any IT shop. Although very important, Data Quality is far from being enough because decisions are based on information, not on data. Having quality data does not assure quality information. To have quality information, it is necessary to have quality data, but this is not sufficient on its own. We need more.

Tech Notes Why Data Warehouse Projects Fail Using Schema Examination Tools to Ensure Information Quality, Schema Compliance, and Project Success Embarcadero Technologies January 2008 Corporate Headquarters EMEA Headquarters Asia-Pacific Headquarters 100 California Street, 12th York House L7. 313 La Trobe Street Floor 18 York Road Melbourne VIC 3000 San Francisco, California Maidenhead, Berkshire Australia 94111 SL6 1SF, United Kingdom

Why Data Warehouse Projects Fail According to a 2003 Gartner report, more than 50 percent of data warehouse projects failed, and the ones that survived were delivered very late with extremely high costs. In a 2007 study, Gartner predicted once more that 50 percent of data warehouse projects would have limited acceptance or be outright failures as a result of lack of attention to data quality issues. DATA QUALITY IS NOT ENOUGH Data Quality is one of the hottest topics in any IT shop. Although very important, Data Quality is far from being enough because decisions are based on information, not on data. Having quality data does not assure quality information. To have quality information, it is necessary to have quality data, but this is not sufficient on its own. We need more. IT IS ALL ABOUT THE DATABASE SCHEMA Information is produced by an application program that accesses data in a database, usually a relational database such as Oracle, DB2, Sybase, SQL Server, etc. The core of the database is the database schema, wherein are stored all the data definitions, the relationships between the data, and the business rules. The quality of the information depends on 3 things: (1) the quality of the data itself, (2) the quality of the application programs and (3) the quality of the database schema. Joe Celko (, a very well known expert and consultant in relational technology, states that without a quality database schema, it is very difficult to: • Achieve good program performance and • Deliver quality information When developing any database application, we must always ensure the database schema has integrity and consistency – and no flaws. This must be done when the schema is created, and every time it is changed. If the database schema has flaws, the information will be flawed and the Data Warehouse projects will fail. MODELING TOOLS ARE NOT ENOUGH Database schemas are normally created using a modeling tool such as ERwin®, ER/Studio®, or PowerDesigner®. These tools validate the data model for completeness of the model, but they do not have the intelligence to “debug” the data model. QUALITY OF FEEDER SYSTEMS Data Warehouse projects depend on feeder systems. If the database schemas of the feeder systems have flaws, the information produced by the data warehouse will not have quality. This is the major reason why data warehouse projects fail. The database Embarcadero Technologies -1-

Why Data Warehouse Projects Fail schemas of the feeder systems must be validated for consistency, integrity and compliance to the rules of the relational technology before a data warehouse project is initiated. This is where Embarcadero® Schema Examiner™ comes in. SCHEMA EXAMINER Schema Examiner was created to fill this gap, providing a means to “debug” the schema. Schema Examiner provides over 50 diagnostics to assure the schema adheres to the relational model, is consistent and has integrity. Schema Examiner can validate the data model, a set of SQL/DDL scripts or the database schema directly. Schema Examiner can also compare schemas, indicating the differences. Manual validation is impossible due to the size and complexity of today’s database schemas. A SUCCESSFUL DATA WAREHOUSE PROJECT A corporation in the telecom business contracted with one of the major consulting companies to develop a large data warehouse project. The cost of the project was $10 Embarcadero Technologies -2-

Why Data Warehouse Projects Fail million. After the project was in production, they discovered that the quality of the information was not good; many answers were inconsistent. They considered to re-do the entire project or even scrap it. Their committee suggested the hiring of a consultant. The consultant used Schema Examiner and after a couple of weeks of analyzing the feeder systems, he made several suggestions based upon the findings of Schema Examiner. Once the recommendations were adopted, the results improved dramatically and the project was a total success. The client has stated that the success of the project was due to the use of Schema Examiner. They immediately purchased an enterprise license and made it mandatory to use Schema Examiner in all their IT projects, internal or external. The project used 4 Oracle feeder system that were “debugged” using Schema Examiner; they were also compared to each other to discover inconsistencies. Below is a graphical representation of the project. SUMMARY • About half of all Data Warehouse projects fail due to poor data quality (Gartner Group) • Data Quality is not enough - decisions are based on information quality, not on data quality • A flawed schema impacts negatively on information quality • Database schemas must be validated for compliance with the rules of relational technology • Modeling tools validate data models / schemas for completeness, not for compliance • Data Warehouse feeder systems schemas must be validated for compliance • Schema Examiner validates schemas from feeder systems and compares them to verify inconsistencies Embarcadero Technologies -3-

Embarcadero Technologies, Inc. is a leading provider of award-winning tools for application developers and database professionals so they can design systems right, build them faster and run them better, regardless of their platform or programming language. Ninety of the Fortune 100 and an active community of more than three million users worldwide rely on Embarcadero products to increase productivity, reduce costs, simplify change management and compliance and accelerate innovation. The company’s flagship tools include: Embarcadero® Change Manager™, CodeGear™ RAD Studio, DBArtisan®, Delphi®, ER/Studio®, JBuilder® and Rapid SQL®. Founded in 1993, Embarcadero is headquartered in San Francisco, with offices located around the world. Embarcadero is online at

Add a comment

Related presentations

Presentación que realice en el Evento Nacional de Gobierno Abierto, realizado los ...

In this presentation we will describe our experience developing with a highly dyna...

Presentation to the LITA Forum 7th November 2014 Albuquerque, NM

Un recorrido por los cambios que nos generará el wearabletech en el futuro

Um paralelo entre as novidades & mercado em Wearable Computing e Tecnologias Assis...

Microsoft finally joins the smartwatch and fitness tracker game by introducing the...

Related pages

Database - Wikipedia

Terminology and overview. Formally, a "database" refers to a set of related data and the way it is organized. Access to these data is usually provided by a ...
Read more

Database Management - Oracle

Database Management Focus Area ... Application Tuning using SQL Tuning Advisor, SQL Access Advisor, Real Time SQL Monitoring, Database Operations Monitoring
Read more

Best Database Management Software Solutions | Free ...

The best database management software solutions to automate database management tasks, optimize performance, and reduce administrative costs. Utilize our ...
Read more

Database Management Essentials - University of Colorado ...

Database Management Essentials from University of Colorado System. Database Management Essentials provides the foundation you need for a career in database ...
Read more

5 Database Management - Oracle

Introduction to Database Management. Database management involves the monitoring, administration, and maintenance of the databases and database groups in ...
Read more

What is Database Management System (DBMS)? Webopedia

A database management system (DBMS) is a collection of programs that enables you to store, modify, and extract information from a database.
Read more

Datenbank – Wikipedia

Einen Stammbaum der Datenbanksysteme findet man als „Genealogy of Relational Database Management Systems ...
Read more

OLTP and Database Management | Microsoft

BI and analytics; Cloud data solutions; Data warehouse and big data; OLTP database management; SQL Server benchmarks and performance
Read more

Deutsche Post | Database Management | Database Management

Database Management - die Basis jeder Kundenbeziehung. Nur wer seine Kunden kennt und weiß, wie er sie erreicht, kann seine Zielgruppen mit den passenden ...
Read more

SQL Database Management Objects (SQL-DMO)

Diese Funktion wird in der nächsten Version von Microsoft SQL Server entfernt. Verwenden Sie diese Funktion beim Entwickeln neuer Anwendungen nicht, und ...
Read more