I often critisize vendors and others for not being thorough enough. Now it's time to critisize science a bit as well as those consultants and analysts that are 'riding the wave' of negative sentiment surrounding data warehousing.
In several papers I am reading at the moment (ranging from MIS Quarterly, Information & Management, Decision Support Systems and many more journals) I encounter something similar.
Please read the following quotes:
"....the road to DW success has been littered with failures [43,63,80]"
"....nearly half of all DW initiatives end up as failures [38]"
"According
to a press release 2005 by Gartner: through 2007 more then 50% of data
warehouse projects will have limited acceptance, or will be outright
failures."
The above quotes I got from one paper in one paragraph in Decision Support Systems (paper is from 2008 - very recent), which is an important journal in our field of expertise. Since I encounter these statements over and over again, I decided to follow up on the citations.
Let me begin by dismissing the quote refering to Gartner's press release. That's not really a sound scientific basis.....So we are left with two more quotes.
[43] C.Hall Corporate use of data warehousing and
Enteprise Analytic Technologies, Arlington, Massachusets, 2003 URL:
www.cutter.com
Ronald: can not validate the information. You need to buy a report......
[63] S.Kotler When enterprise hit open road: move beyond the silos and let the idead roll, Teradata Magazine 3 (3) 2003
Ronald: I read this article...now it becomes shocking. Let me quote this article:
Ok....this citation is actually referencing the first one [43] (I think...can't access it). We got a loop here....
[80]
M.Quaddus, A.Intrapairot, Management Policies and the diffusion of DWH:
a case study using dynamic based decision support systems, Decision
Support Systems 31 (2) 2001, 223-240
Ronald: ok, this is becoming complex. I quote this paper:
So a citation is
used in a paper that refers to a claim in another paper but with
different citations (you guys still with me here?). So we end up with
three more citations which are off course even further in time, let's
examine these:
[24] R. Hackathorn, Data warehousing energises your enterprise, Datamation 41 (2) 1995. 38-42
-
I could not find this article, so I was not able to validate the claim
that was being made. It sounds to me like some sort of column though.
But again - can not validate.
[25] C. Horrock, Making the
Warehouse Work, 1996, available from http://www.computerworld.comrsear
. . . -htm1r9606r960624DW1SL96dw10.html.
- I could not find this article, so I was not able to validate the claim that was being made.
[67]
The Siam Commercial Bank's Staff, Data Warehouse Questionnaire and
Interview, (6 January-28 February 1998). Personal Communication.
- I could not find this 'article', so I was not able to validate the claim that was being made.
Ronald: This is a link to a whole site.......oh my, how on earth can you make a reference to a whole site??
To
summarize things; I was not able to establish any (empirical) evidence
that would support the claim made in the paper of DSS (and in several
other papers as well) - which is by the way quite a recent paper
(2008). Somehow we are being made to believe that Data Warehousing are
failure prone. Increasingly I encounter consultants and analysts that fuel this negative sentiment surrounding data warehousing. I challenge anyone to deliver some real (empirical)
evidence. As for now - I suggest we all use caution in communicating
that Data Warehouse undertakings tend to fail a lot.
On the subject, there is one interesting piece of paper from TDWI, written by Hugh J.Watson (I think in 2006) that seems to be relevant and hitting the nail on the head by saying - and I quote:
"The data suggests that whether data warehouses are failure-prone depends on one's definition of "failure." Varying with the architecture implemented, there is approximately a 32-47 percent chance that a warehouse will be behind schedule, and a 30-43 percent chance that it will be over budget. However, this does not mean that the warehouse will not succeed. By a more global measure of success, only 10-20 percent of warehouses are potentially in trouble, while the others are either up-and-coming systems or runaway successes"Ronald: And yes, Watson is using an empirical basis.
Posted June 22, 2009 3:24 AM
Permalink | 2 Comments |
Twitter 
LinkedIn 
YouTube
Hi Ronald,
the paper by Hugh Watson you are referring to was published in september 2005 in the Business Intelligence Journal.
Another, more general figure from 2003, CHAOS report, Standish group: an analysis of 13,522 IT projects shows that 15% "failed" and another 51% were considered "challenged." While the overall failure rate is down significantly from a 1994 level of 31%, 82% of projects now experience significant schedule slippage and only 52% of required features and functions currently make it into the released product. Source: The Standish Group.
So the measure of 10-20% of the data warehouses seems to correspond quite well to the general rate of 15% of all IT projects that are considered 'in trouble'. The over time/over budget part could correspond to 'challenged'.
Basically, this would mean that data warehouse projects look much like any other IT project. Something I would tend to agree with.
When you're in uncomfortable position and have no money to move out from that point, you would require to take the personal loans. Just because it will help you emphatically. I take secured loan every single year and feel myself great just because of this.