Blog: Rick Sherman Subscribe to this blog's RSS feed!

Rick Sherman

Welcome! In addition to data integration, my BeyeNETWORK blog will include observations on the business and technology of performance management, business intelligence and data warehousing. Most posts will be hosted on my Data Doghouse blog, so feel free to leave comments here or on the Data Doghouse. If you'd like to suggest topics or ask me any questions, please email me at

About the author >

Rick has more than 20 years of business intelligence (BI), data warehousing (DW) and data integration experience. He is the founder of Athena IT Solutions, a Boston-based consulting firm that provides DW/BI consulting, training and vendor services; prior to that he was a director/practice leader at PricewaterhouseCoopers.  Sherman is a published author of more than 50 articles, an industry speaker and has been quoted in CFO and Business Week. He also teaches data warehousing at Northeastern University's graduate school of engineering. You can reach him at and follow him on Twitter at

Editor's Note: More articles and resources are available on Rick's BeyeNETWORK Expert Channel. Be sure to visit today!

(This is part of our ongoing Series of Unfortunate Data Warehousing
and Business Intelligence Events. Click
for the complete series, so far

A fundamental flaw of many business intelligence solutions is recreating
what the company is already using for reporting and analysis. This
takes one of two paths:

1)    The data warehouse is built
using essentially the source systems' data model.

2)    The other end of the spectrum from 3NF is recreating your
current reporting solutions, often data shadow systems or spreadmarts,
that basically flatten out the data.

Click to read the complete post at the Data Doghouse.

Posted June 17, 2010 2:48 PM
Permalink | No Comments |

Over the two decades that companies have been designing and implementing business intelligence and data warehousing solutions they have made the same mistakes over and over again.

Who makes these unfortunate mistakes? Newcomers, mostly. But don't be surprised to see experienced practitioners, who made these mistakes as newcomers, keep making the same mistakes.

Click to read the rest of this blog post on the Data Doghouse.

Posted June 16, 2010 2:45 PM
Permalink | No Comments |

iStock_000003552667XSmall.jpgIt's a vicious battle. No, I am not talking about the partisan wars
between the Democrats and Republicans, but this battle also impedes
progress. Nor am I talking about the decades- or centuries-old
rivalries between religious or ethnic groups across the globe, but this
battle has been going on within enterprises since their first use of

The battle is between the curious and the cautious
regarding the use of technology. The curious would call themselves
innovators while the cautious call them reckless. The cautious consider
themselves prudent while the curious call them Luddites.

battle isn't just being fought on the fringes of the technology
adoption curve. It's raging smack in the middle of mainstream
technology - including technology that's already become a best practice
within Fortune 1000 companies.


Posted December 9, 2009 4:57 PM
Permalink | No Comments |

bigdoglittledog.jpgI'm not only concerned about hand-coding versus ETL tools
I'm also concerned that potential buyers of ETL tools and the market in
general are only looking at a small number of players in the ETL

For many years industry analyst research groups
have identified the top two product vendors: Informatica and IBM (from
its acquisition of Ascential Software). So, naturally, these two appear
on any evaluation shortlist. The rest of the evaluation shortlist
usually includes the bundled products (mentioned in my recent posts)
that come with the databases, BI tools or applications that the
evaluating company already owns. Beyond these usual suspects, other ETL
or data integration products are pretty obscure and almost invisible,
at least from a general market perspective.

>>>Continue this post on The Data Doghouse

Posted November 17, 2009 9:46 AM
Permalink | No Comments |

hand-stop.jpgThis is a continuation of an earlier post that discussed the problems of hand-coding using ETL tools.

What Went Wrong?

are two aspects of effectively leveraging an ETL tool. First is
learning the tool's mechanics. e.g. taking the tool vendors' training
either in a class or through their on-line tutorials. Most IT people
have no problem learning a tool's syntax.  Since they most likely
already know SQL, they learn the tool very quickly.

But the second aspect actually involves understanding ETL processes.
This includes knowing the data-integration processes needed to gather,
conform, cleanse and transform; understanding not only what is
dimensional modeling but why and how do you deploy it; being able to
implement slowly changing dimensions (SCD) and change data capture
(CDC); understanding the data demands of business intelligence; and
being able to implement error handling and conditional processing.

>>>continue reading this post on The Data Doghouse

Posted November 10, 2009 9:34 AM
Permalink | No Comments |

hand-stop.jpgWe are creatures of habit. It's not easy to stop doing something the
way we've always done it. Especially when we think we are right (but
actually we're not). Let's explain.

I have discussed (some might
say preached) in many posts, articles, webinars, podcasts, classes and
client discussions that for any recurring data integration tasks IT
should use an Extract, Transform and Load (ETL)  tool.

certainly has been the best practice for enterprise data warehousing
projects in the Fortune 1000. This is where I got my early experience
in data integration and got to use the ETL tools that annually rank in Gartner's Upper Magic Quadrant and Forrester's Top Wave.
These ETL tools enabled IT groups and SI (system integrator) project
teams to tackle data integration challenges too complex and extensive
for hand-coding.

>>>continue reading this post on The Data Doghouse 

Posted November 5, 2009 5:03 PM
Permalink | No Comments |

In our post Enterprise Software: Is there any one left to buy?
we listed the best candidates to be acquired within the lucrative
Business Intelligence (BI) and Data Warehousing (DW) industry:

  • SAS (Private)
  • Informatica (INFA)
  • MicroStrategy (MSTR)
  • Netezza (NZ)

  • Teradata (TDC)
We have ruled out privately-held SAS because they are very reluctant to
be acquired. (If they were willing then IBM would likely have bought
them versus offense to SPSS but SAS is the 800-lb gorilla in
the business analytics segment.)

big_fish_small_fish_smaller.jpgThe Usual Suspects

The high tech Titans - IBM (IBM), Hewlett-Packard (HPQ), Oracle (ORCL), Microsoft (MSFT), SAP (SAP)
- are already very active in the DW/BI industry selling software,
services and in some cases hardware. Are any of the acquisition targets
on our list attractive to the Titans and would impact their market

>>>Continue reading Enterprise Software: Who Should Buy Whom?

Posted September 30, 2009 12:46 PM
Permalink | No Comments |

titans.gifThe high tech Titans - IBM (IBM), Hewlett-Packard (HPQ), Oracle (ORCL), Microsoft (MSFT), SAP (SAP) and EMC (EMC) - have been buying software firms at an amazing pace over the last half decade. According to BusinessWeek
the tally is: Oracle $30 billion for 56 companies, Microsoft 79
companies, IBM 60 companies, EMC 40, and HPQ 34. Many of these
acquisitions have been "tuck-ins" where the Titans have added the
purchased company's product capabilities into their product portfolio.
This is a time honored practice the Titans and their smaller brethren,
have used to expand beyond organic growth and tap the innovative ideas
of entrepreneurs.

Although tuck-ins have been a mainstay of software mergers and
acquisitions (M&A) for decades, Oracle in the last decade has
raised the bar by being successful in acquiring firms of a much larger
size and, very much against software etiquette, even completing hostile
takeovers. PeopleSoft and Seibel are the two case studies to validate
this new approach.

>>>Continue reading this post on The Data Doghouse

Posted September 30, 2009 12:37 PM
Permalink | No Comments |

database_wars.jpgA recent flurry of news articles in the database industry has
reinforced my perceptions that the software titans have lost touch with
current market needs. 

The PR machines from IBM and Oracle have
been each claiming to have the fastest database and hardware platform
on the market today. Database vendors battling each other on who has
the fastest or the most features is nothing new and has been going on
for a couple of decades. But what has changed is "who cares?"  Of
course it does matter if you are a techie and you are evaluating
which platform has those characteristics. It also matters to industry
analyst and consultants that are evaluating and recommending the "best"
database platform one can buy.  But maybe the key criterion for
future database market share growth is not the best but what is the
"best buy." 

>>>Read the rest of Database Wars: Fighting Yesterday's Battle?

Posted September 21, 2009 3:57 PM
Permalink | No Comments |