Blog: William McKnight Subscribe to this blog's RSS feed!

William McKnight

Hello and welcome to my blog!

I will periodically be sharing my thoughts and observations on information management here in the blog. I am passionate about the effective creation, management and distribution of information for the benefit of company goals, and I'm thrilled to be a part of my clients' growth plans and connect what the industry provides to those goals. I have played many roles, but the perspective I come from is benefit to the end client. I hope the entries can be of some modest benefit to that goal. Please share your thoughts and input to the topics.

About the author >

William is the president of McKnight Consulting Group, a firm focused on delivering business value and solving business challenges utilizing proven, streamlined approaches in data warehousing, master data management and business intelligence, all with a focus on data quality and scalable architectures. William functions as strategist, information architect and program manager for complex, high-volume, full life-cycle implementations worldwide. William is a Southwest Entrepreneur of the Year finalist, a frequent best-practices judge, has authored hundreds of articles and white papers, and given hundreds of international keynotes and public seminars. His team's implementations from both IT and consultant positions have won Best Practices awards. He is a former IT Vice President of a Fortune company, a former software engineer, and holds an MBA. William is author of the book 90 Days to Success in Consulting. Contact William at

Editor's Note: More articles and resources are available in William's BeyeNETWORK Expert Channel. Be sure to visit today!

Recently in Business Intelligence/Data Warehousing Category

If you want proof that business intelligence still has not made it everywhere yet, take a look at what typically happens when a company is purchased by a private equity (PE) firm, whose goal is typically to turn the ship around to (more) profitability and growth and sell in 3-5 years.  Executives are incented with company performance and often put their own skin (i.e., money) in the game.  This creates the effect of a sprint out of what was a marathon.  Waste is removed from the company equation, and then some. 

So, what happens to business intelligence initiatives when this growing movement of private equity ownership happens to a company?  From what I gather, 50% of the time, BI is either substantially reduced or eliminated.  Ouch.  Do the new owners think the company can achieve its new goals better without BI?  Do they see BI as only relevant for companies with goals beyond 5 years?  Some clearly do.

We are now several years into the emergence of the PE trend and are seeing some of the aftereffects of the PE strategies.  One of them is the RE-introduction of BI.  In these cases, either BI was (1) providing value, but it was imperceptible by the PE firm or (2) not providing good value.

Either way, in my experience, companies never ultimately do not do business intelligence.  It may be called something else.  It may be dormant for a while.  It may be floundering and in need of revitalization.  PE firms take note - let's evaluate the BI in place before making changes.  BI can, and ultimately does, work with a strategy of streamlined, 3-5 year horizons.

Technorati tags: data warehouse, Business Intelligence, private equity

Posted January 25, 2009 8:44 AM
Permalink | 1 Comment |

The biggest business intelligence trend in 2009 will be a flight to perceived value. This will include the return of ROI measurements for what we do. This has happened before and will be nothing new to industry veterans who remember 2001 and 2002, when good economic times burst with the dot-com bubble and world events.

Almost instantly vaporized were the speculative, technology-first business intelligence projects, replaced with a return to the cold, hard questions of “Why are we doing this? No, really – why?”. The largely silent response quelled business intelligence progression for a while, but with some confidence returning in 2003, some of the handcuffs were loosened. However, it has never returned to pre-2001 glory, and that is a good thing.

Anytime a discipline gets too self-absorbed and fails to remember its long-term existence depends on contribution to the bottom line of the business, it is set up to fail. However, along with the good of the refocusing of business intelligence come the challenges. Then and now, business intelligence, when not inextricably tied to its business results, loses its way within an organization. The value of the underlying infrastructure, providing well-performing high-quality data to applications, processes and people tends to get overlooked when BI is positioned as technology. BI’s role in the value chain can be lost.

Ultimately, BI is more important than it has been for quite some time. The techniques that are going to emerge in 2009 are those that play to perceived value and the easiest form of that is reducing the expense of BI. These include SaaS BI, open source and quite possibly data warehouse appliances (for replacement deals.) These will eventually include the cloud .
Whatever the method, BI must provide perceived and recognized value.

Fortunately, I was exposed to the ROI requirement in my first business intelligence project back in 1994, thought it to be a great idea even when not required, and include it routinely now.

Technorati tags: data, Business Intelligence, return on investment, ROI

Posted January 12, 2009 12:34 PM
Permalink | No Comments |

I thought I would start off the new year with somewhat of a recurring theme today – saving money. And specifically, one approach to information management that is being touted as very aligned with this strategy is open source. Although open source is generally working its way into organizations from other angles like JBoss and Linux, open source DBMS is a second-tier port of open source into many organizations.

“Enterprise DB is to Postgres what Red Hat is to Linux” is how the folks at Enterprise DB put it. They augment and expand the open source DBMS Postgres with Oracle, and other, functionality. They use open source Postgres and keep to its updates, unlike other vendors like Netezza and Greenplum, who have picked up a version of Postgres at some point and forked off. I’m not necessarily putting either approach forward as a virtue, just as the facts. Other open source DBMS models include the more popular MySQL, which is owned by Sun, who could close-source it at some point if desired. Some of the newer features in MySQL (MySQL Enterprise) are only available to paying customers.

Note some of the legacy of Postgres in the picture. “Post”gres stands for post-Ingress.


Sometimes the functionality in Enterprise DB trails Oracle by a year depending on the need in the marketplace. However, it is a tight fit and code port effort has to do with the depth of recent Oracle SQL extensions used. Enterprise DB is targeting the SMB market, who may not need all the bells and whistles of closed source DBMS. They offer support of open source Postgres and 2 levels of support (code is free) for Enterprise DB, their variation of Postgres. At $4500 for an annual subscription, it’s not free, but enticing.

I will be responding to an ask-the-expert question here on the B-Eye-Network very soon on whether open source can be used to run an enterprise data warehouse. Look for it in the video blogs here.
The open source DBMS are mostly utilized in OLTP. However, I think that is more a result of more DBMS in OLTP. So can open source work in EDW? Yes, it can and yes it is. I've been ramping up my team's capabilities around open source. I’ll have some cautions about open source EDW when I get to the video.

Technorati tags: data, Business Intelligence, open source, Enterprise DB,MySQL

Posted January 7, 2009 2:49 PM
Permalink | No Comments |

In several of my recent presentations, rather than get too caught up in detail of topics tangenial to the presentation, I referenced some white papers and articles that I have done on those topics.

So, to make them easier to find I thought I'd put them here. These also comprise a fair overview of business intelligence related topics.

White Papers:

Data Mart Consolidation: Repenting for Sins of the Past
Modernizing and Advancing Information Management Across the Enterprise
Overall Approach to Data Quality ROI

Articles (to find them, google "McKnight" plus the name of the article):

Justifying and Implementing Master Data Management
BI Business Value: Does it Come from the Program or the Projects?
Supporting KPIs with the Data Warehouse
The Integration of Data Warehouse Data with Campaign Management

Technorati tags: data, Business Intelligence, Data Warehouse, Information Management

Posted November 17, 2008 12:16 PM
Permalink | No Comments |

With a title like “It’s a Data World After All!”, I had to attend because I couldn’t agree more with the principle. I’d also like to attend a presentation with a counter point of view - if you know of any.

Anyway, this session was by Pat Wilson at Disney, from just down the street. She’s on the data warehouse team. The premise was that the approach to gathering requirements for a data project is different from other projects. Like many companies using analytics, Disney makes hotel offers based on your answers to their questions with the offers that are most likely to succeed.

She went through some BI basics and made a good point about not collecting data for which you were unaware of the eventual usage and to avoid the “kitchen sink” requirement which goes like “just give us everything, we’ll use what we need.” Also avoid the “mind reader” approach which goes like “just give us what we want, we’ll know what to do with it.” Also avoid the “trial and error” approach which goes like “we’ll know it’s right when we see it.”

I think this presentation could correlate each scenario above to a Disney character. May I suggest, respectively, Goofy, Evil Queen, and Sleepy.

The (great) overall point is get DEEPER with your requirements for these projects and don’ t accept the shallow “requirements” the business tries to get away with. It’s a recipe for failure. She recommended an approach that follows along with this model:


I definitely agree with everything said here. It tracked pretty closely to my "ROI and Focus" blog in terms of why the data warehouse team exists and my article on Requirements Gathering (search Google for “McKnight Data Warehouse Requirements Analysis”).

Technorati tags: Business rules, Business Rules Forum, Business Requirements

Posted October 29, 2008 8:31 AM
Permalink | No Comments |

I was at the Business Rules Forum in Orlando yesterday to present and take in a few seminars otherwise. I’ll report on a few here. First, I’ll report on “Beyond Subject Matter Expertise” by Steve Demuth at ILOG.

What does business DNA look like? Process is heart and soul of how things get done. Then, decisions go into those processes. Similar to how we’ve evolved, you can change DNA without becoming a new business.

Process describes the how of the core activities of the enterprise. A decision determines the what of enterprise activity and it’s typically automatable or human, but not a mixture.

Automated decisions are ubiquitous (i.e., commissions, cross-selling, fraud.) The talk then focused on how to automate a decision intelligently. Sometimes, it’s a decision table, sometimes a rule flow (flowchart.) How to turn analysis into rules: model the landscape, understand the business goal, and formulate and formalize the solution. The last step is where your business rule management system (BRMS like ILOG, Fair Isaac) comes in. A value-added step at that point is to add simulation on historical data, perhaps in your data warehouse. Then, analyze the simulated outcome.

Then, Steve talked about numerically characterizing history to evaluate those outcomes and predict the best solution to take. It’s about predictability and likelihood. For example, in a group of transactions, how many will be fraudulent or how many will take-up the cross-sell offer?

Then, Steve talked about planning and scheduling with BRMS and how it can create a (for example) optimized nurse schedule for a hospital and deal with the inevitable last-minute decisions that must occur based on last-minute no-shows. This is an example of creating the adaptive enterprise – one that adapts to business changes. However, to get there, we need to break down the hedgerows between business departments, specifically IT and business groups.

Finally, 80% of a business’ problems are about being better at what you do. Business rules can help. 20% of the problems are about being something different than what you are. Both this “adaptation” and “creation” (potentially “destruction”) are necessary.

Technorati tags: Business rules, Business Rules Forum

Posted October 29, 2008 8:28 AM
Permalink | No Comments |

Next up was “Unstructured Text Analysis” from Julie Hartigan, who holds a PhD in natural language processing. First, she established that search is not mining or analysis.
Then I learned another new word: zettabytes, which is a million petabytes. Used in a sentence by Julie: In a study by IDC, it was determined that the digital universe is projected to be 1.8 zettabytes (roughly 1,800,000 petabytes) in 2011.

Since 85% of data is unstructured data, we are making decisions on 15% of data. Search is the backbone of access, but it’s only one way we find information. We also categorize, cluster, and link (if you like a, you’ll like b). She made the case that most searches are small (i.e., few predicates) and query revision is the norm. I think she means queries issued by people, not queries in production, which have tended to get large over time in development and can tend to be quite large (and frequently run.) However, the point that we want our data back fast, whether we’re IT people or business people, and without writing long queries is taken. I.e., select gross-profit, not select . Taking this a step further, I am continually pushing in my consulting for deriving more data in the back end and making it easy on the end user. Easy=they use it, Hard=they don’t.

Back to Julie, she said search is not the answer, but language may be. However, we need to get ambiguity out of language – and there’s plenty of that. I.e., “light brown dog”, “pressing a suit”, “the chicken is ready to eat”, “he saw her duck”. Then there’s similes, metaphors and idioms in language. We need text mining and NLP (natural language processing not neuro-linguistic programming) is the most effective. We decomposed some sentences like the tools do.

Adding unstructured makes it not the “single version of the truth”, it’s the “single version of the whole truth.”

Julie went into “voice of the customer” – mining call logs, for example to determine “unhappy” sentiment was due to the fact that beneficiaries are not getting the support they needed. Regarding VOC, what if call records were not structured? Search (i.e., for the word “billing”) would not suffice for analysis. You really must structure the data to see trends and norms.
She showed a dashboard from Claraview of some analysis of hotel sentiment from, replete with categorization of sentiment and drill-through to the detailed feedback. i.e, “This was one of the worst hotels I have stayed at. The room heater made deafening noises all night long. The staff at the hotel was all extremely rude and condescending.”

Example used were from products Attensity and Clarabridge.

In closing, there was some discussion of mining abbreviations. One language that will be a real challenge to mine is the one we heard Tuesday night at Circque de Solei’s “Ka” show – Grammelot (yet another new word tangentially credited to Partners.)

A couple of classic Teradata case studies and a panel of grumpy old men (their title) - where we were asked not to journal it - later and it was time to leave Partners. As I left Partners, what I’ll remember beyond the conference is the high quality of the Teradata team – both Teradata employees and the community of professionals. That is certainly one of Teradata’s biggest assets.

Technorati tags: Teradata, Teradata Partners, Unstructured Data

Posted October 17, 2008 9:30 PM
Permalink | No Comments |

There were a couple of great best practices in Claudia Imhoff’s operational business intelligence presentation. How about

1. Any weaknesses you have in your BI will only get worse when you speed it up
2. Don’t make (or think you’ll make) big changes to operational processes – that’s done a bit at a time

Technorati tags: Teradata, Teradata Partners

Posted October 17, 2008 9:28 PM
Permalink | No Comments |

First up yesterday for me was “Teradata Spatial” by Michael Watzke of Teradata. He talked about the ST_GEOMETRY data type. Ideas for its use included area, perimeter, census tracks and streets that intersect. For example, find all customers within 100 meters of another customer. Here’s the new word from this presentation: tessellated.

Use cases presented were in insurance (customer density) and communications (find customers near stores or in a marketing area.)

Teradata 14 will add the Raster data type for imagery and geocoding.

Overall, I think this is a nice addition to Teradata’s already strong feature/function set. It will be very important to some and ignored by others.

Technorati tags: Teradata, Teradata Partners, spatial,data warehouse

Posted October 17, 2008 9:25 PM
Permalink | No Comments |

I conducted some podcasts for the B-eye-network this week at Teradata Partners, which are available here on (not surprisingly) the B-eye-network. Check out the podcasts in the lower left corner of the home page.

I spoke with Deb Hoefer of Teradata, who is spearheading a training program for Teradata business users. For example, one course is Teradata SQL for Business Users. I definitely agree they need training and we spent some time talking about how they can customize a program – something obviously needed. I think this is a great idea as user communities are clamoring for empowerment! This topic actually has been a top three requirement in the strategies that I have been conducting lately for clients.

I also spoke with Ray Wilson. We talked about Teradata’s programs for anti-money laundering and meeting the regulations for uncovering the concealment of sources of income.

Finally, I had a podcast with Alok Pareek, the CTO at Golden Gate. We talked about their technology (they did some co-development with Teradata), but primarily we talked about dual-active. It was a good quick journey into the value of dual-active, some of them in ways not so obvious, as in environments where there can be zero down time or risk for an ERP upgrade. By using dual-active, some companies are running multiple versions of their ERP against the SAME data, with only limited pressure on the end users to get on the more recent version. I mention at the end that upgrades are one of the most challenging things IT does and better ways than the high-risk weekend war rooms are needed.

Enjoy all the Partners podcasts.

Technorati tags: Teradata, Teradata Partners, business intelligence,data warehouse

Posted October 17, 2008 9:24 PM
Permalink | No Comments |

Search this blog
Categories ›
Archives ›
Recent Entries ›