Blog: William McKnight Subscribe to this blog's RSS feed!

William McKnight

Hello and welcome to my blog!

I will periodically be sharing my thoughts and observations on information management here in the blog. I am passionate about the effective creation, management and distribution of information for the benefit of company goals, and I'm thrilled to be a part of my clients' growth plans and connect what the industry provides to those goals. I have played many roles, but the perspective I come from is benefit to the end client. I hope the entries can be of some modest benefit to that goal. Please share your thoughts and input to the topics.

About the author >

William is the president of McKnight Consulting Group, a firm focused on delivering business value and solving business challenges utilizing proven, streamlined approaches in data warehousing, master data management and business intelligence, all with a focus on data quality and scalable architectures. William functions as strategist, information architect and program manager for complex, high-volume, full life-cycle implementations worldwide. William is a Southwest Entrepreneur of the Year finalist, a frequent best-practices judge, has authored hundreds of articles and white papers, and given hundreds of international keynotes and public seminars. His team's implementations from both IT and consultant positions have won Best Practices awards. He is a former IT Vice President of a Fortune company, a former software engineer, and holds an MBA. William is author of the book 90 Days to Success in Consulting. Contact William at

Editor's Note: More articles and resources are available in William's BeyeNETWORK Expert Channel. Be sure to visit today!

Recently in EII and EAI Category

This blog entry is co-authored by Mike Ferguson (link) and William McKnight and is being cross-posted on our blogs.

When you’ve decided the appropriate forays into EII (staying away from the times NOT to use EII) and you’ve selected your product, you will need to architect it into your information management environment.

The process of getting the EII tool to learn about a data source is called mapping. From this exposure to the underlying sources, you can use the EII tool to create a virtual schema, which will be used on data access. All EII applications will then see and use the single virtual schema.

The technical base of the data sources can be relational databases, packaged applications, file servers, web services and potentially numerous other data stores – operational and decision support e.g. data warehouses. Indeed, this will be a major criterion of your tool selection. EII platforms differ somewhat in the data source types supported.

Generally, one of these EII instances per environment with multiple data sources is all that is necessary. If your users already use a portal to access information systems, the EII platform can become another of the underlying data stores accessed from the portal.

Applications built on the EII platform can support SQL, XQuery for XML, ODBC, JDBC, and other APIs to access the heterogeneous data sources hidden by virtual view(s) defined in the EII platform.

The implementation of EII in an environment is often the first time that an organization will be capable of providing users with access to an integrated view of both their operational and the post-operational environments. A big decision in your EII architecture will be if you want to expose this fact to the user and for what kind of use should you make such views available. Many organizations start out by leveraging their EII investment to rapidly produce operational and regulatory reports that require data from heterogeneous sources. More mature implementations of EII, in environments that have already been able to shield users from underlying architecture elsewhere in the data access environment, should successfully be able to continue this practice.

Technorati tags: eii, Enterprise Information Integration

Posted January 25, 2007 7:26 AM
Permalink | No Comments |

This blog entry is co-authored by Mike Ferguson (link) and William McKnight and is being cross-posted on our blogs.

If you’re ready to take the plunge into EII, you may be surprised to find that one of your existing information management vendors is in that marketplace. It’s also a marketplace of opportunity, where new, focused vendors have emerged.

There are two main types of EII vendors in the marketplace:
1. Model-driven federated query EII vendors
2. ETL tool vendors providing EII via data integration services built using traditional graphical data flows and published as web services

Federated query EII products include
o Business Objects Data Federator
o BEA AcquaLogic Data Services
o Composite Software Information Server
o Denodo
o IBM WebSphere Federation Server (formerly WebSphere Information Integrator) and IBM Information Server
o Ipedo XIP
o Metamatrix
o Sybase Avaki
o XAware

It is also the case that several ETL data integration vendors have extended their data integration tools to support EII. These products include:
o Ab Initio
o Business Objects - Data Integrator
o IBM - WebSphere DataStage SOA Edition
o Informatica - PowerCenter
o Microsoft - SQL Server Integration Services
o Oracle - Warehouse Builder
o SAS - Data Integration Studio

Technorati tags: eii, Enterprise Information Integration

Posted January 12, 2007 8:17 AM
Permalink | No Comments |

This blog entry is co-authored by Mike Ferguson (link) and William McKnight and is being cross-posted on our blogs.

Much discussion abounds about when not to use enterprise information integration (EII). This blog looks at some situations that are not particularly well-suited to the use of EII technology as a solution. Please note that these criteria should be taken only as guidelines.

Generally speaking EII is NOT suited for
o Complex transformations, fuzzy matching and integrating high volumes of data
o A replacement for data warehousing
o Business process management
o Frequent federated query processing with single federated queries integrating data across a large number of data sources. Performance may become an issue as more and more data sources are added to a data integration server. Several vendors do support caching in order to counteract this problem but nevertheless if you plan to integrate data from a wide range of data sources in a single query you would be well advised to benchmark products and compare results before making any purchase

o High volume transaction processing (insert, update and delete) is required to update virtual EII views of data in multiple underlying systems. This is because update processing via EII is still in its infancy and can be subject to product specific restrictions. Also concurrent access to EII servers may cause problems as workload management is missing from many tools. It is recommended that you investigate carefully how well EII vendors support transaction processing and if they support 2-phase commit distributed transaction processing.
o Transaction processing when integrity constraints across data sources are complex and could cause update processing to fail
o A complete solution to enterprise master data management. EII products can potentially provide a virtual view of integrated master data IF EII tools support global unique identifiers (and the mapping of disparate keys to the global one) and also the matching process to integrate data from multiple master data systems of entry does not require complex fuzzy matching. EII may be offered up as a component technology in an MDM solution but it will not provide everything needed for a full complete solution

Technorati tags: eiiTechnorati tags: Enterprise Information Integration

Posted January 10, 2007 8:54 AM
Permalink | No Comments |

This blog entry is co-authored by William McKnight and Mike Ferguson (link) and is being cross-posted on our blogs. This is the first in a series of entries on Enterprise Information Integration (EII). EII is gaining traction for enabling data integration without the need for the physical instantiation of the integration. In other words, EII adds integrated reporting capabilities while minimizing impact on existing systems. We have been selectively adding EII to our data warehouse architectures. Today, we’ll look at those situations when EII makes sense for data integration requirements.

1. Connecting structured (as in data in a data warehouse) data in particular with unstructured data takes advantage of EII’s strength of leaving data in place that could dramatically increase overall storage requirements if duplicated

2. When immediate data change in response to the data view is desired (changing a copy of the data would not suffice)

3. When data transformation is relatively light or nonexistent and just getting the data together for integrated query is the biggest challenge

4. When the relatively worse query performance of EII query is acceptable (versus the obvious advantages of physically cohabitating all data for the query)

5. Some operational and regulatory reporting where the data needed is not completely integrated in one place

6. Integration of Performance Management software with multiple underlying line of business BI systems to allow a company to see performance management at the enterprise level (across line of business BI systems) using LOB metrics to calculate enterprise KPIs

EII is changing and some of the disadvantages and restrictions will lessen over time but it’s a chicken-and-egg situation. More end clients will need to incorporate small adaptations of EII in their decisioning environment to spur the growth. There may be opportunities and TCO propositions to federate your data acquisition requirements now using EII. Small sized, relatively smaller transformations, unstructured and interactive situations provide those opportunities.

Technorati tags: eiiTechnorati tags: Enterprise Information Integration

Posted January 5, 2007 7:33 AM
Permalink | 2 Comments |

Enterprise Infomation Integration, or EII, is now the province of a multitude of ETL vendors. EII gives the user immediate access to information in disparate data stores. However, other than some light operational message passing, it hasn't had much commercial acceptance and little impact to the data warehouse has been felt. So, at this stage, some are asking if it's going anywhere.

True, it's still immature technology. However, there are a myriad of reasons for the lack of acceptance that would be there even if the technology were mature. It's a paradigm shift in how we've managed data/information for the last 25 years. We're used to moving data where it will be used. Some believe it's the foundation of the enormous DW/BI industry (as opposed to providing business advantage with data/information in whatever architecture fits). Plus, corporations have undertaken an unprecedented number of major projects involving data recently and needed to take on proven approaches in the process.

I think the vendors are onto something here. Ascential (IBM) is focusing efforts there. Informatica has a relationship with EII provider Composite Software, a likely acquisition target by someone at some future point. Other stand-alones are Attunity, MetaMatrix, who added the former head of global markets for Gartner as CEO last month, and Certive, who added the former vice president of engineering at Hyperion Solutions to head engineering last month.

Watch this space.

Posted May 23, 2006 8:09 AM
Permalink | 1 Comment |

According to Research and Markets, the EAI market is expected to grow from $2.4B in 2004 to $8.2B in 2011.

Sun has recently completed its purchase of EAI vendor SeeBeyond. (Link)

EAI, or virtual data integration with service oriented architectures, is a natural evolution of business intelligence. It's BI without a separate instantiation of the data, as in a data warehouse.

Various trends will drive EAI over time such as the homogenization of the operational environment with ERPs and the maturation of SOA and EAI software. Current EAI capabilities do not include robust queries with performance - such as typically found today against data warehouses.

Clearly today, we need the data warehouse, even as a centerpiece of many IT environments. Let's keep an eye on the EAI tipping points however.

Posted August 30, 2005 7:06 PM
Permalink | 1 Comment |

Search this blog
Categories ›
Archives ›
Recent Entries ›