How to mix Business Intelligence and Social Media to enable decision making

Children are constantly learning. First they crawl than they learn how to walk. And the same happens with playing.  At first they start out doing this alone and they find it difficult to share toys with others. They want to keep those fun things themselves.  But in time the children learn that playing together is more fun. And that playing together also means that you have to share your toys with others. This increases the fun and before you know it you have a new best friend. We have an expression in the Netherlands which goes like this: ”Playing together is sharing together”. This Dutch rime helps children share their toys with other children while playing. As an adult the world looks pretty much the same. We also find it difficult to share our toys, let alone enterprise data or corporate analyses. However sharing can create enormous advantages and will increase your Return-On-Information. In this article we look at the role of corporate information (or business intelligence) and how social networks (or internet social media like twitter) can help decision making. Perhaps in the near future a manager will publish his or hers corporate data on the net with a $50 reward for the person that comes up with the best solution.

From bearskin suits to intelligent use of information

We used to learn this in school during history lessons. The first inhabitants of the Netherlands were fishers, hunters and collectors. They wore bearskin suits and chased after mammoths. But after a while they started working the land and established small towns and villages. This urbanization allowed them to split their roles and tasks. The cow gave milk to the farmer which he used to buy bread from the baker. The industrial revolution ended these small scale economic (trading) activities. The modern time demanded mass production and oil and steel were the new, o so crucial, cows.

The next big break was the information revolution. The introduction of the computer was just the beginning. Everything bit of information was changed into bits and bytes. Information became the new oil and is the lubricant for today’s economy. Data or information has become an essential production factor just like labor or capital. But the information revolution came to us in waves. First, information was broadcasted by one source for the benefit of many. Think about mass media like radio and television. But the internet has changed this model for ever. The net maybe started out somewhat one dimensional with website after website where all possible interest known to mankind was broadcasted. But soon after that it became possible to react on the sender’s opinion thus creating some kind of dialogue. The next step was sharing information or creating a dialogue between larger groups of congenial persons. Communities were being formed.

The importance of information has been recognized by many.  Information has become an economic factor of production and therefore it is one of the things that enable enterprises to achieve competitive advantages. For example, by knowing trends and other developments before others. It has given rise to a field of expertise where collecting information to improve performance has become important: Business Intelligence (BI). Intelligence, not by coincidence, reminds is of the CIA. An organization that also values information. Only the BI field has been dominated by IT. As a result there is a focus on the technology with regard to information supply instead of using information to increase business opportunities. BI is mainly engaged with opening up enterprise data to get insights in the corporate performance. Social media are also engaged with information collection. Only the platform is not the corporate IT systems but the internet.

Oil is scarce, information is abundant

In time commodities like oil and steel are going to run out. With information it’s exactly the other way round. There is an abundance of information available. From every corner comes data or information. From our systems like cash register, book keeping, HR or the ERP system. But also from external sources like internet. This information can take on different shapes and forms. Sometimes it’s a comment placed in an online forum, or a picture, but it can also be a pdf file containing market analyses or a press release. The challenge is to find the famous needle in the haystack, or rather in football fields filled with haystacks.

A first step is finding or locating data that can be useful, in other words: where is it? The next step will be controlling the data. To make heads and tails of it. For example by capturing it in a model to create insight. The central question here would be: What is it? After that we have to look at the value of the captured information. Is it indeed relevant? Does it have some sort of economic value? What can I do with it? The last step is an interpretation of the data. By giving the data some kind of rating or importance. In other words: what can I do with it?

The World is continuously in motion and freezing an existing situation is therefore not smart as the value of information decreases directly. Above mentioned steps are therefore not suggested as a well managed linearly process. Especially for social media controlling information can only be done up and until the process of interpretation. For classical BI the process might be a bit more linear and controlled but still some phases will melt together, as pictured in the information cloud example below.

 Locate: Where is it?

Business Intelligence (BI) is still very traditional in retrieving or finding information. Most information comes from the organizations own systems. In every organization IT systems are being used for capturing data, such as ERP systems, Cash Registers or HR. This data capture is needed to support the operational process. For example, in HR we need to know what the employees name is, how long he has been working for the company and so on. Also we need to know how much we have sold and to who or where. All this data is being stored in the operational systems. But the data can also change in time or even be deleted. Historically this is why we have built datawarehouses. It allows us to capture all the data in time and create a historical view of facts and dimensions. For BI the datawarehouse is often the information or data source of choice. Its limitation is that it often offers only internal data. The type of data is also often static, meaning that is consists of number, tables. Retrieving information from a datawarehouse can be done with BI tools which build reports or list. Often these reports are developed by the IT department based on user requirements. All and all it takes a lot of steps before information can be retrieved.

Locating data or information on the internet is not necessary anymore. Any which way you use the net, you are constantly surrounded with information, coming from WebPages or our own social network. The internet is like an array of millions of datawarehouses that all can possibly contribute to your information supply. For example, this is the first year that computer generate more data than people do. Every minute, ten hours of video is added to Youtube. Every week over ten billion pieces of content are added to facebook. Service like lazyfeed and DailyPerfect can,  based on some criteria (or topics), retrieve all the information that can be of relevance. They can do this much faster and more elaborate than any human could have done. Data is everywhere when you are on the web. It is on websites, in RSS feeds, in your social network or hidden within a simple Google query. Data is more than available, rich in content, from many sources and better still, easy to find.

Where BI is focused on retrieving structured data from corporate systems or datawarehouses, social media do not need to focus on data retrieval. The information is just there. This makes BI very internally focused and Social Media externally focused. The internet is just an array of datawarehouses filled with an inexhaustible amount of information. But where to find the relevant information? Search engines can help here, just as web services or having a network of trusted people and websites. Both worlds (BI & Social Media) are complementary. By combining BI and Social Media both structured as well as unstructured data is captured. But it is also a combination of internal and external data. The real value is in taking either an inside out approach of better still an outside in approach to enrich already existing information. Finding the right combinations is the final challenge.

Control: What is it?

Controlling the information from datawarehouses is done with a BI tool in the form or reports, list. Sometimes even OLAP cubes are introduced. These are larger datasets where the facts or measures can be analyzed from different dimensions like date, customer or product. It looks a bit like a pivot table in Excel. Modeling information in a datawarehouse is difficult and often a challenge. Reproducing this data in reports or cubes is even more difficult. The biggest challenge here is getting the definitions right and the same across the whole organization. If this is not done everybody can interpret the data the way they want it. That way there will always be multiple versions of the truth. Too many meeting have been spent on discussing the validity of the figures instead of using them. Controlling internal data might remain difficult, but when done correctly offers a multi dimensional view across the entire organization. With this one version of the truth comes a great return on intelligence.

Controlling data is also a big issue for Social Media. There is so much data, that if you want to read it all, this is impossible. So you need to filter and make choices in what you need. This is often done by working with trusted feeds, people and topics. By selecting a number of websites, data feeds or subscriptions to trusted source you can easily control your dataset. You can also use your social network to get information. Some people are part of your network because you trust them or because they can add value. You can also choose to subscribe to certain topics. Services like Google Alert of Lazyfeed scramble the internet for you. This way you can create a controllable dataset, without drowning in the sea of information that is available online. Also by working with trusted feeds, people and topics you know that the information you receive probably hold value for you. It is not just static or a waste of time.

In order to control the information BI uses data models. Data is captured, clarified and distributed using reports or cubes. The information is there, relevant or not. But it does provide one stable version of the truth. BI often models this information in line with the organization operational processes (finance, sales, and logistics). This makes these processes explicit and the supporting data known. The added value of this that you can create insight into your own performance: it is looking inside.

Controlling information in Social Media is impossible. This means that you have to filter. By using trusted feeds, people, sources or topics. It is about limiting the dataset. If it is not relevant it does not exist. Therefore a dynamic version of the truth exists. The social media often model the information based on trust. This is done based on your gut feeling, so often implicit. The added value of this is that you can create insight into the performance of others: it is looking outside.

BI and Social Media are complementary as they allow for looking inside and outside. BI on its own is not enough. The same applies for Social Media. But together they offer a 360 view. Just think about the sales director that explained a decline in sales for Region X based on a newspaper clipping he found on the internet that a megastore was opened by the competition.

 Value: Can I use it?

The gap between business and IT seems to grow every day. This makes sense as each party has its own tasks, responsibilities and interests. But at the same time this causes problems, because as long as business and IT do not work together, the BI competence will face many challenges. A BI Competence Center where business and IT work together and align their activities is still considered to be the best practice. So why does this gap still exist?

IT is focusing on operational excellence. Their aim is to get a minimum of changes, defects or incidents. In their struggle with a declining budget they are constantly striving to reduce the total cost of ownership. But how can we reduce the costs of BI? The simplest solution is reduce the amount of changes. That means less resources and releases. The result of this is a decline in the information supply. On the other hand business is focusing on a concept we call business excellence. They are confronted on a daily basis with many questions and they need answers fast. They need to reduce the amount of uncertainty by increasing the amount of information. If IT cannot supply this to them they will take control themselves. The rise of all kind of (managed) self service reporting is exemplary. Information does not only reduce uncertainty it can also create competitive advantage. By knowing things first the competition is put at a disadvantage. This means that the demand for information will increase.  In conclusion, data supply by IT is down and data demand by business is up. This will lead to higher value of the information. The problem here is how to value this information? Because it is not only the information but also the experiences of people using this information, their skills and attitude. BI can help structure data which leads to explicit information. However its real value is made possible through implicit interactions. Humans add value to the information, for example by making decisions or taking actions. But why limit something a powerful as that to only one person?

 And what is the value of information that you have retrieved from trusted feeds, people and topics. How do you determine which information is important to you? To determine the value is something very personal. What some people consider valuable others might throw away. All information you received is by definition subjective. The information differs in value based on the subject or source you retrieved your data from. Some sources are more valuable than others. This is based on past experiences, known expert knowledge of the sources or because the source was recommended by a trusted person.

 But there are also some semi-objective measures to rate the value of information. Semi-objective because every opinion is a group result and groups needs some conditions to come to a sound conclusion (diversity, independency, specialization and the ability to aggregate various opinions according to James Sorowiecki in his wonderful book Wisdom of the Crowds). Possible way to quantify the value of information is to look how many social bookmarking sites have acknowledged a certain website, how some article are rated on social news sites (like DIGG) or what the Google Pagerank is, or how many people subscribe to a certain RSS feed.

Netflix

Netflix is a company that rents out movies on DVD, Blueray as well as online streaming. Its customers can use a recommendation engine (relevant suggestions based on content) which was improved by using social media. Netflix published a set of anonymous data and ask ‘strangers’ if they could improve their recommendation engine by 1%. The person with the best improvement could earn one million dollar. The final result was an improvement of their engine of 10%. This leads to much higher revenue and the costs of the competition were easily compensated. Also these kinds of improvement if done by their own IT could not have been done or would have cost them more than the 1 million dollar. It was much cheaper to just give the data to the social network.

In conclusion, it can be said that the value of information is increasing and it is the human being that decides the real value. Classical BI has a somewhat carefully, safe way of using data. It source is often trusted (own corporate system or datawarehouse). Social media are completely different. They use a multitude of sources. Also the safety or reliability cannot always be guaranteed. But in order to come to an optimal performance it is not always black or white. Business Intelligence can create a foundation for decision making, it can formulate a thesis, and social media can serve as some kind of standard deviation. They can make or break the thesis, confirm or reject it, but always enrich it. 

This article was written together with Rick Mans - Social Media evangelist at Capgemini