Big Data Doesn’t Equal Better Decisions

24 Mar

The CEO of IBM, Ginni Rometty, reported by Forbes magazine speaking at a recent event, argued that data will change the way people in business make decisions. Rometty suggests that many more decisions will be based on predictability rather than gut instincts and goes onto say that across domains many decisions are still being based on “anchoring” biases. This means that subjectivity and personal experience are driving decisions, and Rometty is clear about this- this isn’t the best basis for business decision-making. The IBM CEO continues that leveraging data in decision-making will be how organisations gain competitive advantage and how organisations make decisions will become even more crucial.

I think Rometty is half right about the above. The half I agree with is subjectivity and personal experience are NOT a good basis for making decisions, in most cases. Rometty draws upon the word “bias” to illustrate her point, and I’ll try and explain how this applies to data and decision-making in business.

In all areas of our lives we are faced with more information than we can handle or be bothered to deal with, consciously or unconsciously. So in order to remain functional, rather than purely analytical, human beings will apply a short cut, a rule of thumb, when it comes to processing information. When a short cut is ultimately counterproductive it becomes a bias. What produces a bias is part biology, we instinctively think fast and jump to conclusions, but also our subjective experiences.  Great for avoiding potential sabre tooth attacks in the wild, not so great in the data saturated world of business. In my experience the most useful first step in assessing the validity of your gut instinct is to assess the domain in which you operate and make your decisions.

Herb Simon’s example of a chess grand master illustrates the point. A grand master, an outstanding decision maker by definition, to make just a single move, has an overwhelming amount of data in front of him; more than the human mind could possibly process in any reasonable length of time. So, if this is the case, if there is more data than he could possibly handle, how does he make the decision, let alone make decisions consistently effective enough to become a grandmaster? The grandmaster will apply a heuristic, a rule of thumb, an instinct, and a grandmaster, again by definition, has developed very good rules of thumb for sifting vast amounts of data. Does this example actually justify using gut instinct and subjective experience? Yes, but in the right domain.

The domain of the chest master is stable, there are rules, and all the pieces are well-known. In addition, across the stability, through every move and through every game, there is clear and relatively fast feedback- e.g. did that move actually work and how should I adapt? In terms of the game of chess, there will be almost no random factors; the rules, the board, the pieces will not suddenly change. Therefore, perceptual experience is valid, it works. A grandmaster can rapidly pick up on a familiar pattern and act upon it, if it doesn’t work, he can update his experience base and reapply.

The business environment is different. Random events define the domain and relying on yesterday’s moves to play today may turn out to be a mistake, both the board and the pieces might have changed overnight. Any domain which involves human behaviour defining the outcome is complex, unstable and feedback can have a very short shelf life. Applying rules of thumb in complex and dynamic environments can lead to bias decision-making and this what Ginni Rometty means when she talks of anchor biases. Anchoring on what’s worked in the past, when analysing data today, can mean you are focusing in on the parts which confirm your bias- shutting out the disconfirming evidence which can give you something new, an opportunity or a warning.

That’s the half of Rometty’s argument I agree with, the half which I disagree with is Ginni’s statement that decision-making will be based more on prediction. Actually, decision-making in the future might be more based on prediction in terms of frequency, but what I’d like to pick up on is this -is predication as a basis for decision-making the best way to use data? My answer is no, and I think the chess grandmaster example provides the why.

In a complex domain, regardless of the perceived sophistication of your software or methodologies, prediction is simply not possible. It’s not possible because human behaviour and random events, even the smallest human action, are not predictable. Human life is not chess, and the inherent assumption in prediction methodologies is that it can be modelled like chess. So, I would argue that when you’re looking at data ask yourself-  can you be sure you meet the grand master conditions of set rules, set pieces, stable environment and clear, fast feedback? If not, then I’m afraid you can’t predict.

What are the take aways for you and your business from this blog? Well you may wish to consider the following 3 tips when using data for business decision making-

1)      Be careful that you are not using the data to confirm what you think you already know, look at data with the view to disconfirm your beliefs; you’ll explicitly add rigour to your analysis.

2)       Look at the range of possibilities and events that the data is presenting, not just anchor onto the key trend, this will build flexibility into your planning and strategies; just because something seems unlikely doesn’t mean it won’t happen, be prepared!

3)      When looking at data across a range, as opposed to a trend, look at the skills within your organisation which can take advantage of more obscure opportunities, and, mitigate against rare but explosive threats; don’t over specialise, think functional skills not specialism. If you over specialise in your thinking you’ll form an anchor biases around matching data to roles and miss valuable opportunities.

I agree with Rometty, competitive advantage will lay with the organisations which can make the best of use of data to inform their decisions; but don’t predict, prepare. And know your domain.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s

%d bloggers like this: