Human Judgement and Cognitive Computing

13 Sep

McKinsey have published an outstanding interview with Gary Klein and Danial Kahneman. The interview is a reflection on Klein and Kahneman’s classic paper- Conditions for intuitive expertise: A failure to disagree (2009). Whilst the interview reflects on the two authors positions when it comes to intuitive decision making, the prime focus is on executive judgement- is intuition a good basis for top level business decision making? In this article I’ll briefly reflect on some of the key points raised by Kahneman and Klein, and how aspects of cognitive computing could potentially support some of the author’s suggestions.

Both Klein and Kahneman agree that a gut feeling is not a good single basis for making a business decision. Instead, it is an important data point and strategies are required to rule things out. If strategies to rule things out are not in place, then over confidence in gut feelings can occur. This places an emphasis on selling the decision ahead of rigorously testing it.  These conditions, when they don’t encounter luck, have potentially catastrophic results. Klein summarises this argument up succinctly

“Danny and I are in agreement that by the time executives get to high levels, they are good at making others feel confident in their judgment, even if there’s no strong basis for the judgment”

Hindsight analysis of decisions can also reinforce bias. Hindsight, Kahneman argues, can be used to convince that a decision was based in wisdom, as opposed to, for example, luck. This process involves cherry picking events and data to support that a decision was right, or nearly right, or would have worked if wasn’t for a piece of exceptionally bad luck.

To counter the bias that can infect gut decisions, the authors suggest a more measured approach to decision making. This involves treating gut feelings as a data point which should be tested to explore how things could go wrong. Klein suggests some form of simulation where options are explored to investigate how they could go “sour”.  The recommendations also include assessing the quality of data on which a decision is based, Kahneman illustrates the point

“Is it coming from multiple sources or just one source that’s being regurgitated in different ways? Is there a possibility of group-think? Does the leader have an opinion that seems to be influencing others? I would ask where every number comes from and would try to postpone the achievement of group consensus. Fragmenting problems and keeping judgments independent helps decorrelate errors of judgment”

The meaning behind de-correlating errors of judgements is explained by Kahneman with reference to a classic experiment

“…you ask people to estimate how many coins there are in a transparent jar. When people do that independently, the accuracy of the judgment rises with the number of estimates, when they are averaged. But if people hear each other make estimates, the first one influences the second, which influences the third, and so on. That’s what I call a correlated error

A further example most people would be familiar with in organisations is asking an email group for feedback on an idea. Individuals provide feedback and email the entire distribution group. The first feedback is a critique of the idea, the second addresses the first critique and so on. Soon, feedback can lose all shape as focus on the original idea becomes a distant concern.

Kahneman and Klein identify an effective solution; collect feedback on a decision from independent sources. This can simply mean individuals writing down their ideas at the start of a meeting. And this point, along with the earlier arguments put forward by Kahneman and Klein, has influenced our current research.

Recently, we have been carrying out research in cognitive computing. Our focus has been on bridging the gap between human judgement and technology which could potentially support and enhance human decision making. To approach this research, we asked a sample of experienced business professionals, working in a variety of industries, to engage with a simple scenario. The scenario explained a business situation, and was followed by a series of announcements which may or may not affect the outcome of the scenario.

We asked our sample to identify which announcements they considered significant to the outcome of the scenario, and to explain their reasoning why. Each respondent completed the scenario independently. There was no right answer, the method was designed to assess how respondents sized up cascading data and how they explained their own reasoning. Once this was complete, we coded the data for use in cognitive computing software and we are aiming create a “swarm intelligence” for further testing.

Our research is in the very early stages. However, the potential for cognitive computing to support and improve corporate decision making is exciting. Broadly, the software enables the storage and retrieval of de-correlated judgements. This method potentially enables a user to explore and test gut instincts, to investigate what could go wrong with judgements, examine other options and generate new insights on a situation.

The software allows a user access to alternative ways of looking at data, outside of their own individual instincts and conclusions. It provides a mechanism to investigate how decisions could go “sour” through a form of simulation and a method to “rule things out” through the examination of alternative thinking.

This method broadly follows the principles argued by both Kahneman and Klein; a mechanism which allows the user to “post-pone intuition for as long as possible”. This is not to suggest that new technology such as cognitive computing can eradicate human bias, that is unrealistic, but it could be used to improve the quality of decision making and risk analysis by following the conclusions of tried and tested research. I’ll write more on the subject as our testing continues.


The interview

McKinsey Classic Interview

The article

Kahneman, D. Klein, G. (2009). Conditions for intuitive expertise: A failure to disagree. American Psychologist, 80: 237–251


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: