Why Time is Essential to Data Analysis

21 Aug

In this article I’ll discuss how the concept of time is absent in many data analysis methods. The brain receives data as a moving pattern; no image is static or processed as static by the neo cortex. Not factoring in this dynamic in analysis can produce statistically significant results but which ultimately have no worth for organisations.

When exploring how people analyse data it’s actually worth taking a quick look at how the brain operates. The neo-cortex, a thin sheet covering the majority of the brain, houses neurons. The neurons have strands, known as the axons and dendrites, which join together to from synapses which have action potentials. All stimuli which enter the brain are actually a moving pattern (as opposed to just image) and using memory, certain responses are activated in response to certain types of incoming data.

Hawking et al (2004) provide a good example of how this works- when you throw a ball suddenly at someone the brain applies a very quick short cut to catch it by using memory. So, the two takeaway points from this very crude and over simple description of the brain are- the brain is a pattern recognition “machine” which identifies patterns in time\ motion and activates actions available in memory.

The description of the above is fundamental to the analysis both conceptually and practically of data; it’s all about how we recognise and react to patterns. In my last blog I referred to a research study I’d conducted examining how power and influence could be communicated effectively via e mail. A respondent in this study, during the course of 2 years, had changed jobs and had tacitly adapted their e mail style to adapt to the new perceptions people had of this respondent when they changed roles. In short- commands and distance became negotiations and social bargaining.

If the respondent had not adapted their mental model of the world after changing roles they would have lost the ability to get things done effectively (this was empirically evident). This would have meant using the existing memory to respond to different patterns. The respondent had recognised the leverage point between using their old style of communicating and the incoming new style in which people were responding to them, so they expanded and adapted their memory store. This provided the respondent with a richer mental model of the world as they learned a new reservoir of responses to new patterns.

The analysis of data is crucial to making sense of information, and good analysis requires what the respondent in question demonstrated- SITUATIONAL AWARENESS. This basically means maintaining a sense of a situation even whilst the surrounding patterns might change rapidly. This allows you to not only recognise subtle changes in a pattern; it also allows you to produce descriptions and actions. The richer your mental model the more situational awareness you can maintain.

Our brains process life as a song not an image (Hawking et al), absolutely nothing is static, everything is in motion, and this is because everything unfolds through time. We severely undermine our analytical effectiveness by analysing data out of time by trying to get only mathematical traction on events, this doesn’t build our reservoir of memory and potential actions, and it simply produces data. To better analyse data we need to understand the flow of time and seek to maintain situational awareness, not just create chains of events.

A way of illustrating this is by imagining predictive statistics as a snake; the snake starts at the tail, this is the current moment in time, and the rest of the snake, all the way to the end of its head is the predictive trend. The snake is a picture, an image, it illustrates a trend but it exists out of time, to place the snake back in time is where situational awareness comes in. By increasing the understanding of a domain we set the snake in motion, we begin to see potential rocks it might encounter, how a weather system might lead to a flood causing the snake to change direction, a potential avalanche or a particularly good run of weather which might accelerate its journey. As we think about the journey through a domain we can start to plan for potentials, to better equip the snake and identify contingencies it might need to make.

Data analysis is greatly enhanced when we first understand the system we are attempting to analyse and its flow. The respondent in the email study understood that a fluent, dynamic domain first required an understanding of the mental model of the person they were communicating with. By collecting (tacitly) a more expansive memory of the e mail domain they operated in, the respondent was able to analyse data more effectively. They had linked data and time, and I would suggest the sum of data + time is adaption.

Organisational analytics can be improved through capturing and increasing organisational wisdom through memory, a system of analogy and there is likely a software enabler here. This is my current research challenge, to place time in data analysis leading to maintained situational awareness and improved data driven decisions. To analyse data from a pure quantitative stance is like jumping off a cliff and expecting not to fall, universal laws still apply!!

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: