Frozen Decisions

7 Jan

When we read a report, analyse data, examine a survey, how long is it valid as a basis for decision making? Five months? Five weeks? Of course, there isn’t a right answer. Every data source requires a degree of vigilance as time and activity chip away at its relevance. I’ll share an example below where a report I wrote for the purposes of decision making was invalid the moment it was printed.

I was recently reminded of a public health evaluation I took part in many years ago. I was asked to evaluate how well a community initiative was performing. The initiative was a centre, situated within the middle of local community; and the centre’s purpose was to provide a range of health and wellbeing services, community group meeting areas and social meeting facilities. Basically a community centre to service the needs of a local community and provide a place for people to meet. The ambition was for the centre to attract multi-generational users, providing a place where people of all ages could meet, get advice and access services.

The centre had been set up and was already running when I was asked to evaluate the outcome of organising and creating the centre. Basically, I was going to be taking a snap shot of how well the centre was working relatively soon after its creation. The measure of “how well” would be taken from the perspective of people who ran the centre, community members who used the centre, organisations who provided services at the centre, and people who had invested, financially and politically, in the centre. To achieve this I worked as part of team carrying out focus groups and interviews with this broad sample of respondents.

The analysis of interviews and focus groups suggested that the centre was broadly achieving its objectives. People of all ages accessed and used the centre for a variety of reasons. However, there was also a hint in the data that the situation was already starting to change. Several respondents, from across the sample groups, had suggested that use had started to swing toward predominately young people. And this swing had started making some of the older users reluctant to attend. These observations presented themselves in the data as comments made by respondents to the question “what are potential future barriers facing the centre?” The report we wrote highlighted that the balance of the centre was potentially under threat. Under threat not by anything specific, but under threat from how changeable group dynamics can be. Especially when the group is composed of diverse members of a community across age and gender. It was certainly something which needed to be monitored and acted upon if necessary.

With no further centre evaluation planned I moved onto my next project. About three weeks after the centre evaluation had been submitted I returned to the location. I was chatting to a few of the previous respondents who all commented how the dynamic of the centre had completely changed. The centre had become very much about young people. At the time I was surprised. The change must have happened almost immediately. However, the change was well underway before we carried out the evaluation. This anecdotal snap shot made the situation seem like some kind of social revolution. But really it was all down to a lack of regular feedback data.

This was a lesson in how feedback should be regular and projects should be seen as a process not only an outcome.  If only outcomes are recorded then it’s a little like taking a picture of a landscape and expecting the landscape to always be like the picture. This approach is time ignorant.  Instead, plans, strategies and decisions should be evaluated constantly and adapted as necessary. This doesn’t mean unleashing a legion of researchers and consultants. It does mean putting in place simple feedback loops so people who are planning and deciding stay in touch with the ground truth of their decisions and plans. Without feedback adaption is impossible.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: