Here’s a sentence you don’t often hear. I just read a really interesting conference report. Transforming Change: How Change Really Happens and What we can do about it, by Paul Knox Clarke, summarizes a big 2017 discussion on the drivers of change in the humanitarian system, as well as the blockers. I reported on it at the time, but went back to it this week, as I am mugging up on the humanitarian system for some future work (details to follow).
One bit that jumped out was the box on the link between evidence and change – this one’s for all you ‘research into impact’ people.
‘Opinion was divided, both on whether we have enough evidence to support change in critical areas, and on the importance of this evidence in processes of change. While some participants felt strongly that the lack of evidence prevents change, others were clear that ‘we have mountains of evidence’; or that (with specific relation to cash) ‘we had 15 years of evidence collected. It wasn’t an evidence problem why it wasn’t being taken forward’.
When it came to the power of evidence to drive change, participants were able to give numerous examples where change had not occurred despite strong evidence. Although there were several credible investigations into protection failures in Sri Lanka, there seemed to be no change ‘in our consciousness or decision-making’. Similarly, while there is ‘very strong evidence of the value of core funding’ to local and national NGOs, many donors seem to have decided to go the other way, to more project-based funding.
In some cases, evidence is not being used because it is not easily accessible to those who most need it – particularly those making decisions on the ground. But a more common – and probably more intractable – problem lies with the way decisions are made. ‘Often
is only a small part of the decision-making calculus for policy makers’, or, as one participant suggested candidly:
‘as the Chief of Policy Analysis [for a large agency] I never read a journal, I never looked at a lit review’.
The panels supported research by ALNAP and others, which has strongly suggested that the use of evidence in humanitarian decision-making is limited, either because consideration of evidence is not an explicit part of the decision-making process, or because decisions are affected by a number of other considerations: politics, resource availability, or security. And, if we think back to the models of change presented in this paper, we should not be surprised if decision-makers are not entirely rational, or if decisions are strongly influenced by politics.
This is not to say that evidence is never used to support change. The meeting also considered a number of examples where the provision of evidence was an important part of the change process and some participants went as far as to suggest that evidence is a necessary, if not a sufficient, condition of change.
Whether or not this is the case (do we need evidence for change, or can change happen without it?) it is interesting to consider how evidence was used to support change – because it seldom seemed to happen in a ‘straight line’, with people considering evidence, and then deciding to follow the evidence in their actions. Instead, participants explained how evidence was often used late in the change process to provide a sense of certainty, and so emotional security, around decisions that had already been made. Evidence was also used to bring people together and provide a common platform for discussion: ‘when there are disagreements, you can have that conversation at a very different level. You have it around technical evidence…and it allows you to surface what might be a tension’. It is interesting to see, in these cases, how the value of evidence lies in its ability to address some of the social and emotional challenges to change, rather than purely to address technical questions.
It was also interesting to see the circumstances under which evidence became important to a change process. It seems that evidence is more likely to be used where it is specifically wanted or commissioned, or where it is answering a specific question that decision-makers are already asking. It is also more powerful at certain times – in situations of uncertainty or doubt, people may be more likely to turn to evidence. Evidence may also become more important in decision-making where the organisation, or system, ‘raises its sights’ and focuses more on outcomes than on conducting activities according to ‘industry tradition’. And finally – and importantly – the importance of evidence in making decisions may differ from one organisation, and even one individual, to another. As one presenter said: ‘I threw facts at him and I threw argument at him, and he wouldn’t budge … Sometimes it’s only the visual, it’s only the human story that’s going to move people there in their heart, and not up there in their head’.’