Chris Roche, a mate and mentor in all things system-y, reflects on what sounds like a Filipino version of our recent Bologna workshop.
The week before Duncan was slaving away in Bologna on adaptive management I was attending an Asia Foundation ‘practitioners’ forum’ in Manila. The focus of the event was on Monitoring, Evaluation, and Learning for Adaptive Programming.
The idea for the event seems to have been catalysed by an independent review of the well-known Coalitions for Change initiative which – like nearly all reviews I’ve ever read – recommended that the program needed to undertake more systematic monitoring and evaluation. So, in the developmentally entrepreneurial spirit that the program is renowned for, they invited a whole bunch of folk to Manila to help them out.
This involved of people working on a variety of programs in Sri Lanka, Nepal, Myanmar, Indonesia, Papua New Guinea and of course the Philippines, folks from ODI and DLP, a spattering of consultants [is that the right collective noun? ed], a few academics as well as some UN, EU, USAID, and DFAT staffers.
Whilst some of the debate was reminiscent of ‘governance’ discussions of ten years back – i.e. we need to build ‘capacity’ to do this stuff, or we need to get the technical stuff right – there was also some recognition of the politics of evaluation, as well as the organisational obstacles that mean that learning nearly always gets trumped by bureaucratic demands. As one staff member of Coalitions for Change’s put it, as a front line operative James Bond is the best role model, and ‘James Bond didn’t write reports’!.
Some key take-aways for me were:
Simple rules for complex situations. Many of the programs represented were trying to simplify the processes and the language they used. Like others, they recognised that dealing with complexity was not helped by having overly intricate procedures and processes. Rather, simplicity encourages greater flexibility and responsiveness. Experience had also told them that simple evaluative questions and things like outcome mapping’s use of ‘plan to see’, ‘like to see’, ‘love to see’ as progress markers, made a lot more sense to staff and communities than more esoteric categorisation of hundreds of performance indicators into narrowly defined output/outcome/impact boxes.
Reframing accountability. Much of the discussion was focused on the challenge of managing accountability to donors, or ‘principal-agent’ forms of accountability, and how this could undermine processes of adaptation and learning. Yet interestingly, a number of the examples given of effective learning were actually forms of social, peer or indeed political accountability. Participants described the use of Facebook and viber to share pictures and records of meetings or events with colleagues; citizen report cards to get feedback on services; and empowering communities to use drones to detect illegal logging or mining. All of which suggest a more rounded concept of accountability might not only improve monitoring and evaluation, but create different forms of incentives to promote learning and adaptation.
Gender, teams and categorical thinking. As one presenter suggested, we don’t need a theory of change – we need a theory of people! Relationships, trust and notions of ‘being in it together’ are critical for teams, networks or coalitions to admit mistakes, be honest about uncertainty and therefore be in a position to learn from failure as well as success. And yet these things are not only hardest to measure, the measurement process can also distort those relationships. In similar ways the lack of attention that is sometimes given to questions of gender relations in programs that are ‘Doing Development Differently’ or ‘Thinking and Working Politically’, can mean that they focus on formal institutions and more visible forms of power. In both cases, this misses the relational aspect of interactions, power and politics and therefore the underlying dynamics that produce and re-produce outcomes. Ongoing action research, social network analysis, the use of diaries and apps, and varieties of outcome mapping and harvesting perhaps hold more promise for capturing changes in these kinds of relationships
For thoughts from someone else who was at the event see Arnaldo Pellini’s blog. A workshop report will be out in a few weeks.