The war for Twaweza’s soul: the hunger for clarity and certainty v the demands of complexity

This is the last in a series of three posts on Twaweza, a fascinating NGO doing some pioneering work on accountability in East Africa, whose big navel twaweza-logogaze I attended last week. Post one covered Twaweza’s theory of change and initial evaluation results; yesterday I got onto the critique of its thinking and action to date. Today I’m digging deeper into some of the underlying issues.

Given its rethink, Twaweza is now contemplating a shift in direction – while keeping its focus on citizen agency, focus in on education (rather than try and cover education, health and water); reduce the number of partners; do more things on its own (eg research or education programming); expand successful areas such as policy and advocacy; do more experiments to uncover what works and help the organization ‘fail faster’ and so move on to new stuff.

Plenty of good ideas in there, but it also seems to me to mark an intellectual retreat from the initial commitment to finding new ways to achieve change in complex systems. I think there’s a strong case for digging deeper into complexity, rather than retreating from it. One suggestion that moves in the right direction is to set up a ‘positive deviance lab’, dedicated to detecting and then understanding examples of success in citizens’ action across East Africa.

What would digging deeper mean? First, building a deeper understanding of the system, before jumping in with any particular intervention. That requires a much closer scrutiny of power, relationships and all the forces that maintain the status quo. Twaweza doesn’t seem to have spent much time understanding why change doesn’t happen (e.g. through a ‘3i’ combination of ideas, interests and institutions), which turns out to be pretty key as its work has encountered far more inertia than originally expected.

One of the common findings of the evaluators was that where good things have occurred, it is most often down to the charismatic leadership of particular grassroots heroes, rather than systemic interventions by Twaweza or others. So why not, like the Development Leadership Program, try and understand the origins of such ‘Gandhiness’, and see if Twaweza can design programmes to strengthen it or influence future leaders?

Finally of course, critical junctures, aka windows of opportunity. Like much of the rest of Africa, Tanzania is standing on the threshold of a huge resource boom, in this case offshore gas in the South. In terms of the evolution of the wider social contract between citizen and state, it is probably up there with next year’s referendum on the new constitution. Yet neither were mentioned in the two days of debates, suggesting that even Twaweza is a bit too stuck in the Plan, and not looking around it to respond to new opportunities.

An underlying tension throughout the discussion was between Twaweza’s commitment to running experiments/measuring its impact, and its desire to find new ways to work in complex systems. I’ve never spent so much time with a group of academic evaluators and the experimental mind is fascinating: endless discussions over how to measure really subtle, difficult things like agency – signatures on a petition? Going to a meeting? Saying the right thing to a phone interviewer? Every discussion is weighed down by questions of attribution and bias.

Gandhi v logframe cartoonBut all this heavyweight cleverness ended up just reinforcing my sense of the limits to quantitative measurement in development. Getting to some numbers that can be properly crunched requires levels of abstraction (eg encoding the responses of hundreds of communities to identify a handful of variables suitable for a regression) and agglomeration that can only do violence to reality.

Even if you manage to extract some level of numerical relationship, understanding why activity a leads to outcome b requires ‘sending in the anthropologists’ anyway, who will doubtless (being anthropologists) come back and tell you that the abstractions mean that your results are meaningless and/or useless. Any clarity or certainty that emerges from such an exercise may be comforting, but is likely to be illusory.

To be fair, Rakesh thinks I’m caricaturing here and says ‘We have come to care about measurement deeply, not primarily because we need to cater to some donor need or academic joyride, but because we want to be more effective, we want to make a sharper difference, we want to be more accountable for results to the people we claim to serve.’ I completely understand that impatient desire to know whether your work is having impact, but I think that, at least as currently conceived by Twaweza, giving priority to impact measurement could also be a  barrier to working effectively in complex systems.

Maybe Twaweza’s measurement problem (and yes I think it has one) stems from using the wrong kinds of metrics – Kate Dyer of the Accountability Tanzania programme was also in the seminar, and described its attempts to make outcome mapping the foundation for much of its learning, even using it to generate some ‘participatory numbers’ – Twaweza could do worse than draw on that work.

There was much discussion about including the evaluators in the design of new projects, but I was pretty alarmed by that too. Sure they do ask brilliant questions, but their need for numbers inevitably ‘contaminates’ (to use one of their favourite words), their grasp of how to work in complex systems. How to avoid the measurement tail wagging the programme dog? There were several dog metaphors – I (doubtless unfairly) likened some of the evaluators to dogs with a bone, letting their enthusiasm for getting stuck in to extracting numbers cloud the original intention of encouraging citizens to take matters into their own hands.

Twaweza’s growing enthusiasm for experiments also raised some concerns, especially when combined with its hunger for numbers. Is handing tens of thousands of teachers a personal ‘cash on delivery’ bonus for getting their pupils to an acceptable grade (as one Twaweza experiment is currently doing) really compatible with building agency, citizens’ voice etc? Sounds more like an incredibly crude bit of behavioural tweaking, with all sorts of potential unintended consequences for the system (think what the much less immediate incentives of school targets have done to schools in the UK and elsewhere – teaching to the test, excluding those likely to fail etc).

Another notable result of the hunger for number-based certainty is ‘Millennium Village thinking’: if we can’t shift the ecosystem at a national level, why not concentrate all our resources on a few communities? Might not even be a bad idea in this case, but interesting to see how the proposal arises from the initial mindset.

So in conclusion, (as they say): the direction that Twaweza takes over the next few years matters well beyond Tanzania’s shores. It is a global leader among NGOs, heroically intelligent, honest, innovative and yes, courageous in acknowledging failure and trying to grapple with many of the issues that bedevil development wonks everywhere – complexity, scale, what and how to measure impact. If it retreats from its grand ambitions in an understandable search for the achievable and measurable (we all want to know we’ve actually achieved some kind of impact, after all), it will have squandered the chance to do something really innovative and we will all be the poorer.

Last word to another one minute Twaweza video (unless Rakesh and his colleagues exercise their right of reply).

Subscribe to our Newsletter

You can unsubscribe at any time by clicking the link in the footer of our emails. For information about our privacy practices, please see our .

We use MailChimp as our marketing platform. By subscribing, you acknowledge that your information will be transferred to MailChimp for processing. Learn more about MailChimp's privacy practices here.


2 Responses to “The war for Twaweza’s soul: the hunger for clarity and certainty v the demands of complexity”
  1. kieran

    Agree very strongly re your analysis of power. I worked in Tanzania for 6 years and alot of my work was focused on improving education system. My sense in that time was the education officials and politicians were very anti much of the work Rakesh was developing, and worth noting they often saw the man not the organisation. Orgs need charismatic leaders, but this leads to the action being personalised? Politicains seeing the NGO leader as ahving political ambitions and/or being mouthpiece for donors who fund them.. as one once said to me who elected him

    In addition, there are lots and lots of studies carried out in Tz and to the dismay of many activists they show people have a lot of faith in political leaders, and see politicains as people who will solve problems.

    Lastly much of this high proflie media work did create a buzz, set agendas but often in development circles and was often owned and driven by urban based, educated elites… so at field level it was just another message to ignore whilst you wait for politicians to sort your problems out.

    Want to emphasise I think Rakesh is a brilliant thinker but I sensed within Tz, the political power structures chose to ignore many of his messages and stigmatise the man. Perhaps one way to bring about change in Tz, is to get activists to join and work with and through the ruling CCM!

  2. Marc

    It seems to me good old fashioned social movement building still has a role in citizen activism. Twaweza seems to shy away from this. People in poverty need organisation for so many reasons to take more power in their lives.

    If it does not sound innovative enough I would argue that the meaningful innovations with big impact in our daily lives often come from combining a number of existing ‘technologies’ in new ways.

    In exploring how change happens, what about the ‘meme’ concept?

Leave a Reply

Your email address will not be published. Required fields are marked *