In this guest post, ODI’s Claire Melamed, Emma Samman and Laura Kiku Rodriguez-Takeuchi pitch for partners for some work on developing new wellbeing metrics in development. Any takers?
What do we think we’re doing when we do ‘development’? Surely, it has to be about making lives better for people as they themselves experience them. But we know surprisingly little about how poor people actually define ‘making lives better’. And when it comes to aid allocation, poor people are rarely asked about their priorities. Several current initiatives are seeking to tackle this gap, with Oxfam Scotland’s work on the HumanKind Index providing one recent example.
The drive for results and value for money provides an opportunity to focus on developing measures of development outcomes that help 
to identify which – of the many possible outcomes that aid spending can achieve – poor people value the most.
To be effective, any new metric needs to be easy to use for decision makers, so that it becomes an effective tool for allocating resources and planning interventions, and in their monitoring and evaluation. The challenge, and therefore the research question, is to establish whether we can devise a way of measuring outcomes, based on poor people’s own values, which strikes the right balance between a reasonable representation of reality and usability by policy makers.
ODI is currently developing a project that seeks to address this question, and exploring possibilities to work with a local partner to develop and conduct an in-depth pilot study.
Using health as a point of departure
Outside the development bubble, notably in health care, researchers have designed methods and measures that have gained substantial traction with policy makers, and which are widely applied. The measures sport deeply off-putting acronyms such as PROMs and QALYs, but a recent paper by Claire Melamed, Nancy Devlin and John Appleby reviews what can be learned from these methods and argues that importing and adapting health metrics could transform how we think about development outcomes.
In particular, approaches used in the UK health sector manage to reconcile the twin demands of complexity and simplicity. Instruments have been developed that allow the impact of interventions to be measured according to how much benefit they produce using a common outcome measure, where ‘benefit’ is defined on the basis of the values of the general public.
These values are calculated from surveys, in which many thousands of people have been asked questions designed to find out how they rank different health outcomes – such as freedom from pain, ability to move about normally and so on [see pic]. Their answers have allowed researchers to assign “weights” to each combination of different outcomes – numbers that can guide policy. These are now being used to measure outcomes and even, in some cases, to allocate resources.
Potential value of the approach
The key benefit of this approach is that it provides a standard metric to measure and compare the effectiveness of very different types of interventions: for an aid equivalent, the common denominator would be the impact on the lives of aid recipients as they perceive it.
With such data, decision-makers could readily compare the unit cost of achieving an equivalent gain in wellbeing across very different types of interventions, both across and within sectors. It would also provide an invaluable complement to more traditional methods used in the monitoring and evaluation of particular policies and programmes (e.g. Randomized Control Trials). By the conclusion of this project, we aim to produce an instrument that can measure wellbeing in general and within sectors such as health and agriculture, and that recommends investment priorities to policymakers on the basis of this accumulated information.
The pilot
At present we are planning a pilot that will aim to determine to what extent and how we could adapt the methodology used in the health sector to establish the preferences and values of deprived people in a credible and useful manner. Key methodological challenges include developing the relevant instruments, the comparability of the preferences of different people and how preferences change over time, and how best to elicit the relative values that individuals and communities place on aspects of their wellbeing. Finally we will address whether and how we might use the information we generate on preferences and weighting to devise a standard metric that can be compared within and across dimensions of wellbeing.
We are looking for partners for this pilot phase, and would love to hear from any groups who might potentially be interested in collaborating with us as we plan for a pilot study (please email us at e.samman[at]odi.org.uk).]]>