Took me a while to overcome my reluctance to read a document subtitled ‘Deliverable 1.1’ on the front page (yuk), but I’m glad I did so.
The paper ‘Review report on Beyond GDP indicators: categorisation, intentions and impacts’ (cracking title too…..) is published by BRAINPOoL – ‘Bringing Alternative Indicators into Policy’ (is that the sound of teeth grinding?). The authors come from Eurothinktanks CUEC (Czech Republic) and the new economics foundation (UK).
But enough of the snark, because the paper is really good. Rather than add yet another indicator to the smorgasbord of measures that go ‘beyond GDP’, it tries to understand why some of these indicators get picked up, while others sink without trace. Asking the right question, as Einstein said, is a lot more than half the battle.
The researchers sought to answer that question with a mix of desk research, interviews with indicator nerds, and media analysis. First they tried to categorize the ever-growing list of alternative indicators. They came up with:
- Level of impact – international, national, local
- Indicator domains – environmental, social, economic
- Indicator approaches – subjective, objective
- Indicator types – single indicator, set/dashboard, aggregated, composite
- Envisaged users – politicians/policy makers, public, experts
- Link to GDP – adjusting GDP, “replacing” GDP, supplementing GDP
Media analysis showed that single figure indicators get far more pick up than complex dashboards, as do those that enshrine ‘simple and meaningful concepts’ like the Human Development Index or Ecological Footprint, which lead the pack by some way (see chart).
But a media splash, while useful, is not the main purpose of the indicators studied, which was overwhelmingly to influence policy. When have they achieved it and why?
- The most basic influences noted were transmission to or reference by a decision maker.
- The next step up from this is use of Beyond GDP indicators in assessment. Examples of this include GPI, ISEW and QUARS being integrated into regional or local official assessment frameworks in the US, UK and Italy. Also noteworthy is the Ecological Footprint being used to set environmental impact targets in several national governments.
- Actual policy changes connected to indicators were also reported. These include (i) the implementation of schemes to improve habitat for farmland birds after changes to the UK Defra Sustainable Development Indicator set (ii) the investment of $15 billion by the UAE government to develop alternative energy sources spurred on by a poor ranking in the Ecological Footprint.
- The most prolific successes, however, were achieved by local level indicators, perhaps because the distance between producer and user of the indicators is much smaller, making it easier to achieve a better ‘fit’ while also achieving legitimacy and relevance. For example, the Jacksonville Community Council Indicators are believed to have been instrumental in reducing infant mortality rates, recidivism and water pollution in the area. The Council of Europe SPIRAL project, meanwhile, achieved change to the delivery of food relief in France and the transformation of the approach to homelessness in Belgium.
And what lessons did the authors draw from all this?
“Indicators were successful when they had real relevance for policy makers. Crucially they need to measure something that policy makers believe they can influence. Subjective well-being indicators when used incorrectly can appear to fail this test, which explains efforts to build the evidence base for how policy can influence wellbeing. Another factor here is cost. In the current climate, indicators that provide clues for low cost policies, or indeed those that can help save money, are of particular interest.
Salience for a broader audience is also crucial and entails the elements of simplicity, understandability and good communication. Initiatives are effective when they allow one to produce a simple and attractive message that relates a meaningful concept. Using communications experts and avoiding taboo words were also identified as being important.
Indicators need credibility and legitimacy. Aside from the requirement of quality data, the appearance of neutrality was seen as the best route to achieve this with some interviewees comparing advocacy organisations’ data unfavourably with that of National Statistical Offices.
Developing the indicators with the audiences at whom they are targeted and/or encouraging participation (in the way that the Jacksonville Community Council Indicators initiative has done) is also seen as a key success factor. The importance of relationship-building also applies to policy makers with most initiatives that had achieved policy success citing direct face-to-face channels as vital.
Several barriers to the success of Beyond GDP indicators have also been identified. The economic crisis was viewed by many interviewees as a challenge for this agenda as it has moved the policy focus. Ideology and vested interests are also noted barriers with subjective well-being and composite indicators receiving strong resistance from those with libertarian or right of centre political views.”
Not sure I buy that last point – isn’t a crisis just as likely to create opportunities for adoption of new indicators, especially if the failings of existing indicators are seen as partly responsible for the crisis? I remember being struck by a presentation from the South Korean statistics office (I really do have all the fun), arguing that Korea was adopting a new quality of life index, based on over 100 objective and subjective indicators of wellbeing, because GDP was incapable of explaining why an ‘economic miracle’ like Korea had the highest suicide rate and the lowest birth rate in the OECD. Failure is a great source of innovation.
The Brainpool project continues and is worth following (especially if they ditch the naff titles). Oxfam is getting involved, working with nef in Brazil and India to explore likely reactions by politicians and voters to possible indicators. It will build on Oxfam Scotland’s Humankind Index, which was an earlier foray into this field. I hope the research includes applying a more serious power analysis to the findings in this report: what fractions of the state, private sector, political class or popular movements adopt new indicators, either individually, or in coalition with others? When and why? That could get very interesting.