Derek Thorne

A TripAdvisor in development? Turns out it’s a poster on the wall…

Guest post from Derek Thorne

Back in 2015, Duncan Green published a piece on FP2P asking whether a TripAdvisor-style feedback system could work in development. If you follow the link, you’ll see it generated a lot of feedback!

The idea was – and is – that TripAdvisor, and systems like it, have put significant power in the hands of consumers, allowing them to publicly post feedback on hotels and restaurants without waiting to be asked, thus creating pressure for services to improve. Could a tool or approach like this achieve something similar in development and social change work?

Six years on, I’d like to have a go at answering that question. That’s because my organisation, Integrity Action, has been working with partners for some years now to set up feedback systems that have some things in common with the TripAdvisor model.

Let’s be clear about one thing first though: transferring the TripAdvisor model directly into development or humanitarian programming wouldn’t work. And why on earth should it? It is a tool built for a commercial context that doesn’t have a great deal in common with the range of contexts in which development work is carried out. But it is interesting to think about the “essence” of TripAdvisor – the principles that underpin it. Here I will pull out two of them and look at how they have been applied, both in our work and that of other organisations.

Open feedback – but open for whom?

The concept of openly published feedback seems attractive. While submitting feedback in private may feel like tossing it into a black hole, posting it in public means it can’t be hidden and can’t be ignored.

Not so fast. With TripAdvisor, hotels and restaurants exist in an open market, and consumers – who have easy access to that platform – can choose where to take their custom based on the ratings they see. That provides a strong incentive for the “supply side” (hotels, restaurants etc.) to act on the feedback of the “demand side”.

In the development and social change context, where citizens rarely get to choose between different project or service providers, the incentives need to work differently. The obvious route is via donors, who could think twice about funding an organisation if they see poor feedback. It might work – but (1) I’m not convinced donors would consistently care about this, or see it as reliable, (2) uniformly positive feedback is itself problematic, as we shall see, and (3) isn’t it rather disempowering for citizens to rely on donors to exert the pressure that’s needed?

However, it turns out the incentives can work more directly than this. It’s just a matter of making sure open feedback is visible to the right people.

Let’s take a pleasingly offline example. As part of Integrity Action’s SHINE project, secondary school students in Nepal monitored their own schools, “reviewing” things like teacher attendance and sanitation. Instead of using a mobile app or digital tool to report what they found (which, in any case, they wouldn’t be allowed to use in school), they used a simple poster (see pic). It appears as a grid, with a traffic-light rating for each of 12 service delivery areas recorded on a monthly basis.

Feedback in Nepal. Credit: Integrity Action

By being displayed on the wall at school, and shared with local officials, this feedback was made open to all the right people. We saw it being used by student “Integrity Clubs” to figure out what issues to bring to regular meetings with school management; it also allows parents, local officials, or other interested parties to know what’s happening and potentially apply pressure.

There are also some digital tools making open feedback work in the development space. Kujakuja displays simple aggregated feedback from community surveys. Loop is a new tool, capturing feedback from multiple channels (including SMS) and applying safeguards before displaying comments online. mySociety has a set of open source tools, including FixMyStreet, which has been applied in Uruguay, Uganda and elsewhere.  And Integrity Action has developed a tool called DevelopmentCheck, which citizens have used to monitor over $1 billion worth of development spending. However in all cases, there is an acknowledgement that simply sharing feedback online isn’t going to produce the right incentives in places with low internet penetration, so other means may have to be used to disseminate it.

Can the principle of open feedback be applied more generally in how organisations implement development projects? This was one of the questions asked within a recent “feedback pilot”, in which Integrity Action worked with four UK Aid Direct grantees to enhance their projects’ feedback approaches. We learned some useful lessons from this, including the importance of thinking through who is going to use the data and how, and that it’s easier to make quantitative feedback open, rather than qualitative comments. Read more in our learning paper.

Independent feedback – but what about power dynamics?

Another important feature of TripAdvisor is that – unlike a development agency’s feedback system – it is independent of the services being reviewed. This is meant to make it more trusted. It’s also meant to add “strength” to the feedback, because no matter how bad people’s reviews are, the service in question can’t just throw away the feedback or turn off the system altogether.

Many of the tools and approaches mentioned above take this approach – including the poster. However, at least from Integrity Action’s standpoint, the primary reason is not to make the feedback more trusted by people who would look at it. It’s to build trust with the people who give feedback in the first place. That’s because development has a problem which TripAdvisor doesn’t: a steep difference in power between those providing the services and those making use of them.

This gives rise to what is sometimes referred to as “courtesy bias”, where people are more likely to give positive feedback because they worry that negative feedback may offend the service provider and might even result in the service being withdrawn. It prevents serious concerns and safeguarding issues being raised too.

What do we want? Negative feedback!

This raises what might be the most important difference between TripAdvisor and any equivalent approach in development. Above all else, hotels and restaurants on TripAdvisor want five stars and raving comments. That’s what helps elevate their business on that platform.

Development projects, on the other hand, don’t want that – or they shouldn’t. They need to know where they are going wrong, how things could be improved, how people’s conditions are changing, and much else besides. Feedback is primarily for learning, not for recommendation.

This explains why an independent feedback platform and open publishing go well together. If the service provider is gathering feedback and publishing it openly, it provides a disincentive to look for the negative feedback that will really add value to the project and drive improvements. An independent platform doesn’t have this issue – though it could face external pressure to massage the results it displays, and “fake feedback” could become a problem too. (Don’t forget Campbell’s Law: the more important an indicator is, “the more subject it will be to corruption pressures”.)

A citizen using the DevelopmentCheck app to report on a construction project near where she lives in Kwale, Kenya. Credit: Integrity Action

This tension, between sharing feedback and encouraging negative feedback, can be addressed in another way: by using feedback metrics that don’t exactly measure quality, but rather responsiveness. A few years ago Integrity Action started using a “Fix Rate” metric, which tracks problems with a project (as identified by citizens) and solutions to those problems. The Fix Rate is the percentage of problems that have been solved to the satisfaction of the citizens concerned. So, to get a high Fix Rate, it pays to get feedback that identifies problems – and then to solve them.

A final point: I accept it’s pretty Northern-centric to take a tool that is primarily used in high-income contexts, and ask “well, how can we apply it in low-income contexts? Surely this will provide benefits!” What I’ve tried to show here is that it’s not as simple as that – but that it does provide an interesting prompt to explore some interesting features of feedback. But there are now so many interesting feedback and accountability approaches originating in the South that it might even be time to ask: what can other parts of the world learn from an Eyes and Ears, a HealthTools, or a Checkmyschool? Now there’s an article I would love to read.

Subscribe to our Newsletter

You can unsubscribe at any time by clicking the link in the footer of our emails. For information about our privacy practices, please see our .

We use MailChimp as our marketing platform. By subscribing, you acknowledge that your information will be transferred to MailChimp for processing. Learn more about MailChimp's privacy practices here.

Comments

4 Responses to “A TripAdvisor in development? Turns out it’s a poster on the wall…”
  1. George Graham, Humanity & Inclusion

    Thanks Derek. It’s great to see this work underway. The aid sector clearly needs shaking up if it’s to be as relevant, effective and legitimate as it aspires to be. I’d like to think transforming our accountability to affected populations could (still…) be one of the most powerful and helpful ways of doing this.

  2. Masood Ul Mulk

    In communities one encounters a phenomenon called the “noble lie,” where an entire community may tailor the feedback in a way that benefits them. All outsiders bringing in resources and are treated as a “Resource,”. Alternative facts and post truth are not a recent phenomenon in these societies. Communities regularly resort to them. Where does that leave a feedback mechanisms.
    Feedback and responsiveness cannot be isolated within development organisations. They must go together. Communities soon learn that large public sector organisations lack the ability to respond to their feedback and soon give up on them. Their lack of performance draws no criticism because the communities understand that voice would not influence them. NGOs or other development organizations with strong commitment and ability to respond and a track record of responding to them , face intense feedback and demands from communities. The smallest blemish is blown out of proportion. This is sad because these organisations already have a history of responding promptly to needs and problems. Feedback in such instances takes the form of a harassment. Your opponent puts complaints into the feedback system where it takes months for facts to be resolved and truth to be discovered. By that time the damage has been done.
    If the problem was lack of feedback only and not an issue of wider political economy this problem was a very simple one. In that case the electorate in Pakistan would not be electing public contractors to the parliament but would be electing teachers to the parliament like in Germany.

Leave a Reply

Your email address will not be published. Required fields are marked *