Graham Teskey

In praise of…. Logframes

Guest post from Graham Teskey

My friend and colleague Lavinia Tyrell recently posted a note on LinkedIn, highlighting a recent WB Independent Evaluation Group report, which reflected on various methods of monitoring and evaluation currently used in development. In so doing, Lavinia referenced this diagram:

As a fan of diagrams, as well as a long-time user of both the London Underground and the Washington Metro, I looked at it closely. There are a number of stations on the map that I have never been to and probably never will. ‘Regression Discontinuity’ seems to be far too deep in the East End, while ‘Empowerment Evaluation’ sounds very Sloane Square or Friendship Heights. I may take a trip to QCA once the line is built, even if only to find out what the abbreviation stands for. But without a doubt my favourite stop has to be on the Northern Line – the yellow one – ‘Logical Frameworks’ (logframes).

In all the jobs I have held, the only training that has ever stayed with me was a three-day course on logframes, held in a very pleasant beach hotel on Fiji’s beautiful coral coast. This was a few months after I joined what was then the Overseas Development Administration, DFID’s forerunner. Three days on logframes. Yes really. Our Pacific team were gathered together to learn this new skill. The course was designed not only to help us think rigorously about how change happens, but also to ensure that we had a shared understanding of what constitutes good design.

Twenty-five years later I am aware that logframes are out of fashion: references to the ‘tyranny of the logframe’ abound, its vertical determinism, and its lack of flexibility. Donors seem to have replaced logframes with things called ‘Program Logics’, as in the schematic below. You don’t have to be able to read it to get the picture… (and it’s logframes that are supposed to be too vertical…?).

I would argue that program logics like this constitute a retrograde step. My main criticism is that, well, they are just not logical. Program logics contain no explanation of why outcomes at higher ‘levels’ will be forthcoming. The above diagram simply says, “If we successfully deliver the outputs, then we will deliver the outcomes”.  Sometimes a bunch of heroic assumptions may be listed somewhere in the diagram but with no interrogation regarding their political feasibility. This is a theory of action, not a theory of change. A ‘Theory of Change’ (ToC) refers to an informed assessment of why some particular set of activities we think will lead to some higher-level development goal. Why do we judge that relevant actors will make the changes assumed within the program?  Simply giving actors the formal right or skill to do things differently is not the same as those actors using those rights or skills in practice.  We call it a ‘theory’ because it is both explanatory (why things are as they are and not like something else) and predictive, generating predictions that are capable of proof or falsification.  

The basic requirement of any ToC is that it must contain an “if, then, because” statement. For example, “If we deliver effective and valued training to staff in organisation X (the Activity), then we will deliver a certain number of staff with new skills and competencies (the Output). If these staffs subsequently apply their skills effectively and consistently in the workplace (the Intermediate Outcome), then we will improve organisational performance (the end of project outcome or EoPO). We judge this to be the case because (i) the senior management of the organisation has introduced a rigorous performance management system; (ii) the organisation is under intense public pressure to perform; and (iii) and poorly performing staff have recently been dismissed and a cohort of capable graduates has been recruited”. These ‘because’ statements are absent from program logic diagrams. There is no consideration of whether the changes are politically likely, organisationally feasible, or socially possible. This is the discipline that the logframe demands.

I would go as far as to suggest that a good logframe will tell you most of what you need to know about any initiative: its purpose, its ToC, how to measure progress and success, the sources of the data, and how the because statements will be tested. There are six basic rules of logframes:

  • There should be one goal and one goal only. If there are two goals, then at best we run the risk of inconsistency of effort as priorities and context change. At worst we run the risk of having two separate projects.
  • There can be no double hierarchy.  This is a common error in many program logics: i.e., ”‘We will do A in order to achieve B”. Sorry, not allowed. If you want to do A, then fine, do it. But B must appear at the next level up. That is what ‘logic’ means. Recently I reviewed a design which had, at the EoPO level: “Municipal health services have the incentives, capability, and systems to enable improved delivery of the essential services package’. This is a classic double hierarchy, one thing (incentives, capability, and systems) to enable another (improved service delivery).
  • There can be no Outcomes without Activities to deliver them. Outcomes do not appear from nowhere – they require planning and delivery.
  • Outputs do not need indicators. An Output is just that – a thing in and of itself and which is self-defining. It is either delivered as a direct result of the Activity or it isn’t.
  • The Goal (or impact statement) must be tracked. I am dismayed that there seems to be a trend to leaving the Goal statement ‘unmeasured’. While project monitoring and learning efforts should be focused at the Output – Intermediate Outcome -EoPO level and their inter-relationships, the Goal must be tracked. Otherwise how will we judge success? Tracking requires identifying measures at the design stage, which are publicly available, will be collected consistently, and will indicate investment impact.
  • While the Goal in most circumstances will remain constant, Activities and Outputs can change through the life of the project. No logframe is ever set in stone. I do not understand why some claim that they are.

The arrows drawn on the schematic below show how to read a logframe. It does not proceed vertically. On the contrary, it is to be read in a zig-zag fashion: “if we successfully deliver Outputs, then we will deliver Outcomes because we judge the change is both technically desirable and politically feasible (this is our ToC). The column on the extreme right hand of the logframe is the probably the most important. We judge this investment has a good chance of being delivered because…. And these ‘because’ statements must be interrogated as rigorously and frequently as are the ‘deliverables’ in the narrative summary column. I accept that there may be as many lousy logframes as there are lousy program logics – usually due to the fact that the right hand column is labelled ‘assumptions’ and they are merely listed and then forgotten. A good logframe will summarise why the results chain has a chance of actually coming about. Without that the logframe becomes just another way of presenting a program logic.

So a plea to everyone waiting at Evaluation Station Central. Please take the next Northern Line train to Logical Frameworks. Mind the gap as you alight there. Indeed, by alighting at this station you will fill many of the gaps in the current obsession with program logics.

Subscribe to our Newsletter

You can unsubscribe at any time by clicking the link in the footer of our emails. For information about our privacy practices, please see our .

We use MailChimp as our marketing platform. By subscribing, you acknowledge that your information will be transferred to MailChimp for processing. Learn more about MailChimp's privacy practices here.

Comments

10 Responses to “In praise of…. Logframes”
  1. For me Logframes are sometimes ok, though often overly elaborate and tedious, when managing an event or small project, whether isolated or even part of a larger unfolding process of development – so, fine for event management, but less so for navigating complex processes of social change. The ToC behind Logframes is about simple cause and effect in contexts that are not complex, but at most complicated. I have seen practitioners use Logframes artfully, though mostly for their donor’s sake, but within a much more emergent and intelligent understanding of change that was far from linear. The other problem with logframes is a cultural one. I remember conducting an evaluation of a “village shop” programme in Namibia many years ago, to provide market access for inputs and outputs, which failed in 11 of the 12 villages. In the one village they had built this impressive looking shop and in the meeting, under a tree, I asked if I could have a look inside. The community leader, with a most embarrassed look, asked a young boy to take me there, not wanting to come with me himself. The door was unlocked to reveal a spotlessly clean building, but unused and empty. I looked behind the door and there across the wall was the most impressive looking logframe I have ever seen.

  2. Small (pedantic) correction 🙂 Traditionally a LogFrame should be read as a sequence of “If…And..Then” statements. The “And…” is the content of the assumptions column on the right. Relabelling that text as “Because…” risks hiding the high likelihood that this text does involve assumptions – and if so they need to be clearly labelled as such

    If the Assumptions column is not used properly (i.e is filled with ritualistic content) then the LogFrame will have some of the same weaknesses as diagrammatic ToC i.e the causal mechanisms linking events will remain opaque.

    BTW, here is what I think is a good use of diagram ToC that does explain the connecting causal mechanisms. It can be seen on the DCED website here: https://www.enterprise-development.org/what-works-and-why/evidence-framework/ Try clicking on any of the links between events in the diagram- they will take you to a new web page detailing what that link is all about

  3. I would go beyond “If…And…Then…” to suggest that the shape of the narrative (because very few real world problems lend themsleves to the two dimensional world of blocks and arrows – that’s a tyranny which I would love to see die) ought to be:

    “In a context where…” – describe the factors which govern the environment including those which define our action – for example policy change at home;
    “If we…” – describe what we propose to do;
    “And…” – a bunch of RELEVANT assumptions which have to hold good and which are beyond our control (because if we controlled them they wouldn’t be assumptions) – but see below about “Because”;
    “Then…” – what our current understanding and beliefs tell us could now change.
    “And then…” – some higher order change which might in turn now be less impossible to imagine.
    Personally, I don’t have a problem with the “Because…” statement – but I agree that it needs to focus on the assumptions. So I tend to use “In a context where…If we…Then…And then…Because…” which lends itself well to being turned into a narrative (which normally delights senior decision makers), and which lends itself relatively well to being captured in a log frame.
    But the beauty of every ToC I do – starting with context, describing desired change, setting out how incentives might be shifted, showing what could now happen – is normally spoiled by a demand for more blocks and arrows. Sigh.

    JAB

  4. Ed Bell

    Wonderfully written post, Graham. I agree, in particular, with the emphasis on testing assumptions.
    I can’t praise logframes in the same way as you as I tend to find that they are just programme logics put into a table with some indicators and milestones. Highly mechanical and often confusing outputs and outcomes and/or activities, inputs and outputs.
    I think the most important process is to set out what effecting change will look like over 6 months, a year and lifetime of a programme (or the effort) and review it regularly. I’ve found Julian King’s work for OPM really useful for this approach.
    Personally, I’m happy to call that a Results Framework. Whatever it is called, it needs to be anchored in the Theory of Change.

  5. Don McPhee

    I agree that it is unfortunate that LogFrames have gone out of fashion and that they are often very valuable. They are valuable when prepared together by a group of key stakeholders and help the group to agree on on the logic of their objectives, their key indicators, their MOV and the critical assumptions and risks. They are not valuable when prepared by one person, especially when that one person is an external consultant. Also, to be truly valuable they need to guide the key stakeholders during project implementation and be revisited at key milestones and be an important input for monitoring and evaluation.

  6. David Booth

    Yet another timely, and timelessly relevant, contribution from Graham! His complaint that the ‘because’ element the ‘if .., then .., because’ structure of Logframe thinking has fallen into disuse in much current practice is surely justified. However, this is not entirely a matter of the move away from Logframes. The same deficiency is found in the practical application of the various alternative programming tools, where ‘theory of change’ becomes a flow diagram with almost no ‘theory’ about the causal mechanisms. Several high-profile DFID guideline documents were guilty of this. The issue of complexity that has provided one of the reasons for the move away from Logframes, and is also a challenge to theory-based programming/evaluation, is not so easily disposed of. However, the objection here is not to the ‘if …, then …, because’ way of thinking; it is to treating of the analysis as anything other than a highly provisional hypothesis, to be tested and likely revised or abandoned and replaced with a better version in the light of experience over the relatively short term. (I would distinguish the approach of testing and regularly revising ‘mini-ToCs’ in this way from tinkering with a largely fixed Logframe to make it fit the facts, which in some programmes is considered sufficiently adaptive practice.)

  7. David Grocott

    Thanks Graham – great blog.
    At heart, logframes are a way of measuring change over time, and no-one has yet proposed a better alternative, despise the proliferation of ‘stations’. The biggest problem logframes face, is that they are routinely poorly used.
    For ‘complex’ programmes, I propose using a logframe format (incl. indicators) but doing away with the targets, so that reviews can focus on important questions like, ‘what happened’ and ‘what should be change’, rather than ‘did we meet our arbitrary pre-agreed targets’.
    On the question of If…And…Then…Because, my preference is for logframes/ToCs to reflect on both the ‘And’ (i.e. the assumption) and the ‘Because’ (i.e. the evidence to support the assumption).

  8. Julian Hamilton-Peach

    These are just simple aids to thinking and doing. What matters is how well we use the aids. The aids are most effective when used by people that care about the change. The failure of most development is caused by not caring much about the change, but caring more about pleasing the funder, contractor, peer group or self. And the root of that failure is ‘getting away with it’ for year after year. Evaluations do not have much influence to improve development performance because they do not hold people personally accountable.

  9. Katherine Bain

    Great piece and what a passionate plea for logframes! And while I am sympathetic to the plea, and even the theory behind logframes, in practice I worry that they have become the last stop on the map and, generally, there is only one way to get there. If they were used to continually revisit assumptions, based on practice, revising theories to improve effective implementation, with a number of feedback loops before arriving at the first stop on the yellow line that would be one thing. But, instead in practice they seem to have become the stop that you must get to once the train has left the station and there is little room deviation from that route. This may be more because of the institutional culture of compliance and control with which they are used but, as long as that continues to dominate, we need to provide tools that support fewer one way trips to pre-determined destinations!

Leave a Reply

Your email address will not be published. Required fields are marked *