Good things happen in the short term and bad things happen in the long term

This is a long title but I love that sentence, culled from Elliot Stern’s intervention on the Benefits and Barriers to Evaluation Use at the recent evaluation conference in Paris. The one-day conference, convened jointly by the European Evaluation Society, France’s evaluation society, the United Nations Organisation for Education, Science and Culture (UNESCO) and the Organisation for Economic Cooperation and Development (OECD), took place at the quite extraordinary UNESCO headquarters in Paris.

There, you stand in an average foyer, sipping your average coffee break coffee, and all of a sudden you realise you are gazing at Giacometti’s Walking Man. Then you turn around and raise your eyes to see Picasso’s giant mural Fall of Icare, a few tables and chairs scattered at its feet. (Both are important works of European 20^th century art, for those who are not into that stuff.)

The title of the conference was Making Effective Use of Evaluations in an Increasingly Complex World. I wonder whether the real world has ever been simple. In any case, I have always known it as a complex place. Complexity means that all kinds of things happen; there are many causes and many effects, reinforcing or weakening each other in many ways, and the effects of what we do (or refrain from doing) are unpredictable and often long term.

That is quite different from the simpler, fictitious world of results-based management (RBM), practised in development agencies as a way to (mentally) arrange activities, their direct products and hoped-for outcomes in neat pyramids or chains leading to the desired goal. In that world, activities produce planned outputs, outputs lead to outcomes and in the end impact happens. The time horizon of the logical framework rarely extends beyond the project it describes. Such linear logical chains come in handy when you present a project or programme to others. That is why they are quite mandatory in applications for funding from United Nations organisations, the European Commission and other development funders.

How much sense does that kind of thinking make for development interventions that are increasingly trying to change institutional and national behaviour, people’s assumptions and social norms? As pointed out in Stern’s speech, those kinds of changes happen as a result of a confusing array of factors; they are slow and difficult to measure. The intervention you devise might be good in the short term, but contribute to bad, sometimes unexpected changes in the longer term. If, for instance, the goal is to end gender-based violence (GBV) in a place where no-one even talks about the issue, what kind of result can one reasonably expect within three years? I can think of a couple (like, getting a discussion on GBV going), but one will only come up with reasonable ideas if one thinks way beyond the logical framework of your project, examining the different factors likely to support or block our cause – during the lifetime of the project and in the longer run.

The mismatch between the results chain logic and real life tempts people in development projects to forget about logical frameworks once the project is approved. That doesn't mean people are bad at monitoring, evaluation and learning. For example, human rights activists I have worked with constantly monitor national policies, jurisprudence, police (non-) action, the media and so forth, and reflect on what that means for their work. That is, they keep watching people and things that are not part of the project but that massively influence the human rights situation. Arguably, those aspects are more important for human rights in their country than the chain of events foreseen in its logical framework. Of course, on top of all that they’d better check, every so often, whether the activists’ own activities do contribute to the expected results. But it would be unreasonable to expect them to devote all their monitoring and learning efforts to that small fraction of the wider change they wish to contribute to.

I would advocate for project/programme planning and monitoring that looks beyond the confines of simple results chains or results frameworks. Summarising project activities and aspirations in an RBM-style tree diagram or matrix is fine. But to plan and manage a project wisely, it is important to keep an eye on people and actions around you. Questioning the assumptions behind the causal connections in your log frame, mapping actors and assessing risks are relatively simple, useful things one should do, both when planning a project and when assessing its progress.