Getting those indicators right

In the world I work in – social development – the terms "theory of change" and "indicator" have become essential. You can't just try doing something that might improve people's lives. You need to explain why and how the things you (plan to) do will contribute to the improvements you want to happen, i.e. you need to develop a theory of change. A common way to visualise a theory of change is the logical framework, which ideally shows how interconnected activities and their immediate products (“outputs”) are supposed to contribute to further-reaching changes, i.e. goals, objectives, outcomes, impact and so forth.

Your theory of change can be more or less abstract, more or less detailed and restrictive,
more or less adaptable. But nowadays in international social development work, the standard is that you need to explain (i) what change/improvements in the situation you want to achieve and (ii) how you intend to make sure it happens. A whole vocabulary comes with theories of change and results-based management. People talk about the logic leading from "outputs" to "outcomes", or from "objectives" to "impact", or more complicated (matrices of) cause-to-effect-chains combining all those terms and more. 

The term “indicator” has become so common that I start to hear strange things like:

“We’ve got to reach our indicators.”

“They won’t obtain their indicators if they continue like that.”

“Their indicators are not sustainable.”

Eh?

An indicator is something that you use to measure a phenomenon. For example, if you want to assess a person’s weight, you can ask her to stand on a scale, which measures how many kilos the person weighs. To determine whether anything changes (or not), you take a measurement several times, at intervals that allow the change to happen. 

For instance, to assess whether a child grows, you can measure her height every year. Or, if I have a flu and I want to know whether I get better, I measure my body temperature every few hours. If a farmer wants to know whether she earns enough income from her crops, she counts the cash she has received for the crops and subtracts the costs she has incurred. A person’s weight and height, a person’s body temperature and the cash a person has at the end of an economic operation are indicators.

It doesn’t matter whether my fever drops or not – I will always have a body temperature. That is, there will always be something that you can measure against that indicator. That is, it makes no sense to talk about “reaching indicators”. What one can strive to produce are results, outcomes, outputs. But you will always “reach” an indicator, regardless of your weight, temperature or income. The indicator is just the concept you use for measurement. In most cases (at least when we talk about social development), the indicator represents just a tiny part - a clue - of the improvements in people's lives you want to obtain.

Apart from the sloppy use of precise vocabulary, trying to “reach indicators” carries a real risk: It can divert attention from the goal that needs reaching or the results that are expected from an activity. For example, if I have a fever, I can submerge my body in crushed ice for half an hour and then measure my temperature in an armpit: Most likely, the thermometer will show a lower reading even though my cold might turn into a deadly pneumonia as a result of the procedure. I would have made progress against the indicator, but I would have sabotaged the aim to cure my cold.

Indicators do not deserve to be treated as goals. Every so often, the development world learns the lesson the hard way. For instance, in the early years of "Education for all" campaigning, success ways measured by calculating the percentage of school age children who were enrolled at primary schools. Obtaining "good indicators" - or rather, high achievement rates measured against the indicator of school enrolment - is easy: You make sure everyone gets their sons and daughters registered at school once a year. Surprise, surprise: not all of them stay at school, some - sometimes many, and especially girls - drop out even before they complete the first couple of school years. Getting registered doesn't give you an education. You need to stay at school. Nowadays, those interested in progress in education measure both the enrolment rate and the retention rate, i.e. the percentage of children who complete full primary education.

The retention rate is still a very weak indicator for effective education. Children whose educational attainment you try to measure might have been subjected to unsanitary conditions in overcrowded classrooms, violent teaching, sexual abuse and fake exams, while completing all years of primary education. Some of them might have been better off staying at home. So, to get a full picture of educational attainment, you need to look into the quality of teaching as well. Still, it is useful to measure enrolment and retention rates.

Indicators will never capture the entire goal that you strive to reach. They measure aspects that suggest you’re doing the right thing, or that the change you want to contribute to is actually happening. (Btw. don't forget to remember that the desired change might happen in total independence from or despite the things you are doing.)

This is a plea for a bit of rigour when using the term "indicator". Misusing words risks emptying them of their meaning. In social development, misuse of that term might even sabotage the purpose which you have introduced the term for - more effective social development work.