In this holiday season I visited my sister, who has been passionate about gardening. I brought a beautifully illustrated book for her, about pear orchards in Prussia. Prussia, a belligerent kingdom that ceased to exist in 1918, was mainly known for its military dominance in the region, and for an obsession about order and discipline. So I was hardly surprised to find, in that book, a table showing drawings of differently shaped pears, arranged in neat rows and columns. The roundest pears were displayed near the top left corner, the thinnest, longest ones near the bottom right, with dozens of intermediary states in-between. Every pear came with a drawing of its appearance, as well as a transversal cut, which was criss-crossed by lines and dots dividing it into neat circles and measurements. "Look," I exclaimed, "the Prussians developed a system to classify pears!" My sister took a quick glance and responded, somewhat bitterly, "according to size, of course".
Arguably, size was the only aspect of pears that could be reliably and replicably measured in those days. But what does its size say about a pear? It doesn't tell me whether the fruit is ripe or green, hard or soft, sweet or bitter, juicy or dry.
The Prussians were great simplifiers. They were (in)famous for organising forestry in a way that measured effectiveness only in terms of its marketable yield. Basically, a forest was an amount of timber, nothing else. In "Seeing Like a State - How Certain Schemes to Improve the Human Condition have Failed", 1998, James C.Scott explains how that reductionist approach produced those sad monocultures of same-age trees in rank and file which have disfigured German landscapes ever since. The model was adopted throughout the world, a disaster for those who had used the many fruits and animals of rich, diverse forests. Rather than learning to produce maps and measurements faithful to the phenomena they found in nature, the Prussians and their followers aligned reality to their systems of measurement, cutting down forests to turn them into easily manageable and measurable plantations.
You have guessed why these musings appear on a blog on development and evaluation. Counting things can be useful, and often it is necessary. If you sell timber, you want to know how many cubic metres you can get out of those trees. But if you're into human development you need to know when quantity ceases to be the only important dimension, and where you have to look for quality to understand a phenomenon. If you do your qualitative homework well, you might even find ways to accurately describe the characteristics you are looking for, and turn those aspects of quality into scales that allows you to put a number to them.
It is a risky idea to try and bend reality until it can be counted. (Please, no chopping down of forests just to count the trees.)
It is a good idea to think about monitoring and evaluation as soon as you start designing a programme. But let's resist the temptation to build only programmes that are geared to produce the kind of change that can easily be measured. Following that temptation would generate a very barren development landscape.
