Evaluating efficiency in campaigning

The OECD Development Co-operation Directorate (DAC) defines efficiency as follows: "Efficiency measures the outputs -- qualitative and quantitative -- in relation to the inputs. It is an economic term which signifies that the aid uses the least costly resources possible in order to achieve the desired results. This generally requires comparing alternative approaches to achieving the same outputs, to see whether the most efficient process has been adopted. When evaluating the efficiency of a programme or a project, it is useful to consider the following questions: Were activities cost-efficient? Were objectives achieved on time? Was the programme or project implemented in the most efficient way compared to alternatives?"  (DAC criteria for evaluating development assistance)

All this is nice and clear if you evaluate a project that produces tangible outputs, through predictable processes - as in manufacturing, for example. Things tend to be more complex in social development.

Earlier this year, I conducted an evaluation of an innovative, multi-country campaign which aimed to create a popular movement and a shift in  social norms regarding gender equality. How do you apply the DAC definition to such a broad initiative, with such diffuse outcomes? What are the inputs, what are the outputs? You would only cause confusion if, say, you decided to call the campaign activities "inputs" and the social movement its "output". I have seen overly simplistic solutions - of the type, "our agency has spent 200,000 US Dollars on TV announcements which have reached an audience of 2 million people; i.e. at 10 cent per person, which is highly efficient." But what if, in this fictitious example, the viewers totally misunderstood the campaign message, or felt offended by it? Is it efficient to spend 200,000 Dollars on anything that doesn't work?

A few simple moves may help to generate valuable insights on efficiency even in complex settings: 

1. The DAC criteria (relevance, effectiveness, efficiency, impact and sustainability) are interdependent. Hence, you cannot look at efficiency in isolation. An activity that is not likely to be useful and that doesn't work is wasteful. Hence, to put it sloppily, start by finding out whether the campaign you examine makes any sense - if it doesn't, then efficiency may not matter anyway! 
2. Efficiency is mainly about the appropriate use of resources. Don't get mesmerised by concepts such as "input" and "output"; squeezing complex realities into ill-fitting concepts only confuses readers. Rather, think of the different types of resources that come into play, for example: people's work time, knowledge and contacts; money; any material assets such as cars and office equipment. Are these resources used appropriately and wisely? "Quality" matters as much as "quantity".
3. Arguably, the discipline specialised in effective use of resources is management.  You can examine decision-making structures and processes on resources against pre-determined standards, for example when conducting a financial audit. Or you can find out from the different people who work with the resources (staff, volunteers, "clients") whether they are satisfied with the resources and the ways in which they are used.
4. Effective monitoring - purpose-driven observation of what is happening within the campaign and around it - is a key aspect of management. "Efficient" in this context means that the resources used for  monitoring (including people's patience) are commensurate to the use of the information thus gathered. 

The aspects outlined here are mainly about quality. You can't measure campaign effectiveness by  inserting a thermometer somewhere (the strictly "quantitative" approach). You need to find out how the resources are used, and compare that with existing standards or the participants' aspirations. I'd be keen to have more tips on the subject!