Take time to prepare evaluations!

Since September, there has been an extraordinary flurry of calls for proposals for external evaluations. This is good news; it suggests that people find it important to evaluate their work. But, upon closer examination, you'll notice that many calls for proposals envisage the evaluations to begin just a couple of weeks after the deadline for offers, and to end within a month or so. That is frustrating for busy consultants, who tend to be fully booked several months ahead. Narrow time-frames may also make it difficult for those who commission the evaluation to identify sufficiently skilled and experienced candidates. If you take evaluation seriously, then surely you want it to be done in the best possible way your resources allow? 

Over the years, I have come to appreciate time as a major element of evaluation quality.
Most development organisations (not only NGOs) cannot and do not want to afford full-fledged scientific-quality research, which typically involves plenty of academically trained people and several years of work. That is perfectly reasonable: if you need to make programme decisions on the basis of evaluations, you can't afford to wait for years. (The programme would be over, the context would have changed, your organisation would have changed their priorities, to quote but a few likely problems.) But what one can afford - even on a shoestring budget - is to allow plenty of time for thinking and discussing during the preparatory phase of an evaluation, to make sure (among other things):

• the TOR express exactly what you need
• the participating organisations are well-prepared and welcoming (which they are more likely to be if the TOR have been worked out with them and take their wishes into account)
• the evaluation team can think through different options, discuss these with key evaluation stakeholders, and let their thoughts mature over a few weeks before deciding on the final design
• there is enough time to select sites & samples so as to achieve a maximum of representativeness (in many evaluations I know, "purposive" sampling often means "opportunistic" sampling: the evaluator is directed to the most easily accessible places and meets "the usual suspects". Unless she gets a chance - and time - to negotiate different choices, that is ;-)
• initial data collection can happen "at a distance" (pre-testing of questionnaires, web-based or e-mail surveys)

All this can take place within the typical 5-15 preparation days, if they are spread over several weeks. So: make evaluation planning and preparation a formal part of project/ programme planning so as to be sure that  calls for proposals are published at least 3-6 months before the evaluation is supposed to start. You are likely to see a huge difference in the quality of the offers you get, and - more importantly - in the evaluation process and reports.