Less is more - also in evaluation questions

Writing evaluation terms of reference (TOR) - that is, the document that tells the evaluators what they are supposed to find out - is not a simple exercise. Arguably, the hardest part are the evaluation questions. That section of evaluation TOR tends to grow longer and longer. This is a problem because: Abundant detailed evaluations questions may lock the evaluator into the perspective of those who have drawn up the TOR, turning the evaluation into an exercise with quite predictable outcomes that limit learning opportunities for everyone involved. 

Proud to be a mentor for Nurshaim Tilenbaeva

Nurshaim Tilenbaeva was my second mentee under the Global Change Leaders Programme. Mentoring Nurshaim, I embarked on a learning journey about about Kyrgyzstan, where Nurshaim lives. I do hope I'll get a chance to travel there! 

Photograph by Kusekhaya

There was one day when I was moved to tears by Nurshaim's honesty in describing difficulties she was facing at the time, at a different work place. Reading her narrative of that conversation now, I remember interviews I have carried out as part of a recent study on gender policies and practice in four German organisations. A couple of persons told me about a leader in their organisation shedding tears in public - which they described as a display of emotional intelligence and an empathetic form of leadership. Interesting! Meanwhile, as I have accumulated more experience in mentoring and coaching, I have learned to empathise differently - but I still get visibly moved when a mentee tells me about her success! And here are Nurshaim's answers to my questions: 

Proud to be a mentor for Kateryna Kravchuk

Five years ago, the Coady International Institute at St. Francis Xavier University in Nova Scotia (Canada) contacted me with a request to mentor a woman who had just graduated from its Global Change Leaders Programme (GCL). The idea was that we would have a one-hour Skype call every two weeks over a six-month period, to support the graduate in her efforts to translate learning from GCL into practice at home. I loved the idea! My first mentee was Kateryna Kravchuk from Ukraine, very close to my home time zone in Berlin.

Mentorships are great learning processes for everyone involved, I believe. I have happily agreed to mentor further GCL graduates since that first experience. Some mentees have answered a short set of questions to present themselves on my blog. I will post a mini-series of menteesin the coming weeks, starting today with my first GCL mentee, in her own words: 

Photograph by Andriy Maksymov

Kateryna Kravchuk - the economist, cultural researcher, focused in strategy, monitoring and evaluation of the projects aimed at improving the quality of life. Her main areas of expertise are community development, responsible entrepreneurship, cultural and creative industries, and cross-sector cooperation. 

(1) What is the main issue - or are the two or three main issues - that you are currently working on?

Currently I am busy with three projects: Creative Enterprise Ukraine (workshops and mentring for young entrepreneurs; keynote speaker for the Gender equality in culture and creative industries research), the evaluation of the UNDP/ADA project “Sustainable local development in rural areas” (two regions in Ukraine) and strategy development for Ivan Honchar Museum – centre for traditional culture in Kyiv. The main issues that are calling me since my childhood and more or less predefine everything I do are:

Classism in evaluation design

Individual interviews for "important persons", focus groups for "beneficiaries", right? Wrong!

These days I have been reviewing evaluations of projects supporting survivors of traumatising human rights violations in countries that are not quite at peace, or even still at war. One would think that in such circumstances, evaluators would be particularly respectful and careful with their interlocutors, avoiding questions and situations that would make them feel uncomfortable, trigger difficult emotions or cause a resurgence of their trauma. In some cases, the opposite is true:

Small group work - keep it fresh and productive!

It is the early afternoon of the second workshop day; the participants are a bit drowsy from a rich lunch; messages have piled up in their smartphones and some people would prefer to deal with those rather than discussing strategy or whatever the workshop is about. Small group work is on the workshop plan. What can you do to keep it lively and productive?

Two or three reasons why two are better than one

Evaluations come in many shapes and sizes. I have led multidisciplinary teams in multi-year assignments, and carried out smaller assignments all by myself. Last year was a lucky year, because most of my work happened in one of my favourite configurations: the tandem or duo - as in two competent persons with complementary or partly overlapping skills and knowledge working together as evaluators on an equal or near-equal footing. Two evaluators working together - even if one of them participates for a shorter spell of time than her colleague - means so much more than the sum of two persons' capacities. 

Is there a way to measure efficiency in human rights work?

Jasmin Rocha and I reflect on measuring efficiency in human rights programmes in a new guest post on Zenda Ofir's evaluation blog. Have a look here - and enjoy browsing!

Why internal evaluations need external perspectives

Internal evaluation can be an excellent way to check the quality of one's work, to track progress (in programmes or projects, for instance) and to gather information for management decisions and longer-term learning. To make the most of such exercises, they should go beyond self-reflection. Especially for small to medium-sized teams or organisations, sitting around a table and contemplating one's strengths and weaknesses, as well as successes and failures, is a good start, but just not enough.

Quick evaluation: What a difference a couple of days make!

You know those evaluations that come with dozens of questions on a whole complicated international development programme (or even set of programmes), to be answered within one week of desk work, one week on-site and one week to tie it all up? They are still around and they are not about to disappear. I used to hate them: The time frame makes it near-impossible to draw reasonably rigorous samples of respondents (for interviews or for a survey) and there is so little you can do and see in a week at the programme site. What can an outsider find out in one week that an insider doesn't know yet? After having worked on a couple of "quick evaluations" in recent months, I have adopted a milder stance. They can generate useful insights. But how? Here are a few tips.

VEPR Certified

I am delighted to announce that I have been an EES VEPR certified evaluator since autumn 2018. EES stands for European Evaluation Society, VEPR for Voluntary Evaluator Peer Review - a stringent process whereby the evaluator, in cooperation with two experienced international colleagues, identifies own capacities and weaknesses (and ways to deal with the latter) in a documented dialogue that spans over several weeks. 

Join the EES and try out the process, it really helps to know and hone your assets!