Five top tips for developing high quality evaluation ToRs

The terms of reference (ToR) is one of the most important building blocks of the entire evaluation process, setting out expectations as a whole and serving as a basis for the evaluators’ work.

A tip-top ToR is critical in establishing the quality of the evaluation, as choices about scope and approaches can have a large impact on how usable the result will be. Evaluators develop their design and methodology based on the ToR, while it is also key to ensuring key stakeholders are involved in the evaluation process from the start and participate adequately.

Here are five top tips for developing a high quality evaluation ToR. These were presented at a recent training with staff from ALNAP member Solidarités International.

Tip#1

Clearly identify the use and the users of the evaluation.

• What are you going to use the evaluation for? Evaluations can be used for learning or accountability: they can be summative, formative or developmental.
• The use of the evaluation should guide you in the selection of evaluation criteria and questions.
• Identify the users of the evaluation: they should be involved from the start of the evaluation process to the end – from when the ToR is drafted until you respond to recommendations and make a plan for action.

Tip #2

Allow yourself enough time to plan the evaluation and develop the ToR.

• Start developing the ToR several months before the evaluation starts.
• The different users of the evaluation should be given enough time to provide feedback on and contribute to the draft ToR.
• For external evaluations, enough time must be allowed for the procurement of the evaluation team.

Tip #3

Prioritise what you need to know - limit the number of evaluation criteria and questions to the essentials.

• If you want evaluators to do a thorough analysis the number of questions needs to be realistic. Ask too many questions and the quality of the data collection and analysis will suffer. The evaluators will only be able to give surface level answers.
• If you are using the OECD DAC criteria, it is not advised to cover all the evaluation criteria at once. They can guide your thinking but are not a strict recipe.

How do you prioritise evaluation questions? Ask:

• Are they central to the evaluation’s objective?
• Can they immediately have their answers applied?
• Can they be answered unequivocally by the evaluation?
• Can they be answered only by an evaluation?
• Are they central to your core mandate?
• Could they improve the quality of the service you provide?
• Could they reduce the costs of providing a service?
• Would they affect or be of value to most stakeholders?

Tip #4

Dedicate enough financial resources to the evaluation

• The budget should match the complexity of the evaluation.
• If the evaluators are consultants, the budget should match the number of days they need to work on the evaluation. If the budget is too small, the evaluators will not have enough time to properly deliver.

Tip#5

Give the evaluators enough time to do their job rigorously.

• Adapt the evaluation’s time plan to fit the complexity of the evaluation questions and the context.
• If the process is rushed, the quality will suffer.

For further guidance on developing evaluation ToRs, please see ALNAP’s Evaluation of Humanitarian Action Guide.