Assessing the Strength of Evidence: How to Note: A DFID practice paper

Strong evidence is of central importance in informing policy and programming decisions
across UK government departments. Robust research and evaluation generates the
evidence required to form judgements, deliberate options and make intelligent decisions
about how to spend scarce financial resources on behalf of taxpayers. It is critical to the
work of all DFID staff and Ministers, and especially for the presentation of appraisal
options in Business Cases. It is vital that research evidence is evaluated in a fair and
balanced way.

What is the purpose of this guidance note?

2. This Note provides a thorough introduction to:

a. the appraisal of the quality of individual studies;

b. the assessment of the strength of bodies of evidence.

3. The Note is an integral part of DFID’s commitment to equipping staff with the skills and
specialist advice to help them improve their use of evidence. Other resources include:

a. A Guide to Research Designs & Methods;
b. The Evaluation Handbook, providing guidance on evaluation designs and
methods;
c. ‘Using statistics’ How to Note;
d. A set of guidance materials about how to summarise research evidence.

4. Assessing the strength of evidence is a challenging task. This Note sets a high standard for DFID staff. It requires a combination of technical knowledge and individual
judgement. It may also require consultation with research specialists within and outside
DFID. Proper assessment of evidence will help staff use evidence responsibly and
judiciously for the benefit of better policy and programmes. Specifically, this Note will:
a. help staff to broadly understand the distinctions between different data collection and analytical methods and what they can and cannot conclude as a result;

b. establish a common language that can be used in the discussion of the strength of evidence.

5. This guidance is applicable to all categories of research and evaluation evidence used by
DFID staff, especially in the social sciences. It applies to evidence generated through both
quantitative and qualitative research methods. It recognises that some academic
disciplines, such as medicine, and the methodologies associated with them, have a stronger tradition of assessing quality of research than social science disciplines.
The Note references alternative evidence grading frameworks accordingly. To ensure consistency, this Note also draws on approaches developed in other parts of Government to assess research and evaluation evidence.

6. A summary of this Note is incorporated into the revised Business Case guidance for ease
of reference.