Together let’s shape new guidance for humanitarian evaluation

ALNAP is updating its guidance on the OECD DAC Evaluation Criteria. Initial research will inform wider consultation through events and an online survey. We want to hear from you. 

In 2006, ALNAP published Evaluating humanitarian action using the OECD DAC Criteria, an ALNAP guide for humanitarian agencies, the most comprehensive guidance on the OECD DAC criteria’s use in humanitarian evaluations.  

ALNAP’s guidance is now 17 years old and is in need of a refresh. 

The OECD DAC evaluation criteria 

The OECD DAC evaluation criteria are the pre-eminent criteria for evaluating development and humanitarian assistance. Updated by the OECD DAC in 2019, the revised criteria are: effectiveness, relevance, efficiency, impact, sustainability and coherence. ALNAP and humanitarian actors add the criteria of coverage and connectedness (which replaces sustainability).  

The criteria are broadly used in humanitarian evaluations, contributing to the sector’s ability to improve the quality of evaluation and compare findings across evaluations.  

As they are widely applied, the OECD DAC criteria make evaluation synthesis easier, helping to capture achievements and common challenges in humanitarian action and making it easier for evaluators across the globe to work together.  

There is, however, a lively debate regarding many aspects of the criteria. They have been subject to criticisms, including the criteria not enabling the evaluation of transformational change and an insufficient focus on gender, equity or human rights concerns. 

Other common issues include:  

• Positionality: the criteria do not take into account whose perspective is used in defining the evaluative questions and who conducts the evaluation. This is related to calls for the decolonisation of evaluation.  
• The need for more guidance to improve standardisation, while maintaining flexibility in application.  
• The variable utility and application of the criteria, which depends on the type of programme, organisation and evaluation purpose.  

Time for a refresh 

The endurance of the OECD DAC criteria and having a common framework for evaluating and understanding performance in the humanitarian sector, are both reasons to ensure there is adequate guidance for humanitarian practitioners in using them.  

Contemporary critiques, changes in concepts, topical issues and language over time, and the varied application of the criteria in practice, suggest updated and additional guidance focused on their application in humanitarian contexts is needed.  

Initial findings  

In the ongoing process of updating ALNAP’s guidance, key questions have included: What are the main challenges and issues in applying the criteria? What can we learn from examining these challenges? How can we use this learning to improve existing guidance?  

To help answer these questions, ALNAP has commissioned a literature review. The findings of the review are captured in this summary brief, covering each of the seven OECD DAC criteria, as defined in ALNAP’s 2006 guide, and examining cross-cutting issues and potential additional criteria. Each section includes definitions, a summary of key issues and questions for further exploration.  

The review includes an exploration of issues, including common constraints and challenges in applying the criteria, drawing on examples from real-world humanitarian evaluations, notes on good practice and learning since 2006. 

Wider consultation 

The review will now inform further exchanges in the humanitarian evaluation community, as ALNAP seeks to refresh its guidance and understand contemporary perspectives on challenges in applying the criteria in humanitarian contexts.  

Issues identified in this paper are but the beginning.  

An online launch event for the wider consultation process began a series of planned in-person and online events in the coming months to gather views on the criteria, their use by humanitarian actors and suggestions of key issues to address in an updated ALNAP guide for evaluating humanitarian action in 2023.  

In partnership with the Asia Pacific Evaluation Association (APEA), ALNAP will be hosting an interactive online consultation event for the Asia Pacific humanitarian evaluation community on Thursday, 26 October 2023. Find more here and register.

A further event for the Francophone evaluation community will be held before the end of the year.

ALNAP is looking for feedback from a broad spectrum of evaluators, humanitarians and other practitioners, from large agencies to small frontline actors, local community groups and first responders, working in diverse locations and different humanitarian contexts.    

Take the survey, help shape the guidance 

Given the popularity of the existing 2006 ALNAP guidance, there is likely to be great interest in contributing to the update, as well as diverse views on how to best apply the criteria in humanitarian contexts. 

ALNAP has launched a global survey of humanitarian and evaluation practitioners, a further opportunity to share views on changes you would like to see in ALNAP’s evaluation guidance.  

Are the current criteria fit for purpose? If not, what set of criteria and approach to applying them do we need to be able to better assess the performance of humanitarian action? How can ALNAP assist in providing useful guidance for the broader sector? How closely should ALNAP’s guidance align to the OECD DAC’s? 

These are fundamental questions. So, let’s hear what you think. 

- Visit our website to find more information about participating in the survey and upcoming consultation events. If you are interested in co-hosting an event, being a speaker or otherwise wish to get in touch, contact Sarah Gharbi at: evalcriteria [at] alnap [dot] com