Evaluation of the UNICEF education programme in Timor Leste

In September 2009, UNICEF?s Evaluation Office in New York contracted the American Institutes for Research (AIR), International Development Division (IDD), to conduct an independent summative evaluation of UNICEF?s work in the area of basic education and youth literacy, life skills and participation in Timor-Leste. UNICEF?s Education Programme, “From Emergency Responses to Sustainable Development for Children and Adolescents in Timor-Leste,” is at a critical juncture as it nears completion and UNICEF and its partners seek to evaluate the efficacy of the myriad of interventions sponsored during the initiative to consider further collaboration.
In response to the numerous challenges facing the education system following Timor-Leste?s independence in 1999, the primary objectives of the Programme have been to:

(1) assist in the development of a high-quality basic education system through curriculum development and reform, as well as capacity-building efforts at all levels to support development of a basic education system;

(2) assist in advancing the Government of Timor-Leste?s (Republica Democratica of Timor-Leste Government or RDTL Government) goal of universal access to education for all Timorese children through institutional capacity building and modernization of education management and information systems to address barriers to access; and

(3) build youth capacities directly through programmes targeting literacy, life skills and participation.

Consequently, the objectives of this evaluation, as specified in the Terms of Reference (ToR) underlying it, were to determine: (1) the relevance, appropriateness and coherence of the UNICEF Timor-Leste Country Office Education Programme as a whole; (2) the effectiveness and impact of the programme in relation to its objectives; (3) the efficiency with which its project outputs and activities have been delivered; (4) the connectedness of UNICEF?s programme to those of other actors; and, (5) the programme?s coverage and sustainability. The evaluation, though summative in focus (and therefore “backward oriented”), also sought to uncover potential areas of learning so that the programme might improve moving forward.

UNICEF requested that this evaluation examine each of the main projects and sub-projects subsumed under the programme. However, rather than only assess the effectiveness of each individual programme, UNICEF also requested that AIR examine the programme as a whole as its unit of analysis, framing its findings in terms of the programme?s overall success in achieving its overarching objectives (e.g., increased access to high-quality basic education, youth literacy and life skills, participation of stakeholders in the educational process and so on).