Terms of Reference for Evaluator of Oxfams Disaster Risk Reduction Programme

Supporting community resilience in the South Caucasus

The purpose and scope of the evaluation
In July 2014 Oxfam GB and local partner organisations started the third phase of a participatory multi-hazard
disaster risk reduction programme in Armenia and Georgia, funded for 15 months by the European Commission,
Department of Humanitarian Aid and Civil Protection. The Supporting Community Resilience in the South
Caucasus programme is due to end in September 2015 and an external evaluation is sought to determine the
impact of this programme to date and to make recommendations to inform future programming for the Oxfam team
and partners, as well as the donor, ECHO. The evaluation itself will focus on a number of key questions in the
following areas: overall impact of the programme, impact on gender equality, sustainability as well as effectiveness
of coordination with government and non-government actors.
What the programme aimed to achieve
The specific objective of this programme is to increase resilience and reduce vulnerability of local communities and
relevant national institutions by supporting strategies that enable them to prepare for, mitigate and respond to
natural disasters in the South Caucasus region, through 3 main results:

Increased capacity of local communities (especially women) to identify and address the needs of most
vulnerable populations (including women, men, children, disabled and elderly) in the wake of disasters
Vulnerable communities face less risk to potential disasters through planning, mitigation and adaptation
Leading agencies and government authorities demonstrate increased awareness and acceptance of gendered
community-based DRR

Questions to be addressed by the evaluation

Overall Impact

To what extent were the project results and indicators achieved during the project?

Are the actual project outputs (e.g. the small-scale mitigations, new technologies, the community rescue teams
trained, DRR centres, green laboratories, resilient communities, village disaster management, local disaster
management plans, etc.) present within a community at the time of evaluation, and to what extent are they
actually utilized (or could be meaningfully utilized) by the project beneficiaries?

To what extent can the preparedness-related project outputs potentially be used in actual emergency situations
(e.g. whether a rescue team already rescued people, whether it would be physically available during an
emergency, whether it could or had the means to - assemble in time, whether the community was ever
evacuated in time during drills and simulation exercises etc)?

What were the unintended positive/negative impacts of the programme?

Gender and inclusion of the most vulnerable groups

What impact has the programme had on gender equality and womens realities? (e.g. increased decision
making, control over resources/assets, leadership roles within community management structures etc)

What steps were undertaken to ensure the active participation of women within the programme and what steps
could be taken in the future to improve this practice?

Did the programme meet specific gender indicators and targets outlined within the project document?

How were vulnerable groups (including children and the elderly) involved in the project? What impact has the
programme had on reducing vulnerabilities of these age groups?
Sustainability, institutionalisation and replication of Oxfam and partners interventions;
To what extent are changes made/achieved likely to be sustained?1
Are there examples of replication of developed models?
Do communities (or the final beneficiaries) genuinely have ownership of the project outputs? Will there be
motivation to continue and develop the project activities after Oxfam and partners withdraw?
1

o
o
o
o

distinguishing between the following types of sustainability

financial sustainability, i.e. ensuring that the project activities are financed after the EC funding ends
institutional sustainability, i.e. ensuring the local ownership of the results of the action;
local sustainability, i.e. ensuring that the links and coordination with the relevant local authorities are maintained;
policy-level sustainability i.e. ensuring that the action has a structural impact e.g. improved legislation, following codes of conduct,
methods and methodologies adopted etc.

How cost-effective was the programme?

How successfully did the project maximise government, community and other agency resources?

How successfully did the programme leverage additional sources of funding?

Examples of good practices and lessons learned;
o
Do any of the activities/projects and if so which show results, innovative qualities and potential for
replication?
o
Do any of the activities/projects show results or innovative qualities in relation to gender equality or
womens active participation in the project?
o
To what degree have Oxfam and partners learned from this experience as an organisation and shared the
learning with partners and other actors? What are the lessons learned?
Advocacy and coordination with the Government, amongst the other actors and with donors;

To what extent does Oxfams projects fit into and strengthen the existing local government or wider national or
regional emergency response and DRR government systems, institutions or plans?

To what extent has the programme worked with the government (participated in design of activities, utilised
government resources, worked to support the capacity development of the government structures)?

To what extent the level of coordination amongst partners and other NGOs contributed to useful exchange of
information, sharing of resources, prevention of duplication and did this impact on the beneficiary
communities?

To which extent donors are as well coordinated?

Resilience and integration of programmes:

How successfully did the project connect with other Oxfam projects and how could a one-programme
approach on resilience encompassing several Oxfam projects be designed?
Regional End of Programme Workshop & Evaluation Process
The evaluator will be requested to actively participate in the Regional End of Programme Workshop to be held in
Armenia at the beginning of September. During this workshop country teams will present successes and challenges
faced during the implementation of the project. The evaluator will also conduct evaluation field trips in Georgia
(Adjara, Tbilisi, Mtskheta) and Armenia (Vayots Dzor, Tavush, Yerevan), conduct a review meeting in Tbilisi and will
develop the evaluation report. The evaluator will be asked to do a desk work prior to the Regional Learning Review
and field visits.
Documentation review/ Existing data to be used in evaluation

Review of programme proposal documents

Baseline/endline report findings
Monitoring review reports
Frontline national reports and related case studies collected (in Georgia and Armenia)
Best practice write ups for DIPECHO
Review of project materials
Feedback/results of activities / Photos
External evaluation and final report from the second phase programme

Overview of programme

Briefings and overview of programme with Programme Coordinator

Briefings with each country team including partners on key progress and challenges: review of data collected
including media work, photographs plus overview of mitigation and adaptation work including process and
documentation as outlined in log-frame. Country specific indicators and activities will have to be taken into
account when evaluating the programme.
Field based data collection

Interviews with national, and local government (6 interviews)

Interviews with other actors (DIPECHO partners, ECHO local representative plus one other per country)

Focus group discussions with beneficiaries (in 6 villages- 3 per country)

Visit to mitigation and adaptation projects (in 6 villages)

Restrictions to consider

Some members of communities (particularly in Georgia) leave for upper mountains for cattle-herding
Key outputs
Executive summary and full report in English including key recommendations (max 22 pages plus annexes).

Suggested contents page

Executive summary x 2 pages
Introduction and methodology x2 pages
Overall Impact x 6 pages
Gender x 1.5 page
Sustainability of Oxfam and partners interventions x 1.5 page
How cost-effective was the programme? x 1 page
Examples of good practice x 2 pages
Coordination with the government and amongst the other actors x 2 pages
Resilience and integration of programmes x 2 pages
Conclusions and recommendations for the next stage of the programme x 2 pages
Additional responsibilities
As a part of Oxfams commitment to the capacity building of staff, the evaluator is requested to allow for the
shadowing of their work in the field and in all data collection. In addition, the evaluator will be requested to allow for
for regular Q&A sessions by shadowee. Where the evaluator sees appropriate, they may use the shadowee as
a resource to undertake relevant data collection or analysis tasks.
Relevant skills sets and knowledge the consultant should have

Please note that as per donor requirement the evaluator must be an external candidate and not an
existing Oxfam member of staff

Consultant should have permission to travel in the region (Georgia and Armenia) visa or non-visa
travel right, and be based in one of the countries.
Essential - Skills & Experience

Knowledge and proven experience in managing and leading evaluation assignments for a range of major aid
agencies or NGOs in particular evaluating complex multi-country or large donor programmes e.g. EC, ECHO

Strong data collection, analysis and writing skills

Broad knowledge of development issues and national policy and practice in DRR (in particular community
based DRR)

Familiar with the resilience concept

Good understanding of gender equality issues

Proven experience of working in DRR and/or humanitarian response programmes

Fluency in English
Desirable - Skills & Experience

Experience or knowledge of the South Caucasus

Management experience of community based disaster risk reduction programmes

Spoken and written Russian

Essential - Competencies

Ability to work with significant levels of autonomy

Proven analytical skills and ability to work with others with a dynamic and proactive attitude
Recruitment procedure
Candidates are asked to submit a copy of their CV, a cover letter outlining how they meet the key competencies as
well as a writing sample (ideally a previous evaluation authored by the candidate) and a financial proposal for the
assignment with a lump sum covering a fee and per diems. In addition, Oxfam will provide travel and
accommodation in both countries.
Assignment dates
31 August - 23 September 2015
Timeline and deliverables
Work will be carried out during the following times and reports shared with Oxfam on end dates:
Preparatory work, desk analysis of delivered documentation & materials: 31 August 1 September 2015
Participation in Regional End of Programme Workshop in Armenia, Field visits and Interviews in Armenia (Vayots
Dzor, Yerevan and Tavush) and Georgia (Adjara, Mtskheta and Tbilisi): 2-15 September 2015
Summary review meeting in Tbilisi: 16 September 2015
Report development and delivery of the evaluation report: 23 September 2015
Application deadline: 17 August 2015