Acf Pakistan DRM (Rne) Final Evaluation

External Evaluation
Reducing Vulnerability through Disaster Risk

Management (DRM) in Khyber Pakhtunkhwa
(KPK), Pakistan
Funded by Royal Norwegian Embassy in Islamabad
Niaz Murtaza, January 2012
Photo NIaz Murtaza
This report is commissioned by Action Against Hunger | ACF International. The comments contained herein
reflect the opinions of the Evaluator only.
ACF Pakistan RNE DRM Project Evaluation, 2012
TABLE OF CONTENTS
Executive Summary
Chapter 1: Brief Background
4
4
4
5
Emergency Context in Pakistan

Overview of the Project
Project Details
Chapter 2: Methodology
Evaluation Questions
Evaluation Approach
Evaluation Constraints
Chapter 3: Findings
Targeting
Project Design
Project Implementation
Project Monitoring
Project Sustainability
Cross-Cutting Issues
Chapter 4: Conclusions and Recommendations

Conclusions
Recommendations
Appendix
Damage in KP from 2010 Floods
Scoring Matrix
Acronyms
List of interviews
List of documents reviewed
Good practice case study
7
7
7
8
9
9
13
16
19
20
22
25
25
26
29
29
29
31
31
32
32
XECUTIVE SUMMARY
Over the last decade, Pakistan has experienced large-scale internal displacement
caused by a range of natural and human-made disasters. Structural poverty,
inappropriate development, rapid urbanization, inadequate infrastructures,
increased deterioration of the environment - deforestation, increased human
settlements in hazard prone areas, etc. have increased the vulnerability to disasters. Thus, even
a moderate flooding could have devastating effects in the future. An ACF DRM assessment
identified a lack of awareness amongst stakeholders concerning DRM, and thus a need for
capacity building. Thus, ACF carried out a DRM project funded by the Royal Norwegian
Embassy during 2011-12 to reduce morbidity and mortality risks by improving community
resilience to natural disasters in Nowsherra, Charsadda, Mardan and Lower Dir regions of KPK
province in northern Pakistan which are vulnerable to both serious floods and earthquakes.
At the end of 2012, ACF commissioned an evaluation to evaluate the impact and approach of
ACFs RNE funded DRM project. ACF subscribes to the Development Assistance Committee
(DAC) criteria for evaluation: Impact, Sustainability, Coherence, Coverage, Relevance /
Appropriateness, Effectiveness and Efficiency. ACF also promotes systematic analysis of the
monitoring system and cross cutting issues (gender, HIV/AIDS etc). These dimensions served
as the main evaluation criteria. The tools included documents review, interviews with ACF
international and national staff, government officials and partner staff and household interviews
(with 204 community members), Focus Group discussions and transect walks in 18 villages.
Conclusions and Recommendations

Overall, the project is highly relevant to Pakistan given its high disaster vulnerability and is a
well-conceived one with an excellent and comprehensive mix of hardware and software
activities. Many of the project activities have also been implemented well though others do have
some room for improvement. The main findings can be summarized as follows: i) Targeting of
the most vulnerable is excellent in Charsadda and Nowshera but is less accurate in Mardan and
Dir; ii) ACF has set-up village committees to enhance the sustainability of project activities but
the committees must be linked adequately with other NGOs and the government to enhance
sustainability, iii) The hardware activities should focus more on those which provide more
structural and pro-active protection against floods, iv) Spreading the message to the whole
communities in villages has proven to be a challenge due to their large sizes, v) A sizeable
proportion of the project activities got delayed to the last month due to government permission
delays, procurement delays and staff training, vi) there is a need to enhance the technical
capacity of the team in software DRM activities, vii) monitoring and accountability mechanism
need to be strengthened. Thus, the main recommendations are as follows:
I. Targeting: It is recommended to continue work in Dir, Charsadda and Nowshera but to
reconsider working in Mardan due to its relatively low vulnerability and to undertake a more
thorough needs assessment in Dir to be able to target the most vulnerable village there. ACF
2

is also advised to focus on districts highly vulnerable to major floods based on the following
three criteria: River and drainage channels mapping; Damage incurred during 2010 floods and
UNDP One DRM project list of most vulnerable districts. Consider Sindh for DRM work too
since it has recently become the most repeatedly struck province in Pakistan.
II. Number of villages: Review the possibility of increasing the number of villages in the next
phase by increasing the number of field staff, covering each village with only one hardware
construction activity and linking with other NGOs and government agencies to cover some of
the construction work that ACF cannot afford in its budget.
III. Selection of hardware activities: Use the prevention-mitigation-capacity-evacuationrescue-relief continuum of DRM activities presented in Chapter 3 to select hardware activities.
The contingency stocks are an important hardware input given their relatively low cost and the
fact that they cover three categories in the continuum even though they are the three bottommost categories. Select other hardware construction activities from the highest possible
category of this continuum which is relevant and affordable.
IV. Village committee sustainability: Enhance the sustainability village committees by
providing them training on CBO management issues, having them adopted by other agencies
working long-term in the area (e.g., Concern), and linking nearby committees with each other
for mutual support and learning during crisis and normal times
V. Community DRM awareness-raising: Follow up more closely with committees about their
community DRM awareness-raising session schedules and also look at the possibility of
training young educated and unemployed persons in each UC to conduct sessions in all
targeted villages in that UC for a small honorarium
VI. Monitoring and accountability: Develop formal monitoring plans which clearly specify the
role for all relevant staff in the projects monitoring, including the frequency, modality, project
dimension and indicators for each relevant ACF staff and how the information coming from the
monitoring done by each staff will be analyzed and summarized Also institute strong and
responsive accountability measures in future projects. Build sufficient time in projects for
government approvals and procurements.
VII. External linkages: ACF should aim to increase government involvement in the project. ACF
could get its village committees registered, and advocate with the DDMAs to treat NGO village
committees as valid structures any government future grass-roots DRM initiatives. It should
sign an MOU with government at time of starting the project which mentions the roles,
responsibilities and inputs that both sides will provide. ACF should also enhance its profile in
coordination sectors like the DRR Forum and the UNDP DRM project and obtaining greater
technical inputs from them. Enhance technical inputs on DRM work by having CPDM provide
some degree of field-based support to ACF staff on software issues, e.g., by developing the
software package in one model village.
3
HAPTER 1: BACKGROUND
This chapter provides an overview of the emergency context in Pakistan, and

the specific project implemented by ACF with Royal Norwegian Embassy
funding which is the subject of evaluation in this report. The chapter serves the
purpose of placing the whole evaluation in its proper context and in familiarizing
the reader with the project constraints and scope.
1. Emergency Context in Pakistan

Over the last decade, Pakistan has experienced large-scale internal displacement caused by a
range of natural and human-made disasters. Large-scale displacement occurred in Pakistan in
2010 after the worst flooding to hit Pakistan in memory affected 20 million people, forcing over 7
million people from their homes. Although most of the flood IDPs returned to their home areas
soon after floodwaters receded, most were living in the open for prolonged periods as over 1.9
million houses were damaged or destroyed. The crisis in 2010 was only the latest in Pakistan. It
had been preceded by dislocation of population following clashes between rival militant and
sectarian groups in the tribal areas; military operations against extremist militants and
insurgents; generalized violence and violations of human rights; a devastating earthquake in
2005; and annual floods across the country. Structural poverty, inappropriate development,
rapid urbanization, inadequate infrastructures, increased deterioration of the environment deforestation, increased human settlements in hazard prone areas, etc. have increased the
vulnerability to disasters. Flooding recurred in 2011 and 2012, which combined with incomplete
recovery in the 2010 flood-affected districts. Thus, even a moderate flooding could have
devastating effects in the future.
2. Overview of the Project

An ACF DRM assessment in 2010-11 showed that community and provincial DRM plans do not
exist (except for symbolic or outdated documents). It identified a lack of awareness amongst
stakeholders concerning DRM, and thus a need for capacity building. It also recognized a need
for structural mitigation to reinforce infrastructure against the damage caused by a disaster, a
need for improved management of agricultural land as well as a need for community level DRM
activities. Thus, ACF carried out a DRM project funded by the Royal Norwegian Embassy during
2011-12 to reduce morbidity and mortality risks by improving community resilience to natural
disasters. With a focus towards infrastructure that is highly affected by disasters, yet vital after a
disaster occurs, as well as early warning systems which are imperative for reducing loss of life
and environmental degradation stemming from natural hazards, ACF aimed to develop local
capacity and introduce risk mitigation in the intervention areas. The project was a multidisciplinary initiative that aimed to involve a variety of stakeholders including ministries, local
authorities, disaster management agencies, NGOs, the academic sector and communities. The
direct beneficiaries of the project were the vulnerable populations affected by natural and manmade disasters in Nowsherra, Charsadda, Mardan and Lower Dir regions of KPK province in
4

northern Pakistan which are vulnerable to both serious floods and earthquakes. The basic
premise of the project was that communities generally understand local realities and contexts
better than outsiders. If at-risk groups are involved in all stages of the disaster risk
management, a stronger foundation will be created for the development of sustainable
programs for risk reduction. Through the use of participatory techniques for data collection and
analysis, hazard analysis, trainings, ACF aimed to ensure full participation.
3. Project Details
Project General Objective
To reduce vulnerability and improve resilience to disasters in four districts of KPK, Pakistan
Specific Objectives
To enable communities to prepare for and mitigate the impacts of disasters using participatory
methods aimed at strengthening the self-reliance capacities of the population and reinforcing
or adapting infrastructure
Results
R.1- The ability to recognize hazard events and to cope with their effects is improved
R.2- The capacity to reduce the effect of hazards on communities and services is improved
R.3- The understanding/application of DRM at community and institutional level is improved
Programme Activities
R.1- The ability to recognize hazard events and to cope with their effects is improved
80% of intervention communities have functional local early warning systems
80% of target community can claim access to contingency stocks
90% of target communities are covered by disaster management plans.
R.2- The capacity to reduce the effect of hazard event on communities is improved
Facilities serving 60 communities are structurally strengthened to resist hazard events
90% of intervention communities have identified and equipped evacuation sites
R.3-The understanding/application of DRM at community/institutional level is improved
80% of intervention communities have disaster management units
240 individuals from local or district authorities receive DRM training
25500 individuals in the communities receive DRM Training
Specific Activities
Installation of Early Warning Systems in 48 Villages
Formation of 48 early warning groups in the community
Training of 48 early warning groups in the community
Development of emergency response plans and contingency stocks in 48 villages
5

Development of disaster preparedness coping mechanisms in 54 villages
Conducting of PCVA assessments/community risk and vulnerability mapping
Preparation of village disaster management plan in 54 villages
Signing Memorandum of understanding with 48 communities
Rehabilitation of 14 Flood affected small scale irrigation channels for agriculture
Construction of 5 Retaining and Flood protection walls
Rehabilitation of 13 hazard proof water supply schemes
water quality testing (Bacteriological and chemical testing)
Rehabilitation/reconstruction of water and sanitation facilities at evacuation centers
Rehabilitation of culverts at 15 evacuation routes
Rehabilitation of 13 evacuation roads
Formation of 48 Village Disaster Management Units within the community
Training of 48 DMU's on CBDRM
Training and sensitization of 240 local authorities and local institutions in CBDRM
Development of Contingency Stock at District Level
Community training on CBDRM
Awareness sessions on DRM at Boys & Girl Schools
HAPTER 2: METHODOLOGY
At the end of 2012, ACF commissioned an evaluation of the RNE funded

DRM project activities in Khyber Pakhtunkhwa (KP). ACF subscribes to the
Development Assistance Committee (DAC) criteria for evaluation: Impact,
Sustainability, Coherence, Coverage, Relevance / Appropriateness,
Effectiveness and Efficiency. ACF also promotes systematic analysis of the monitoring system
and cross cutting issues (gender, HIV/AIDS etc). These dimensions served as the main
evaluation criteria.
1. Evaluation Questions
The detailed evaluation questions are provided in chapter 3 and were grouped as follows:
Targeting
Project Design
Project Implementation
Project Monitoring
Project Sustainability
Cross cutting Issues
2. Evaluation Approach
Review of Relevant Documents
Prior to the field work, the evaluator reviewed the relevant documents, including those related to
this project, ACFs general work in Pakistan and DRM globally and documents related to DRM
and emergencies in Pakistan in order to gain a better understanding of the project and its
context and to help develop the evaluation methodology and instruments. Sufficient
documentation was available.
Initial Briefing
Prior to the field work, the evaluator participated in a teleconference with ACF New York HQ.
Indirect Information
Interviews were conducted with local authorities in Lower Dir, Mardan and Charsadda; project
expatriate and national staff in Islamabad and the districts; donor representatives and the
Director CDPM University of Peshawar.
Field activities
Household interviews with community members and Focus Group Discussions (FGDs) with
village committee members based on participatory evaluation methods were utilized. The
villages visited were selected randomly from lists provided by ACF while the community
members for household interviews were selected based on convenience sampling. Random
sampling for household interviews was not attempted since the project provided community7

level services rather than individual household level services and as such there were no project
beneficiary lists from which people could be randomly selected. In each village, transect walks
were also conducted to the source of disaster vulnerability of each village (e.g., rivers near the
village) to get a better understanding of the disaster profile of the village and the manner in
which ACF work addressed the source of vulnerability. During the transect walks, the hardware
work constructed by ACF in each village were also visited to get a better understanding of the
manner in which ACF work addressed the source of vulnerability of the village.
Village Sampling Details
Villages
FGDs
HH interviews
Charsadda
60
Mardan
36
Nowshera
60
Lower Dir
48
18
18
204
Total
3. Evaluation Constraints
A major evaluation constraint was the budget available for the evaluation which meant that while
the evaluation was conducted by an external evaluator, he was assisted in interviewing females
by two ACF female staff members from other ACF projects. In order to analyze the impact of the
participation of ACF staff in the data collection on the neutrality and objectivity of the findings,
the field data is presented separately by males (interviewed by external evaluator) and females
(interviewed by ACF staff). As seen later in chapter 3, except for one section of the household
questionnaire, the differences in the responses given to the external evaluator and internal ACF
staff were not found to be unusually different. However, for the future, it would be advisable for
ACF to recruit independent females for evaluations wherever possible even though the external
evaluator did accept the ACF proposition in this case. The budget implications of this strategy
could be managed by reducing the number of field visit days, which the external evaluator found
to be on the high side compared with other evaluations that he has conducted. Thus, the
additional villages visited did not necessarily add to the richness of the data collected and the
number of days in each district could easily have been reduced by one without affecting the
quality of data collection. Another constraint was the security situation in Pakistan as a result of
which the evaluator could not visit Peshawar due to the sudden deterioration of the security
situation during the evaluation. Field work on two particular days was also scaled back due to
security incidents. However, given the fact that the external evaluator is originally from Pakistan
and speaks the national language, the impact of security incidents on the evaluation activities
was minimized. Finally, timing was also a constraint as the evaluation was conducted near the
end of the year when some of the staff members were on leave, ACF was also focused on
finishing project activities and the Project Manager had finished his contract. However, despite
all these constraints, the evaluator succeeded in collecting adequate amounts of quality
information, mainly because of his strong background in conducing evaluations in Pakistan.
8
HAPTER 3: FINDINGS
This chapter provides the findings according to each question included in the
TORs under the headings mentioned in the last chapter. However, the
sequence of the questions and their placement under the headings has been
changed in some places to enhance readability. The findings under each
question are based on triangulation of information from the various sources of information
mentioned in the last chapter.
1. Targeting
This section focuses on the following questions
a) Did the communities selected for the project classify as being most vulnerable and
highly exposed to disaster risk, and did they meet the criteria established in the
project design? Could the PCVA process have been in any way improved to better
achieve this?
The evaluator visited 18 of the 60 villages targeted by the ACF for this project. All the villages in
all four districts were found to be vulnerable and those in Charsadda and Nowshera were also
among the most vulnerable villages in KP province highly exposed to disaster risk in the opinion
of the evaluator based on the numerous evaluations that he has conducted in Pakistan after the
2010 floods. The worst-affected villages were those where the floods destroyed most houses in
the village and most of the families in the villages were displaced for weeks or months, finally
returning to live in their villages in tents or makeshift houses for prolonged periods until they
could construct or received houses. Thus, the difference across the districts can be clearly seen
in the response of individual households to the following questions across the four districts:
Difference in disaster vulnerability across ACF districts
When was your village last affected by a major
disaster? (% identifying 2010 floods)
Was your family displaced in the disaster?
(% responding yes)
If yes, for how long? (no. of months)
Was your house damaged during this disaster?
(% responding yes)
Charsadda
Nowshera
Mardan
L. Dir
100%
100%
100%
100%
96%
92%
62%
10%
2.1
1.2
0.7
0.4
96%
92%
60%
40%
In all districts, the 2010 floods were identified by people as the most devastating disaster in
living history. FGDs revealed that the 2010 floods were viewed as more damaging than the
2005 earthquakes in the villages which experienced both mega-disasters. However, there is
significant difference in the damages experienced across the four districts. While 90%+ of the
families were displaced in Charsadda and Nowshera were displaced during the floods, only
10% of the families in Dir were displaced while 62% were displaced in Mardan. The period of
displacement ranged from a high of 2.1 months in Charsadda to 0.4 months in Dir. Finally, only
9

40% of the families reported house damage in Dir compared with 90%+ in Charsadda and
Nowshera. Furthermore, FGDs revealed that even where houses were damaged in Dir, the
damage was relatively minor, consisting of cracks or a wall partially collapsed compared with
whole houses being swept away in Charsadda and Nowshera. However, villages in Dir did
suffer crop and infrastructure damage. Even so, it would be useful to conduct more thorough
assessments in Dir to see if there are more vulnerable areas for future work. Thus, the evaluator
would recommend the adoption of formal and clear targeting criteria so that the most vulnerable
villages can be targeted more consistently across all project areas. ACF currently has targeting
criteria which include need, direct access, absence of other agencies and security. However,
there is a need to operationalize the need criteria further for the KP context. The following
sections provide the basis for developing such operationalized needs criteria.
International NGOs can generally intervene with emergency response only in cases of major
disasters because of the shortage of funds and based on the premise that communities and
local authorities can deal with minor emergencies on their own. Thus, it is recommended that
even for DRM work, international NGOs should focus on natural hazards which are capable of
producing major disasters and on geographical areas where these natural hazards can be
expected to produce major emergencies with a view to reducing the needs for major emergency
response by INGOs subsequently as a result of this DRM work. Villagers and ACF staff in each
area mentioned a wide range of natural hazards to which communities are exposed, including
earthquakes, floods, snowfall, landslides, seasonal diseases and drought. However, it is also
clear that within this long list, it is only earthquakes, droughts and floods that are capable of
producing major disaster globally. Snowfalls, landslides and seasonal diseases almost never
produce large-scale disasters which would justify emergency response by international NGOs.
Moreover, even droughts are unlikely to produce major emergencies in the context of KP
province. Thus, the starting point of formal targeting criteria for ACF should be to concentrate on
areas which are highly prone to earthquakes and floods in KP. Even among these two, it is
clearly floods which are more amenable to DRM work due to their more predictable nature,
especially DRM work by ACF given the fact that earthquake-proofing generally requires
structural improvements in shelter construction, which is not a core ACF sector. Thus, ACF
should primarily focus on areas which are vulnerable to major floods and then incorporate
earthquake DRM concerns if those flood-prone areas are also vulnerable to earthquakes.
With floods identified as the most important natural hazard to focus on for KP, the next task
would be to identify the most flood-prone areas in KP. Unfortunately, ACF is constrained in
doing so by the fact that the Pakistani government has yet not developed rigorous, technologybased hazard mapping. Thus, it would have to rely on less rigorous sources of information. The
starting point for the future would be to look at where the major rivers flow in KP. This would
immediately reveal the fact that Nowshera and Charsadda host all three major rivers in KP
(Indus, Kabul and Swat), while Mardan hosts at most major drainage channels (e.g., the Kalpani
drainage channel) while Dir hosts a couple of smaller rivers. It is the location of rivers which
explains the vast differences in the responses above. All the villages in Charsadda and
Nowshera were on major rivers banks, those in Mardan were generally next to the Kalpani while
10

those in Dir were not near any major river or even drainage channels. Another source of
information for identifying the most vulnerable districts, UCs and village would be to look at
which areas incurred the most damage during the 2010 floods. Appendix A, which shows the
damaged caused by the 2010 floods in KP, reveals that Charsadda, Nowshera, Dir and Mardan
were number 1, 2, 8 and 20 among KPs 24 districts in terms of number of affected people.
More disaggregated information at the UC and village levels may be available with the
government. This district ranking reveals that while Charsadda and Nowshera are prime targets
for future DRM work, Mardan seems to be a low priority district since very few people were
affected in the first place. Clearly, Dir is a very isolated area and one which is probably more
poverty-stricken than the other three districts. Figures also reveal that it incurred a high degree
of loss. While Dir has a relatively large number of people officially recognized as being affected
(i.e., eligible for government assistance programs, such as the Watan cards), it seems that the
Dir team has not been able to focus on some of them due to security issues as some of the
areas identified by the Dir government authorities as highly affected (Lal Qila and Samarbagh)
were perceived by the Dir team as unsecure. Thus, it would be useful to conduct more in-depth
needs and security assessments there for future work. Finally, the UNDPs one DRM project
has identified 30 most vulnerable districts in Pakistan. This list could also serve to guide ACFs
geographical targeting. Thus, the PCVA process could be strengthened by first weeding out the
less vulnerable villages by macro-level criteria and focusing the PCVA process on the most
needy villages to begin with.
Recommended needs criteria for future DRM geographical targeting

Basis for initial targeting should be natural hazard, with floods being the main priority
Subsequent targeting should be based on the districts, UCs and villages most flood-prone
with the following being the main sources of information:
-River and drainage channels mapping
-Damage incurred during 2010 floods
-UNDP One DRM project list of most vulnerable districts
11
Vulnerability Zones and ACF Project Sites in 4 KP Districts
The NGO Code of Conduct asserts that agencies will distribute assistance based on objective
needs. The evaluator strongly feels that NGOs should have strong information within their files
to demonstrate that they are doing so for each project. It would also be advisable for ACF to
ensure that it does so for each project. Currently, while the targeting is excellent in Charsadda
and Nowshera, even for those districts, the corresponding information to reveal that these really
are the most vulnerable villages is not immediately available in project files. Having a map
immediately available in its files which shows that all the villages in Nowshera and Charsadda
are next to major villages along with information about the damage caused by the 2010 floods in
various districts, UCs and villages will help ACF in clearly demonstrating its adherence to this
NGO code of conduct principle. Currently, ACF has the maps with its project areas shown next
to rivers in Nowshera and Charsadda (see maps below) but not the information about 2010
damages. As can be seen in the maps below, the project areas in Dir and Mardan are not next
to major rivers. Similarly, if some areas are being left out due to security reasons, as in Dir, it
would be good to have a formal security report by its professional security team and a
subsequent SMT decision to back up this decision. If some areas are being left out because
12

other agencies are already present there, it would be good to have on file minutes of meetings
of coordination bodies where such coordination among NGOs normally takes place.
b) Did the project strike the right balance between number of villages covered and the
amount of work done in each village? If not would it have been better to cover more
villages with less work, or fewer villages with a greater amount of work?
While detailed appropriateness of the software and hardware work done by ACF will be
discussed in the next section (project design), in order to answer this question, it can be
asserted briefly here that the evaluator finds the particular mix of such activities to be highly
appropriate and well-selected, which means that it would not be advisable to drop any of them
in order to increase the number of villages. At the same time, given Pakistans large size and
high vulnerability, the number of most vulnerable villages is very high compared with the 60
villages covered in this project and probably runs into hundreds if not thousands. At the same
time, the number of agencies undertaking DRM work is small. Thus, there is a clear need to
increase the number of villages from the present 60 in a future similar project without completely
dropping any category or even sub-category of work. This obviously poses a dilemma. As far
as the evaluator can see, the main bottleneck to increasing the number of villages is the number
of front-line field staff who work at the village level. Fortunately, such staff does no cost much
and an additional staff member could perhaps be hired for as little as 12,000-15,000 dollars per
year, which could easily be absorbed by some minor reconfiguration of the overall budget. The
other bottleneck is the cost of hardware work which is much higher. However, the evaluator also
noticed that in some villages more than one hardware activity was being implemented. By
keeping hardware activity down to one activity per village, it may be possible to cover more
villages. Also, ACF could overcome the hardware-related constraint and increase the number of
villages by trying to leverage its work by getting the government and other NGOs to cover some
of the hardware activities which may be beyond ACFs budget.
2. Project Design
a) Were the objectives, results and activities selected in the implementation of this
program relevant for the identified population needs? Was the participatory approach
used appropriate an effective in identifying and selecting specific interventions? Was
there sufficient consultation and participation of beneficiaries in the design, targeting,
implementation and post-implementation stages? Did the activities match with what
people wanted?
The starting point of ACFs work for this project is the Participatory Community Vulnerability
Analysis (PCVA) exercise which aims to view each villages vulnerability and possible steps to
reduce it through the eyes of the community by using participatory methods to understand the
main hazards, community strengths and weaknesses, and possible remedial measures for
strengthening the communitys resilience. The use of such an approach has helped ACF to
adopt a highly participatory process and identify the main components of the project in light of
community perspectives. While many of the project components, such as early warning
systems, contingency stocks and village plans, are obviously based on industry standards for
DRM work, their specific form in different villages has depended on community perspectives. To
13

begin with, the identification and development of the village hazard profile and community
resilience and weaknesses is largely based on community perspectives. Furthermore, the
village committees, which were the main lynchpin for project implementation, were selected by
gathering villagers and having them select responsible, trustworthy people who represented all
segments of the community. Furthermore, the village disaster plans were based largely on
community perspectives. The specific contents of the early warning systems, evacuation sites
and contingency stocks were based on community participation. Finally, the hardware work
done in villages was largely selected by villagers within the constraints of the project budget.
b) What are the main problems (negative or unexpected impacts from the interventions)
that the population identified concerning the implementation of the activities and
what solutions were identified by the project team?
The responses to the following questions reveal the high degree to which communities felt
involved in the project. Satisfaction was very high across all districts, though slightly lower in Dir,
and both sexes. The main problems identified by the committees related to the difficulty in
covering the whole communities with DRM awarenessraising given the large sizes of some of
the villages. A couple of villages also mentioned that ACF staff sometimes came late to
meetings, a problem created often due to the security situation in the area because of which
ACF security staff asked ACF program staff not to proceed until the security situation cleared.
However, there were no complains with the basic contents and approach of the program.
Community satisfaction with ACF project approach
Percentage saying Yes
Charsada
Nowshera
Mardan
Were you consulted about the types of services and

their content?
98
96
92
Did agency staff always treat you according to your

cultural norms?
95
95
Were services provided keeping in mind your work

schedule?
96
Were the services timely for you in light of your

needs?
97
Lwr
Dir
Male
Fem
ale
84
90
95
97
96
95
99
97
97
88
95
94
96
97
81
99
92
c) Did the project strike a good balance between hardware and software? If not, define
what the balance should have been? Was the scope of hardware activities sufficient?
If not, what other activities could have been added? Was the scope of software
activities sufficient? If not, what other activities could have been added?
The main hardware activities within the project consisted of the provision of contingency stock
for evacuation, rescue and relief activities and the construction of latrines, evacuation paths,
drainage and irrigation channels and flood protection walls to reduce the risk of disasters. The
main software activities were the setting up and training of village committees, development of
village disaster plans and early warning systems, awareness-raising about DRM among
communities and schools, training on DRM for construction workers and training of government
14

officials. These activities can be better compared with the help of a conceptual framework to
evaluate and compare the value and impact of different DRM activities. As shown in the table
below, DRM activities can be classified along a continuum consisting of six categories with each
higher category representing a more structural and stronger form of disaster risk reduction than
those below it. Prevention activities include efforts to reduce the occurrence of destructive
hazards, e.g., through reforestation and riverbed dredging, and are obviously the most effective
form of DRM. However, such activities are generally done by governments rather than NGOs.
Given that even with high prevention efforts, there will still be some hazards occurrence, the
next best option is mitigation, i.e., redirecting hazards away from communities through flood
protection walls, drainage channels etc. Given that even with high mitigation efforts, there will
still be some hazards reaching communities, the next option is to move communities out of the
way of the hazards through evacuation before the hazard reaches them. The next option is
rescue efforts for those who could not be evacuated in time. Finally, the provision of timely relief
activities to both evacuated and rescued people reduces the risk of epidemics. In addition,
building the capacities of communities can also enhance their resilience to disasters.
Continuum of DRM Activities
DRM categories
Purpose
Examples
Related ACF
activities
Prevention
To
reduce
the Global climate change None
occurrence
of advocacy;
riverbed
destructive hazards
dredging; reforestation
Mitigation
To
build
buffers Drainage
channels, Drainage
channels,
between hazards and flood protection walls
flood protection walls
communities
Community
capacity-building
To
enhance
the Livelihoods
work, Irrigation
channels;
resilience
of nutrition work, CBO VDMUs; community
communities
by strengthening
awareness-raising;
strengthening
their
Village disaster plans
socio-economic status
Evacuation
preparedness
To help communities Early warning systems; EWS,

evacuation
escape approaching evacuation paths and paths
and
sites
disasters
sites;
selection, contingency
stock items (e.g.,
megaphones)
Rescue
preparedness
To help rescue people Rescue

tool
and Contingency
stock
caught by hazard
equipment stocks
items
(e.g.,
life
jackets)
Relief preparedness
To provide life-saving Watsan,

health, Evacuation
center
services to evacuated shelter, NFIs stocks
latrines; contingency
and rescued people
stock items (e.g.,
water jerry cans)
15
With the understandable exception of prevention activities, which are generally out of the range
of NGO technical and financial capacities, ACF work is covering all the remaining categories of
DRM work. This reflects the comprehensive nature of the DRM package that ACF is providing.
On the software side, activities like village committee and disaster plan development, early
warning system and evacuation site identification do not cost much and can also be seen as
mandatory components of DRM work. Thus, those activities should all be retained. The real
issue is about hardware activities since they cost much more. The above framework can help in
making decisions about specific hardware work. Thus, the objective in each village should be to
select activities which fall in the highest possible category above while being relevant to the
village and affordable within the ACF budget. So for example, flood protection walls, drainage
channels and irrigation channels are much more durable forms of DRM work than activities such
as evacuation paths and latrines in evacuation centers since the former reduce the chances of
communities getting affected by disasters in the first place while the latter are about helping
communities once they have been struck by hazards. Flood protection walls tend to be
expensive and should preferably be built where benefiting large or several villages os that their
per beneficiary cost is closer to other mitigation activities. More specifically, the evacuation
paths built will not be helpful in major disasters as they will become fully submerged. Within
community capacity-building, especially those related to livelihoods strengthening, activities
which benefit the whole community, such as irrigation channels, are more beneficial than
activities which target individual families, such as provision of agricultural inputs to individual
households. Finally, ACFs contingency stocks packages are also a very useful input even
though they target the lower categories for the simple reason that they have a low cost but still
cover three DRM categories (evacuation, rescue and relief categories). In summary, the
software activities and contingency stocks should be mandatory components given their low
cost and wide-ranging coverage. Among the remaining hardware activities, those targeting
mitigation and community capacity-building are more useful than evacuation paths and latrines.
That said, it is recognized that the latter may be the best affordable option for some villages.
3. PROJECT IMPLEMENTATION
a) Evaluate the impact of the DRM training and disaster plans e.g. impact on capacity of
trained population to be better prepared through the training and preparedness plans.
Evaluate the effectiveness of the Early Warning system that was put in place.
DRM village committee and training: In all the villages visited in the four districts, the
evaluators found that both men and women village DRM committees had been formed and had
received training from ACF on DRM activities. The village committees seemed representatives
of all sections of the community in each village and seemed highly engaged in the DRM work
initiated by ACF. As a result of their DRM training, the committee members generally seemed
familiar with DRM concepts and fully geared up to play an active role in keeping their villages
safer during future disasters. They were highly appreciative of the DRM training and seemed
well aware about things like the values of early warning system, contingency stock and
16

evacuation sites. The committee members demonstrated ownership about the work done by
ACF and expressed a willingness to continue it after ACFs withdrawal.
...We can vouch that ACF has
done excellent work with us,

better than any other NGO or
government agency and we
are very happy with its work
and find no problems in it.
Committee members
Charsadda village
in
Village disaster and contingency plans: Village disaster plans were seen hanging on the
walls in all the villages visited in Mardan, Charsadda and Nowshera although there was some
variation in the level of details contained by the plans across the villages. Thus, it would be
useful for ACF to develop a clear outline for the topics to be covered by such plans and ensure
that all villages complete all the topics. In Dir, the team had not yet engaged the villagers in any
village to complete such plans by the time of the evaluation which was undertaken during the
last two weeks of the project. The Dir team felt that they would still be able to complete the
plans in all the villages by December 31. However, clearly, even if they did so, it would seem to
be a rush, last moment job with little follow-up possible, which would affect the quality of the
whole exercise.
Early warning system: The early warning systems planned by ACF include linking the village
committees with governmental sources of information (by giving both parties the telephone
numbers of each other) about approaching hazards and also providing them with equipment,
such as megaphones and battery-operated loudspeakers for communicating the information
received from external sources within the villages. Most of the village committees had been
provided with the telephone numbers of relevant local authorities, such as police stations and
Civil Defence offices. However, the village committees had not been registered with government
departments so that those departments could proactively provide information to villagers. This
issue will be discussed in more detail under the sustainability section. Finally, due to some
delays in procurement, the equipment for intra-village dissemination of early warning systems
was being delivered in villages around the time of the evaluation. However, even so, such
systems would still be up and running before the 2013 flood season in July-August. However,
the success of ACFs EWS will of course depend on villages getting timely and accurate early
warning from government departments. Unfortunately, the technical and management capacity
of governments departments to provide such information is very low in Pakistan at the moment,
which is a major challenge for ACF. The final chapter will look at this issue in more detail.
17

Evacuation sites selection: In almost all villages, the committees had selected evacuation
sites in the village. Those sites generally consisted of schools, mosques or high, open places
within the village which were already being used as evacuation sites by villagers. In many of
these sites, ACF has built latrines. However, it would also be useful for ACF to help each village
select evacuation sites outside the village as well in case the floods are so severe that they
threaten to submerge the whole village, including the internal evacuation sites.
DRM construction training: These trainings had not been delivered yet in any district by the
time of the evaluation because of the delays in the project caused by late government
permission and procurement delays. According to subsequent information sent by ACF, these
trainings were completed before the end of the project.
Community DRM awareness-raising: These 3-hour sessions were supposed to be carried
out by the village committee members once they were trained by ACF. Such sessions were held
by committee in many villages. However, their efficacy was affected by two major reasons.
Firstly, in most of the villages, the committee members and in fact people in general did not
have the immediate capacity to become good trainers based on the 3-day training that they
received. Secondly, the population in some of the villages ran up to 3-4,000 persons. Thus,
assuming that in one session, the committee trained up to 100 villagers, it would still take it
around 20-25 sessions to train all adults and children above the age of 5. This obviously is not
an easy task for people who have full-time vocations and only work as volunteers on the
committee. These community trainings were supposed to be the main conduit for imparting
knowledge to the community members on DRM concepts and about the village disaster plans,
early warning systems, contingency stocks etc. As a result of these constraints, understanding
within the larger community about DRM concepts and the village DRM activities was found to be
variable across villages, districts and gender as shown in the table below which is based on
household interviews. The table reveals that while familiarity with the village committees and the
hardware work is very high (90+) across all districts and both sexes. However, familiarity within
the general community is much lower and highly uneven across the districts and sexes.
Familiarity in Dir is lower than in the other three districts. The most notable difference is across
the two sexes, with women reporting much greater familiarity with project activities than men,
which in some ways is counter-intuitive since usually Pakistani men have more opportunity to
participate in project activities due to the cultural conservatism. This difference may partially be
due to the fact that women are more village-bound and hence easier to attract to awarenessraising sessions and meetings. However, the fact that women were interviewed by internal ACF
staff while men were interviewed by the external evaluator could also have been a reason for
the higher positive responses reported by women. In any case, even if women are as well
informed as reported below, increasing the awareness among men would be crucial since
women enjoy much less freedom of movement and initiative in Pakistan during disasters and in
normal times. As mentioned earlier, spreading awareness across such large, illiterate and busy
communities is an enormous structural challenge faced not only by ACF but all agencies. Some
related recommendations are provided in the last chapter.
18

Community familiarity with ACF DRM project components
Are you familiar with:
(Percentage responding yes)
Charsadda
Nowshera
Mardan
Lwr
Dir
Male
Female
1. DRM village committee
98
98
97
95
94
100
2. Early warning system
75
61
73
50
38
94
3. Village disaster plan
94
74
73
30
48
85
4. DRM contingency stock
64
58
56
21
68
5. Evacuation sites
89
85
61
52
55
100
6. DRM training
95
91
71
73
69
94
7. DRM construction work
95
98
85
93
95
92
Overall, the communities generally felt that the project activities have benefited them in a variety
of ways, including increasing their self-reliance, ability to deal with crisis and physical dangers,
security of property knowledge about DRM and access to water and sanitation facilities. The
differences in responses across the four districts and the two sexes were minor with respect to
these issues. Communities mainly perceived these improvements due to their familiarity with the
establishment of the village committee, which they felt was a huge boon for the whole village,
and the visible DRM construction work that had occurred in all the villages and to a lesser extent
the other software activities with which their familiarity was lower as seen earlier.
Community perceptions about project impact
Has ACF DRM work
increased your:
(Percentage responding yes)
Charsadda
Nowshera
Mardan
Lwr
Dir
Male
Female
Ability to stand on own feet
96
98
97
92
94
98
Ability to deal with crisis
98
98
100
90
94
98
Safety from physical dangers
98
98
97
78
94
91
Security of property
98
95
97
80
93
90
Knowledge about DRM
92
88
78
75
69
100
Access to water and

sanitation services
96
98
97
95
95
98
4. Project Monitoring
a) How suitable and effective were the M&E systems in place through the duration of the
project and how could these have been improved? What systems were put in place to
ensure that outputs provided were of the highest quality possible and were
acceptable to beneficiaries?
19

ACF staff is regularly monitoring field activities, starting from field-level staff all the way up to the
Country Director. However, it would be useful for ACF to have formal monitoring plans for each
project which clearly specify the role for all relevant staff, starting from field-level staff all the way
up to the Country Director in the projects monitoring. This would include the frequency,
modality, project dimension and indicators for each relevant ACF staff. More importantly, the
plan would also specify how the information coming from the monitoring done by each staff will
be analyzed and summarized (preferably through a software-based monitoring application) and
then fed to progressively higher levels of the ACF Pakistan team, terminating with the Country
Coordination Team and what decisions and follow-up each level would undertake to ensure that
the project achieves its objectives on time. In this way, the team could constantly monitor the
progress on the common program dimensions (e.g., effectiveness, efficiency etc) that eventually
will be part of the project evaluation TORs (instead of being informed about shortcomings by
external evaluators) and take remedial timely action to ensure high project quality. While project
monitoring is clearly occurring, it is also true that there have been schedule slippages along
some project components, especially in Dir. The presence of such a formal plan would have
made it much easier to ensure timely achievement of objectives.
Another missing piece in ensuring quality and timely achievement of objectives is the institution
of strong accountability measures, which can enhance the ability of the ACF team to gain
information about project achievement beyond that coming from its own physical monitoring.
ACF Pakistan has now set up a Performance, Quality and Accountability unit in Islamabad and
plans to incorporate accountability mechanisms in future projects.
5. Project Sustainability
a) Were the construction material and masonry techniques applied appropriate (were
they disaster-resilient?) What alternatives would have been better?
Since there was no technical person on the evaluation team, the analysis of technical
construction quality undertaken as part of the evaluation was limited and was based on three
dimensions. Firstly, the external evaluator observed the construction quality visually for the
following common construction problems in all villages and only minor, occasional problems
were found:
Are there any cracks or plaster falls in the construction?
Have the fixtures and fittings (e.g., doors and windows) been fixed properly?
Dampness on walls/floor?
Is the toilet design ok?
Does the foundation look straight?
Are any hinges and bolts loose?
Secondly, the evaluator reviewed the construction-related documents. ACF has qualified
engineers within its DRM teams who had developed detailed Bills of Quantities for each
20

construction work. The evaluator requested the ACF team to share sample documentation for
some sites to show whether the BOQ material was actually delivered and used in the
construction and the required documentation from Nowshera, Charsadda and Mardan was
reviewed and found to be in order. Finally, the evaluator also checked with village committee
members about the technical quality of work who in some cases did have some background in
the small-scale construction work that ACF undertook as part of this project and they expressed
a high degree of satisfaction. Thus, overall, the construction quality seems sturdy, disasterresilient and best possible option under given conditions, though it would have been better for a
qualified technical person to physically observe the work and review construction documents. It
would also be good for ACF to undertake standard construction tests, e.g., brick quality, wood
quality, sand quality and retained weight of crush tests.
b) Evaluate whether the maintenance requirements of the structures were within the
capacity of the beneficiary users, if ACF did enough to reinforce these local
capacities. Evaluate the willingness of communities to maintain the infrastructure and
propose how community ownership could have been improved or reinforced.
The main mechanism used by ACF to ensure maintenance of both the software and hardware
work is the setting up of village committees. While in a few villages, the committees already
existed, in most villages the committees set up by ACF were the first experience of communities
with such committees. Committee members seemed motivated and keen to maintain the
committees even after ACFs withdrawal while communities saw the committees as an
extremely helpful form of social capital developed for them by ACF. So, the chances seem high
that most committees will survive even after the withdrawal of ACF and will maintain the
software and hardware work. However, given the high migration in some areas in KP, turnover
within committees could be high and it would be a test of their endurance whether villagers keep
refilling the vacancies. In this regard, some additional steps by ACF could increase the chances
of committee continuation. Firstly, while ACF has trained committees on DRM issues, it would
also be useful to give them some basic training on management, CBO operations and
development issues. Second, it would also be advisable for ACF to see whether there are other
international or local NGOs working long-term in these areas which could adopt the committees
after ACFs withdrawal. Third, it would also be helpful for ACF to link committees of nearby
villages with each other for mutual support, information sharing and learning during floods and
even normal times. Finally, it would be important to link these committees more strongly with
government structures, as discussed in the next section.
c) Evaluate whether links between VDMUs and authorities are sufficiently reinforced.
Evaluate the impact of the DRM training for local and district authorities. What were
the main issues/weaknesses and how could this have been improved? Were the local
disaster authorities (PDMA and DDMU) sufficiently involved at all levels of the
project? If not, in what aspects of the project could their involvement have been
improved?
ACF organized DRM trainings from relevant district-level departments in all four districts though
some of the training in Mardan was still outstanding at the time of the evaluation. The evaluator
21

interviewed district staff in three of the districts (except Nowshera) and all of them expressed a
high degree of appreciation and satisfaction with the training. With respect to involvement of
disaster authorities, all NGOs have to take official permissions from government authorities and
to that extent the government authorities were involved in the project. However, government
departments in Pakistan tend to be highly bureaucratic and it is not easy to get them involved to
any greater extent, beyond granting permissions, in NGO projects. Nevertheless, such
involvement would be crucial for the success of ACFs DRM project more so than for other
watsan or food security projects. Viewed so, it is felt that ACF should have built upon the good
will created by the government trainings and expended greater effort in increasing the
involvement of government departments in the project and enhancing the linkages between
them and its village committees in various ways despite the bureaucratic nature of Pakistani
government departments. To begin with, ACF could have got its village committees registered
with the Social Services Departments in district governments to enhance the chances of the
survival of these committees after ACFs withdrawal. Secondly, ACF could have advocated with
the DDMUs, preferably in coordination with other DRM NGOs) to treat the village committees
set up by ACF as valid structures for disseminating early warning information, and involving in
any government future grass-roots DRM initiatives. Thirdly, ACF could have followed up on the
trainings by trying to have some practical linkages between government departments and the
village committees, e.g., by arranging mock drills between the Civil Defence Departments
(which has grass-roots presence through a cadre of community-based volunteers), and
encouraging the government to initiate small-scale mitigation measures in ACF villages which it
could not afford itself.
6. Cross-Cutting Issues
a) How realistic and appropriate was the budget of the project? Was it respected?
The table above provides information on the original budget, actual % expenditure by October
2012 and expected % expenditure by project end. This information reveals that almost all major
cost budget lines are being respected and will be fully utilized by the end of the project with an
overall burn rate of 97%. However, there were some budget issues due to the exchange rate
fluctuations given that three currencies (Nok, USD and PKR) were involved. Overall, ACF lost
budget of around $100,000 due to the depreciation of the USD against the Nok despite the fact
that the PKR has also depreciated significantly against the USD. According to the ACF Pakistan
Finance team, this loss was covered by a budget modification where district-level contingency
stock items were dropped. Overall, the Finance team felt that the DRM team had done good
budget management which helped in the budget being respected.
ACF DRM Project Budget utilization

DESCRIPTION
Budget
(Nok)
% SPENT
(oct 12)
Estimated %
Spent , 31-1222

2012
Personnel
475,953
84%
98%
Expatriates
238,076
87%
98%
Local staff
237,877
82%
97%
4,900
4%
104%
Equipment Procurement
17,164
100%
100%
Equipment Hire
64,445
89%
108%
External Evaluation
20,000
0%
50%
Project Costs
597,098
76%
97%
Construction mitigation
458,165
83%
97%
Disaster preparedness in community
67,380
47%
125%
Capacity building
71,553
62%
69%
107,772
88%
97%
1,287,333
80%
97%
115,860
80%
97%
1,403,193
80%
97%
Communication, Visibility, Information
Other Costs
TOTAL DIRECT COSTS
Indirect Costs (9%)
TOTAL COSTS
b) Were ACFs rules on procurement adhered to? Have concerns relating to a zero
tolerance on corruption policy been adequately observed in the project
implementation?
ACF has a whistle-blower policy and an anti-corruption policy to reduce the chances of
corruption in the projects. Moreover, the approval limits of the Country Office and field offices
are much lower than that seen by the evaluator in other agencies with a view to reducing the
chances of corruption. The ACF Pakistan Finance team also diligently works to minimize the
chances of corruption by having a zero-tolerance policy against corruption so much so that a
driver was fired for a few hundred rupees violation. The team also calls additional suppliers by
telephone even where the logistics and field teams have sent in the three mandatory quotations
in order to further minimize the chances of fraud. All these steps ensure strong financial due
diligence and the evaluator did not come across any reports of corruption within villages or field
offices. However, this due diligence did slow up procurement to some extent with some of the
delays mentioned earlier being related to delays in procurement, which in turn were also
exacerbated by the lack of clarity among staff on procurement rules. ACF held a team workshop
recently to smoothen out these issues.
c) What measures were taken to ensure a gender balance at all stages of the project
(assessment, design and implementation)? How could this process have been
improved?
KP is an extremely conservative environment even by Pakistani standards where womens
rights are low and it is very difficult for NGOs to work with women to enhance their rights due to
23

the significant threat of violent backlash by militants who in recent weeks have killed scores of
female teachers and aid workers in KP. Within this background, ACF has still succeeded in
involving women in the project. Thus, all villages had separate womens committees who were
also provided DRM training like the male committees. Overall, womens understanding of the
project activities seemed higher than that of men, as mentioned earlier under the Community
DRM awareness-raising section. In many villages in Dir (which is more conservative than
Charsadda etc), women had formed the committees without the knowledge of men who did not
approve of such activities. In some Dir villages, ACF did not work with women because it was
not given access to women by the community men. However, given the severely conservative
culture and the serious threat of violent backlash, there are severe limitations on the extent to
which the involvement of women could be increased by ACF or other NGOs. There are greater
opportunities to do so in the districts other than Dir. Thus, men in these villages admitted that
they initially had some misgivings about allowing women to participate in the project as they had
this impression that foreign NGOs spread vulgarity among women and encourage them to rebel
against the local cultures. However, they also said that these misgivings had been largely
removed after the start of the project due to its positive impact and the fact that they saw ACF
staff constantly respecting local cultural values..
d) Evaluate the added value to the project from the partnership with the Centre for
Disaster Preparedness and Management (CDPM), and whether the involvement of
CDPM was maximised.
The CPDM is the only institution focused exclusively on DRM activities in Pakistan. ACF
requisitioned the CPDM to undertake the trainings for its own staff, the government departments
and the village committees. The trainings were highly appreciated by all the stakeholders and as
such the engagement with the CPDM seems like an excellent move by ACF to enhance and
work with local capacity. However, there could have been scope to increase the utilization of
CPDM services further since DRM is such a new area for ACF staff in Pakistan. While the
training was certainly good, given the limited initial background of ACF staff in DRM work, it may
have been useful to involve the CPDM at the field level to a limited extent, e.g., by having them
undertake the software component in one model village and monitoring ACFs work in an
additional 1-2 villages initially and providing feedback. This suggestion is based on the
observation that the ACF staff, especially in Dir, needed more capacity and support in
implementing the software component which was very new to them.
24
HAPTER 4: CONCLUSIONS AND

RECOMMENDATIONS
This chapter summarizes the main findings and conclusions from the last
chapter and provides recommendations for enhancing the quality of the project
further in the future along each dimension of improvement identified in the
conclusions section. The emphasis is on providing recommendations which
would be feasible within the difficult working environment faced by ACF in Pakistan.
1. Conclusions
a) Did the program reach the intended results and objectives? If not, what were the main
constraints faced by the organization and were they a consequence of poor quality of
the initial assessment, poor project design or poor implementation? Did the project
sufficiently identify and manage risk related to the project activities?
Overall project achiovements
Results and related indicators
Status at time of
evaluation
The ability to recognize hazards and cope with them is improved

80% of communities have functional local early warning systems
80% of target community can claim access to contingency stocks
90% of communities are covered by disaster management plans.
Work in progress at time

of evaluation; ACF later
confirmed completion by
end of project though
quality would have been
better with more spaced
out completion
The capacity to reduce the effect of hazards on communities is improved

Facilities serving 60 communities are structurally strengthened to
resist hazard events
Done
90% of communities have identified and equipped evacuation sites
Identified everywhere but

not all equipped due to
government
permission
issues
The understanding of DRM at community/institutional level is improved

80% of intervention communities have disaster management units
100% have units
240 individuals from local or district authorities receive DRM training
Higher number trained
25500 individuals in the communities receive DRM Training
Trained
but
variable
absorption of concepts
25

The table above provides the status on the three results and corresponding indicators at the
time of the evaluation field visits. Thus, the project is likely to achieve its objectives by
December 31, 2012 though the quality of the work could have been higher if so much work was
not squeezed into the last weeks of the project. These delays happened due to procurement
and government permission issues. Overall, the project is a very well-conceived one and aims
to fill a critical need in enhancing resilience within KP. Many components of the project have
been implemented very well, such as the setting up of the village committees and their training
and the completion of the construction activities. However, other components did have room for
improvement. These shortcomings were caused by a wide range of external and internal
constraints faced by ACF, including high insecurity, governmental bureaucracy, shortage of
skilled staff, stringent procurement rules, low literacy levels within communities and the
conservative culture in the country. Among these, ACF has managed security risks well and has
comprehensive and efficient security systems in place. However, more attention should have
been devoted to dealing with the risks associated with governmental bureaucracy, shortage of
skilled staff (both quantity and skill level), stringent procurement rules, low literacy levels within
communities and the conservative culture in the country. While the communities did not identify
major areas of improvements (beyond the standard requests for more hardware services in the
future), the evaluator himself has identified some. The next section provides recommendations
for improving the quality of the project further in these areas in the future.
2. Recommendations
Targeting: Targeting of the most vulnerable is excellent in Charsadda and Nowshera but is
less accurate in Mardan and Dir. Mardan has structural issues as very few people were affected
during the 2010 floods. In Lower Dir, a large number of people were affected, so the problem is
more about ensuring more thoroughly that ACF is focusing on the most vulnerable areas. It is
recommended to undertake a more thorough needs and security assessment in Dir. In general,
ACF is advised to focus on districts and villages which are vulnerable to major floods. To
identify such areas, the following three criteria are suggested: River and drainage channels
mapping; Damage incurred during 2010 floods and UNDP One DRM project list of most
vulnerable districts. Consider Sindh for DRM work too since it has recently become the most
repeatedly struck province in Pakistan.
Number of villages: Review the possibility of increasing the number of villages in the next
phase by increasing the number of field staff, covering each village with only one hardware
construction activity and linking with other NGOs and government agencies to cover some of
the construction work that ACF cannot afford in its budget.
Selection of hardware activities: Use the prevention-mitigation-capacity-evacuationrescue-relief continuum of DRM activities presented in Chapter 3 to select hardware activities.
The contingency stocks are an important hardware input given their relatively low cost and the
fact that they cover three categories in the continuum even though they are the three bottommost categories. Select other hardware construction activities from the highest possible
category of this continuum which is relevant and affordable. For example, wherever possible
26

prioritize drainage channels, irrigation channels and protection walls over evacuation routes and
latrines. Focus on community-level activities, .e.g., irrigation channels, rather than individual
household-level activities like provision of agricultural outputs given that the basic purpose of
the project is community joint work which may be undermined by helping certain families with
individual-level inputs and ignoring other families.
Village committee sustainability: Enhance the sustainability village committees by
providing them training on CBO management issues, having them adopted by other agencies
working long-term in the area (e.g., Concern/IRSP), linking nearby committees with each other
for mutual support and learning during crisis and normal times, getting committees registered
with the Departments of Social Welfare and getting them recognized by DDMAs as valid
structures for providing early warning information and inclusion in the governments DRM work
in coordination with other agencies.
Community DRM awareness-raising: Reaching to the larger community has proved
challenging due to the large size of the communities, busy schedules and low literacy levels.
This challenge could be reduced by having communities develop detailed community
awareness-raising schedules and following up more closely with committees about the
schedules and also looking at the possibility of training young educated, unemployed persons in
each UC to conduct sessions in all targeted villages in that UC for a small honorarium
Village disaster plans: Develop a clear formal outline of what each plan should contain and
then ensure that all villages closely follow and complete the plan according to this list to ensure
consistency across villages
Evacuation sites: Beyond the within-village evacuation sites identified now, also help
communities identify evacuation sites outside villages which could be accessed in case the
whole village gets submerged.
Monitoring and accountability: Develop formal monitoring plans which clearly specify the
role for all relevant staff in the projects monitoring, including the frequency, modality, project
dimension and indicators for each relevant ACF staff. More importantly, the plan would also
specify how the information coming from the monitoring done by each staff will be analyzed and
summarized (preferably through a software-based monitoring application) and then fed to
progressively higher levels of the ACF Pakistan team, terminating with the Country Coordination
Team and what decisions and follow-up each level would undertake to ensure that the project
achieves its objectives on time. The presence of such a formal plan would make it much easier
to ensure timely achievement of objectives and avoid slippages. Also institute strong and
responsive accountability measures in future projects.
27

Technical capacity/CPDM: Enhance technical inputs on DRM work by having CPDM provide
some degree of field-based support to ACF staff on software issues, e.g., by developing the
software package in one model village.
Government linkages: ACF should build upon the goodwill created by the government
trainings and expend greater effort in increasing the involvement of government departments in
the project. To begin with, ACF could get its village committees registered with the Social
Services Departments. Secondly, ACF should advocate with the DDMAs, preferably in
coordination with other DRM NGOs) to treat NGO village committees as valid structures for
disseminating early warning information, and involving in any government future grass-roots
DRM initiatives. Thirdly, ACF could try to have some practical linkages between government
departments and the village committees, e.g., by arranging mock drills between the Civil
Defense Departments, and encouraging the government to initiate small-scale mitigation
measures in ACF villages which it could not afford itself.
Coordination with other stakeholders on DRM: Having successfully completed one DRM
project, ACF should enhance its profile in this sector by increasing its visibility and inputs in
coordination sectors like the DRR Forum and the UNDP DRM project and obtaining greater
technical inputs from these forums.
28
APPENDIX
1. Damage in KP from 2010 Floods
KPK - Flood Affected Population

S#
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
District
Abbottabad
Bannu
Battagram
Bunair
Charsadda
Chitral
D.I.Khan
Dir Lower
Dir Upper
Hangu
Haripur
Karak
Kohat
Kohistan
Lakki
Malakand
Mansehra
Mardan
Nowshera
Peshawar
Shangla
Swabi
Swat
Tank
Grand Total
Total_Population
Total_
UCs
1169904
971930
451340
738496
1492939
465075
1234706
1131676
753313
459275
1007960
622919
774318
684004
715139
660046
1431208
2123149
1174961
2860402
632670
1494583
1834756
349373
25234142
Affected_UCs
Affected_Population
HH_Damage
Aftec_pop_pec
51
49
20
27
49
24
47
37
28
19
45
21
32
38
33
28
59
75
47
92
28
56
65
16
49
9
24
34
12
26
7
14
19
42
21
32
38
26
6
12
43
27
16
7
11
42
16
54,473
10,416
5,614
502,732
69,164
394,608
180,686
210,498
45,841
56,646
50,935
38,716
464,333
28,092
45,086
22,870
19,992
499,818
237,068
83,649
15,389
634,654
148,890
7,782
1,488
802
71,819
9,881
56,373
25,812
30,071
6,549
8,092
7,276
5,531
66,333
4,013
6,441
3,267
2,856
71,403
33,867
11,950
2,198
90,665
21,270
0
6
2
1
34
15
32
16
28
10
6
8
5
68
4
7
2
1
43
8
13
1
35
43
986
533
3820170
545739
16%
2. Scoring Matrix
Criteria
Impact
Rating (1
Rationale
low, 5 high)
4 The data in chapter 3 from household interviews clearly
shows that over 90% of the respondents felt that the project
helped improve their resilience due to the committees set up
and the construction work. These aspects include:
Ability to stand on own feet
Ability to deal with crisis
Safety from physical dangers
29

Security of property
Knowledge about DRM
Access to water and sanitation services
Sustainability
Coherence
Coverage
However, impact could be increased further by enhancing

understanding within the larger community about other
aspects of the project through more effective community
awareness-raising sessions. As shown in chapter 3,
understanding among men about other project components
like EWS, contingency stopcock etc is variable.
3 The committees set up are the main mechanism for
ensuring project sustainability. However, there is scope to
improve sustainability further by:
linking communities with other agencies, government
departments and with each other and
through further training of the committees on CBO
management issues
4.5 This is the strongest point of the project since the
combination of hardware and software activities cover
almost all categories of the DRM activities continuum
presented in chapter 3. Coherence could be improved
further by concentrating more on the higher categories of
the continuum in terms of hardware construction work, e.g.,
by doing more irrigation and drainage channels and
protection walls than evacuation routes and latrines
wherever feasible. The project is also in line with the
Pakistan governments Disaster Management strategy and
the grass-roots DRM approach of the DEC-funded NGOs,
whose DRM project the evaluator reviewed recently.
3.5 The villages in Charsadda and Nowshera are all clearly
among the most vulnerable villages in KP. However, there is
a need to enhance the coverage of the most vulnerable
villages further.by using the following criteria:
Basis for initial targeting should be natural hazard, with
floods being the most priority natural hazard to focus on
Relevance
Subsequent detailed targeting should be based on

collecting information about which districts, UCs and
villages are the most flood-prone with the following being
the main sources of information:
-River and drainage channels mapping
-Damage incurred during 2010 floods
-UNDP One DRM project list of vulnerable districts
4.0 Relevance is again a very strong point of the project given
that Pakistan has become increasingly vulnerable to major
30

disasters in recent years and very few agencies,
government or NGO, are doing much active work in DRM
though some of the project areas, e.g., Mardan, are not very
highly vulnerable.
3.5 Some project components have been implemented highly
effectively, including hardware construction work and many
software components in Nowshera, Charsadda and Mardan.
However, there were also much delay in completing many
project activities, especially in Dir which reduced the
effectiveness of the project.
4 ACF has managed to stay within budget limits till the end
through efficient project financial management achieved
through very close and regular coordination between the
Finance department and the Project Manager. The budget
itself reflects efficiency as it increases the resilience of more
than 50,000 persons at relatively low cost.
Effectiveness
Efficiency
3. Acronyms
BOQ
CBDRM
CBO
CDPM
DAC
DDMA
DRM
EWS
FGD
IDPs
KPK
NDMA
PCVA
RNE
UC
UNDP
VDMU
Bill of Quantities
Community-based Disaster Risk Management
Community-based Organization
Center for Disaster Prevention and Management
Development Assistance Committee
District Disaster Management Authority
Disaster Risk Management
Early Warning System
Focus Group Discussion
Internally Displaced Persons
Khyber-Pakhtunkhwa Province
National Disaster Management Authority
Participatory Community Vulnerability Analysis
Royal Norwegian Embassy
Union Council
United Nations Development Program
Village Disaster Management Unit
4. List of Interviews
Noor-Ul Amin
Abigail Chatagnon
Waheed-ul Haq
Dildar Hussain
Dr. Amir N Khan
Mr. Zahoor Khan
District Disaster Management Officer, Lower Dir

ACF Field Coordinator KP- Pakistan
Civil Defence Officer, Mardan
ACF-Pakistan, Deputy Project Manager, DRM
Director, CDPM University of Peshawar
District Liaison Officer Charsada
31

Clement Philat
Nick Radin
Zach Watson
Anwar Zeb
ACF Field Coordinator KP- Pakistan (ex)

ACF-USA, Water and Sanitation Advisor
ACF-Pakistan Water and Sanitation Advisor
ACF-Pakistan, Deputy Project Manager, DRM
5. List of Documents Reviewed

ACF Global Guidelines for DRR Mainstreaming
ACF Global DRM for Insecure Environments Briefing Paper
ACF Global DRM Policy
ACF PCVA Practitioner Manual
Bills of Quantities (various) for hardware activities
Deputy Project Managers Monthly reports (various)
DRR Forum Pakistan minutes of meetings (various)
PCVA reports for Charsadda, Lower Dir, Mardan and Nowshera
Project Manager Hand-over notes
Project Proposal and interim report, DRM Project
Training reports for government officials and VDMUs
Village Profiles (various)
6. Good practice case study

Village Profiles in ACF Pakistan DRM Project
Innovative Features & Key Characteristics
Based on Participatory Community Vulnerability Analysis (PCVA) exercises, ACF Pakistan
helps community members develop detailed Village Profiles for the villages that it finally selects
to be part of its DRM project. The Village Profiles include the following information:
Overall general village information
Geographic and demographic information
Accessibility and security
Socio-economic situation
Local resources and critical facilities
Other NGOs present
Hazard profile and seasonal risk calendar
DRM capacity analysis
Preparedness and response prospects
DRM needs identified
This is an innovative and useful exercise for the following reasons:
The exercise is led by and based on the perspective and knowledge of the local
community and is thus more likely to be accurate
It helps the community to view its vulnerability in a systematic fashion and also identify
32

its internal strengths as well as the gaps for which it would need external help. This
exercise then helps the community to develop Village Disasters Plans
It helps ACF to gain a better understanding of local hazard patterns and community
strengths and weaknesses. This information can then be used to fine tune the ACF
program in line with local realities.
Practical/Specific Recommendations for Roll Out
In order to ensure that ACF programs around the world get the maximum out of this exercise,
the following preparation and follow-up will be helpful
Ensure that all sections of the community, particularly women and other highly
vulnerable groups participate in the exercise
Ensure that the community has been adequately briefed about basic DRM concepts so
that it can participate more meaningfully in the exercise
Set realistic expectations about what ACF would be able to provide
Encourage local government officials to participate in some such exercises wherever
feasible so that they develop a better understanding of local vulnerabilities and villagers
develop closer linkages with them
Based on the information collected from several villages in a locality, develop overall
hazard profiles of different localities
33

Acf Pakistan DRM (Rne) Final Evaluation

Enviado por

Dados do documento

Título original

Direitos autorais

Formatos disponíveis

Compartilhar este documento

Compartilhar ou incorporar documento

Opções de compartilhamento

Você considera este documento útil?

Este conteúdo é inapropriado?

Direitos autorais:

Formatos disponíveis

Acf Pakistan DRM (Rne) Final Evaluation

Enviado por

Direitos autorais:

Formatos disponíveis

External Evaluation

Reducing Vulnerability through Disaster Risk

ACF Pakistan RNE DRM Project Evaluation, 2012

Chapter 1: Brief Background

Emergency Context in Pakistan

Chapter 4: Conclusions and Recommendations

ACF Pakistan RNE DRM Project Evaluation, 2012

Conclusions and Recommendations

ACF Pakistan RNE DRM Project Evaluation, 2012

ACF Pakistan RNE DRM Project Evaluation, 2012

This chapter provides an overview of the emergency context in Pakistan, and

1. Emergency Context in Pakistan

2. Overview of the Project

ACF Pakistan RNE DRM Project Evaluation, 2012

ACF Pakistan RNE DRM Project Evaluation, 2012

ACF Pakistan RNE DRM Project Evaluation, 2012

At the end of 2012, ACF commissioned an evaluation of the RNE funded

ACF Pakistan RNE DRM Project Evaluation, 2012

ACF Pakistan RNE DRM Project Evaluation, 2012

ACF Pakistan RNE DRM Project Evaluation, 2012

ACF Pakistan RNE DRM Project Evaluation, 2012

Recommended needs criteria for future DRM geographical targeting

ACF Pakistan RNE DRM Project Evaluation, 2012

Vulnerability Zones and ACF Project Sites in 4 KP Districts

ACF Pakistan RNE DRM Project Evaluation, 2012

ACF Pakistan RNE DRM Project Evaluation, 2012

Were you consulted about the types of services and

Did agency staff always treat you according to your

Were services provided keeping in mind your work

Were the services timely for you in light of your

ACF Pakistan RNE DRM Project Evaluation, 2012

To help communities Early warning systems; EWS,

To help rescue people Rescue

To provide life-saving Watsan,

ACF Pakistan RNE DRM Project Evaluation, 2012

ACF Pakistan RNE DRM Project Evaluation, 2012

...We can vouch that ACF has

done excellent work with us,

ACF Pakistan RNE DRM Project Evaluation, 2012

ACF Pakistan RNE DRM Project Evaluation, 2012

1. DRM village committee

2. Early warning system

3. Village disaster plan

4. DRM contingency stock

7. DRM construction work

Ability to stand on own feet

Ability to deal with crisis

Safety from physical dangers

Knowledge about DRM

Access to water and

ACF Pakistan RNE DRM Project Evaluation, 2012

ACF Pakistan RNE DRM Project Evaluation, 2012

ACF Pakistan RNE DRM Project Evaluation, 2012

ACF DRM Project Budget utilization

ACF Pakistan RNE DRM Project Evaluation, 2012

Disaster preparedness in community

Communication, Visibility, Information

ACF Pakistan RNE DRM Project Evaluation, 2012

ACF Pakistan RNE DRM Project Evaluation, 2012

HAPTER 4: CONCLUSIONS AND

The ability to recognize hazards and cope with them is improved

Work in progress at time

The capacity to reduce the effect of hazards on communities is improved