Statement of Work (SOW) for Final Evaluation

Building Institutional and Community Resilience (BICR)

Award Number: 720BHA22GR00273

1.    Background

Concern Worldwide is an international, non-governmental, humanitarian organization dedicated to the reduction of suffering and working towards the ultimate elimination of extreme poverty in the world’s poorest countries. Concern has been working in Pakistan since 2001, when it initiated an emergency response program to address the Afghan refugee crisis in Balochistan. Concern later moved into emergency, early recovery, and long-term development programming in Balochistan, Khyber Pakhtunkhwa (KP), Punjab and Sindh.

Concern has been implementing the USAID/Bureau of Humanitarian Assistance (BHA) – funded “Building Institutional and Community Resilience (BICR)” in Tharparkar and Umerkot districts of Sindh, Pakistan. The BICR project is designed based on the learning from USAID/BHA-funded Sindh Drought Resilience Program in Tharparkar and Umerkot districts of Sindh and capacity building of LNGOs and DMAs in Sindh and Balochistan under RAPID III. The main purpose of the project “to build institutional capacities (of government and national NGOs in KP and GB) and resilience of highly vulnerable communities in Sindh to recurring hazards through improved preparedness and response capacities”.  The program is in line with the national DRR policy and USAID/BHA’s priorities aimed at assisting in early recovery and building resilience of institutions and drought-affected communities, which are extremely vulnerable and highly exposed to drought risk.

With the support of USAID/BHA, Concern is providing assistance to the local NGOs and DMAs in vulnerable districts in KP and Gilgit Baltistan for the institutional capacity building and drought-affected populations in Umerkot and Tharparkar districts in Sindh for community level resilience building with the following key objectives/sub-purposes:

1. i.      Institutional Capacity Building – To enhance institution and disaster response capacity of government officials, disaster management authorities (DMAs) and local NGOs in Gilgit-Baltistan, tribal and other vulnerable districts in Khyber Pakhtunkhwa Province – Pakistan.
2. ii.    Community-level Resilience Building – To build the resilience of drought-affected communities and structures in Sindh through Community-Based Disaster Risk Management (CBDRM).

This Statement of Work (SOW) aims to provide a roadmap for the external evaluation of the “Building Institutional and Community Resilience” at the end of the project and will be revised based on the feedback of USAID/BHA.

2. Purpose and Objectives of the Evaluation

The overall purpose of the summative evaluation is to evaluate the Building of Institutional and Community Resilience to the extent to which the program met the objectives as outlined in the program proposal, with particular emphasis on DAC criteria such as appropriateness/relevance, coherence, coverage, efficiency, effectiveness, impact, connectedness and sustainability of the interventions carried out and approaches adopted. This will allow the consultant to extract lessons learned and recommendations in relation to operational and programming aspects. This evaluation should yield lessons learnt from the perspective of Concern, USAID/BHA overall resilience-focused program as well as National Disaster Management Authority priorities.  It should capture achievements of the program’s results and indicators, and the initial impact of the action in the light of theory of change and M&E plan. The overall purpose of this summative evaluation;

-          To evaluate the project on DAC criteria and generate lesson learned in the perspective of Concern, BHA, NDMA and other relevant stakeholders.

-          To what extent the activity/project met its objectives of building the resilience of DMAs, NGOs.

Considering DAC criteria for program evaluation, which includes relevance, coherence, efficiency, effectiveness, impact and sustainability. This exercise will also explore the questions related to coordination and cross-cutting themes such as human resources; protection; participation of primary stakeholders; coping strategies and resilience; gender equality; HIV/AIDS; and the environment and accountability to the affected people. The specific objectives of this evaluation are to:

• Assess the relevance, coherence, efficiency, effectiveness, impact, connectedness and sustainability of the program.
• Assess the coherence of the DRR component and its cross cutting links with other sectoral interventions, including an assessment of the connectedness, partnership and coordination mechanisms.
• Assess the success and impact of participatory approaches used in program implementation.
• Identify lessons to be learned to inform the future program i.e. formation of local community structures at various levels, their linkages, initiatives taken, institutional capacity building of NGOs and DMAS
• Recommend practical recommendations for replication of the program in different contexts of the country (recommendations need to be specific, practical/feasible and achievable).

Appropriateness of program design

• To what extent does the program respond to particular needs, expectations and priorities of the DMAs, LNGOs, and target communities, local authorities in strengthening community and institutional level resilience and disaster response capacities?
• How much is the program aligned with the National, Provincial and District Disaster Risk Reduction priorities, principals of inclusions, gender equality?
• How relevant was the project activities and strategies in addressing the root causes of poverty?

Relevance:

• Were beneficiary needs considered sufficiently when planning the responses?
• Was there a clear decision-making process in place when designing the programme?
• Are there any examples of adaptations to the evolving context or lessons learned? Can they be considered as successful?

Efficiency in the use of resources

• What was the level of efficiency (outputs and outcomes being achieved at an acceptable cost compared to alternative approaches) and timely delivery (adherence to planed implementation schedule) of the goods or and services?
• Could the same or better results have been achieved with same or lower inputs or by doing things differently?

Effectiveness of program interventions

• Did the activities achieve satisfactory results in relation to stated objectives/results? How did the program perform against the log frame indicators?
• Were there any unintended consequences/outcomes/impacts?
• To what extent have the activities contributed to enhancing local communities resilience and how?
• What is the effectiveness of a) drought mitigation measures and b) Early Warning System c) Institutional capacity building of NGO and DMAs?
• To what extent has the program met its stated objectives.

Sustainability/ Connectedness

• How well Concern implemented the exit strategy and ensured that the outcomes/impact resulted by the project sustained once the award complete? Is the sustainability strategy fully understood by the partners and therefore reflected in their implementation plans? How the exit strategy was managed at the end of the funding period?
• How has the project improved the knowledge and skills of target households/individuals in terms of crop production, livestock, livelihood and DRR? How well the households/individuals retained and utilized the improved knowledge and skills?
• To what extent are the benefits of the program likely to continue after the completion of the program? Were the various installations/infrastructures properly handed over to the relevant stakeholders? If not, why not?
• How the communities and other stakeholder will ensure the operation and maintenance and use of the provided equipment/infrastructures?
• What are the major factors, which influenced the achievement or non-achievement of sustainability of the program? How will the demonstration of various approaches and techniques work for replication at a considerable scale?

Coverage

• Who was supported by humanitarian action, and why.
• Have we reached major drought-affected communities, which are extremely vulnerable and exposed to drought risk?
• Was the project reached to the districts in KP and GB that are prone to disasters and lacking the required capacity? How effective was the targeting process in reaching the disaster-prone districts and communities for capacity-building activities? Was the project ensured the participation and consultation of NDMA, PDMA and other relevant authorities in targeting the DMAs and NGOs?

Coordination/ Coherence/Participation of primary stakeholders

• How well the program ensured community participation in the decision making? How effective was the Compliant Response Mechanism (CRM) in receiving and responding to community feedback and complaints and maintaining coordination with the community?
• How well did the program coordinate with Government line departments and NGOS coordination forums such as PHF/NHN during design and implementation of program? To what extent has this cooperation resulted in better programming? How they will play their role in sustainability of program interventions e.g. planning and responding to emergencies, DRM plans, EWEA, Early Warning System, agriculture, livestock etc. 

Impact

• To what extent the program achieved its stated objectives at Output, Intermediate Result (IR), and Sub-Purpose and Purpose levels? What was the performance against the indicators at different levels in the log frame?
• To what extent has the program contributed to building the institutional capacity of NGOs and DMAs to effectively plan and respond to emergencies?
• To what extent has the program contributed to the resilience of communities (improving food security and livelihoods of the target drought-affected communities as well as access to hygiene practices)
• How has this program contributed to communities’ resilience for disasters? How has the program contributed towards preparing communities to cope and withstand in disasters?
• How has the program contributed towards gender equality and empowering women to take an active part in the disaster risk management activities? 
• What changes - intended and unintended, positive and negative - were experienced by the targeted beneficiaries (communities, LNGOs, DMAS) and other stakeholders? What factors appear to facilitate or inhibit these changes?

Integration of Cross Cutting Issues

• How well did the program integrate gender equality, HIV & AIDS, disability and environment?
• To what extent was ‘accountability to beneficiaries’ promoted and the progress made against the achievement of Core Humanitarian Standard (CHS) improvement plans? To what extent were complaints addressed?
• How has the institutionalization of drought mitigation interventions of the program contributed to the overall community resilience?
• What opportunities the local government system offers to this program and how best the interventions can benefit from those opportunities?

Replicability

• What aspects of the program are replicable in similar areas in Pakistan and elsewhere?
• Under what circumstances and/or in what contexts would the program be replicable?

Lessons learned, information sharing and dissemination

• Were there any significant changes in the program design or the program context? What were the reasons for these and can any useful lessons is learned from this for application elsewhere?
• Based on the lessons learned verify a causal relationship between program and observed changes and estimate its contribution to the change.

Recommendations

• Targeted recommendations for (a. Concern and implementing partners, b. government and c. donors) for improvements based on findings during the evaluation process (e.g. for sustainability, future program design).

3. Methodology

The evaluation process should be comprised of a mixed model approach to collect both quantitative and qualitative data. The process should include:

• A desk review of program key documents listed below
• Primary data collection through HH survey from sampled project beneficiaries to assess the effectiveness, efficiency and impact of program activities at household level and community levels.
• Focus Group Discussions (FGDs) with beneficiaries, stratified based on nature of different program’s activities, to allow easier triangulation of data with other findings. The process should be as participatory as possible.
• Key Informant Interviews (KIIs) with project staff (Concern and partners) and stakeholders including government departments (Agriculture, Livestock, District Administration, AZRI, DDMA, PDMA, GBDMA and NDMA), capacity building partner NGOs, and other stakeholders such as PHF, NGN to collect information on achievements and impact and difficulties faced.
• Meetings with the senior management of local NGO partners to document their perception about change in their organizational capacities as a result of this project.
• Presentation of draft findings to a) field/partners’ staff and b) Islamabad-based staff.  These feedback sessions will help finalize the conclusions for the report.
• Submission of the draft evaluation report to Concern for feedback. Incorporation of Concern’s feedback in the draft report. This should have a three-page executive summary designed as a stand-alone document as well.
• Submission of the final report following comments received from Concern on the draft report within the agreed timeframe.
• The evaluation consultant should consider the DAC criteria and cross cutting issues and provide an appraisal of how well the, intervention has fared against each using the following grading scale, where:

4. Deliverables

There will be two deliverables, an inception report and the final evaluation report.

Inception Report: the inception report will be submitted for approval prior to data collection, and should contain several key sections:

• List of secondary data and documents provided by Concern for review
• Methodology and sampling approach, including disaggregated sampling targets, and any anticipated limitations
• Updated work plan, including dates in which data collection will be conducted, timeline for preparation of report, and including persons responsible for managing listed tasks

Note: The inception report should not exceed 15 pages of text, excluding tables, infographics and maps.

Final Report: The evaluator(s) will produce/submit a report in hard and soft form along with relevant annexes (in Microsoft Word using Times New Roman font 12). The report should include:

• Basic information (1 page)
• Executive summary (max. 3 pages)
• Introduction/background of the project (1 page)
• Evaluation methodology
• Major findings against the objectives of the evaluation (i.e. serial number 2 above) coupled with an appraisal and scoring each of the DAC criteria on a  scale of 0-4
• Summary of targeted recommendations/lessons
• Success stories/case studies
• Annexes: Evaluation ToRs, evaluation schedule, list of persons interviewed and sites visited, documents consulted declaration of independence from the project team, data collection tools and raw data, and the updated program M&E Plan/Indicator Table with achievements against each indicator.

Note: Note: The final report must have a stand-alone Executive Summary not exceeding on 3 pages. The main evaluation report (without annexes) will not exceed 30 pages to keep the audience straight to the main point.

5. Documents to be shared with the evaluators

• The approved project proposals, relevant policies
• Baseline report, knowledge retention survey reports
• LNGOs and DMAs capacity assessment reports
• Records of Institutional Support provided to LNGOs and DMAs
• NDMA’s vulnerability drought assessment report
• UCs Level DRR Plans
• Program reports, including financial information
• Survey reports (knowledge retention, effectiveness of early warning system)
• Reports/ policies of government departments and other stakeholders
• Concern’s HCUEP (How Concern Understands Extreme Poverty) document
• Concern’s PME guidelines and Strategic Plan and other related policies/documents. 
• Any other relevant document

6. Duration

The evaluator will complete the work over a period of thirty (30) working days in the months of October and November 2023 (please note the start/end date of the assignment may change due to unavoidable circumstances in which case a revised timeframe will be drawn up with the mutual agreement of both parties) at the date of signature of the contract and ending with the acceptance of the final report. However, the evaluator will complete all the evaluation activities during the award period (No evaluation activity will be planned after the end date of project).

7. Reporting Line

The evaluator will report to Concern’s Director of Program and will liaise closely with the MER Specialist and Program Manager Sindh and his team and partner staff.   

8. Evaluators’ Expertise

• Advanced University degree (at Master’s degree level or higher) in Disaster Risk Reduction, Climate change, Environment, Research and/or related field
• Proven knowledge of development sector particularly in the fields of community resilience along with knowledge of financial and economic analysis
• At least 5-8 years of experience conducting evaluations of development projects, especially resilience and drought mitigation related projects
• Experience in the use of participatory methodologies and developing gender-sensitive evaluation methodologies
• Familiarity with DAC criteria for evaluating development assistance
• Excellent analytical, facilitation and reporting skills
• Knowledge of the Government of Pakistan’s DRM policies, frameworks and architecture
• Good understanding of the local context, knowledge, culture and languages will be an advantage.

While applying, please note that the evaluator will be assessed on the following areas:

• Understanding the context
• Technical approach
• Overall budget costs, and
• Proposed evaluation team

Safeguarding at Concern: Code of Conduct and its Associated Policies

Concern has an organisational Code of Conduct (CCoC) with three Associated Policies; the Programme Participant Protection Policy (P4), the Child Safeguarding Policy and the Anti-Trafficking in Persons Policy. These have been developed to ensure the maximum protection of programme participants from exploitation, and to clarify the responsibilities of Concern staff, consultants, visitors to the programme and partner organisation, and the standards of behaviour expected of them. In this context, staff have a responsibility to the organisation to strive for, and maintain, the highest standards in the day-to-day conduct in their workplace in accordance with Concern’s core values and mission. Any candidate offered a consultancy opportunity with Concern Worldwide will be expected to sign the Concern Code of Conduct and Associated Policies as an appendix to their consultancy contract. By signing the Concern Code of Conduct, candidates acknowledge that they have understood the content of both the Concern Code of Conduct and the Associated Policies and agree to conduct themselves in accordance with the provisions of these policies. Additionally, Concern is committed to the safeguarding and protection of vulnerable adults and children in our work. We will do everything possible to ensure that only those who are suitable to do consultancy work or volunteer with vulnerable adults and children are recruited by us for such roles. Subsequently, being hired as a consultant with Concern is subject to a range of vetting checks, including criminal background checking.