Terms of Reference (TOR)

Project End Evaluation

 

Project:	Emergency relief after floods in Umerkot and Khairpur (Drinking water and hygiene; primary health care; cash assistance), Pakistan
Donor	Humedica International-GFFO
Project Start Date:	01st July, 2023
Project End Date:	31st July, 2024

 

1. 1.     Introduction/Background
   Nationwide, the floods inundated an area of approximately 75, 000 square kilometres and caused enormous devastation: more than one million homes were destroyed, over two million homes were damaged; 22,000 schools were permanently damaged or destroyed. The harvest of over 2 million hectares of farmland was destroyed; 1.2 million farm animals died.  Over 1,700 people lost their lives. Approximately 80,000 births took place in the open. A total of about 33 million people were affected and 20 million were dependent on emergency aid. Millions of people lost their shelters and had to live for months in makeshift shelters, emergency tents and camps. Critical infrastructure (in addition to shelter, especially water supply, schools, health facilities, roads) was damaged or destroyed across the country. Enormous losses of crops and livestock were recorded. Damage to infrastructure and buildings is estimated at over 5 billion €, and damage to agriculture and food at over 3 billion €.

 

The two districts of Khairpur and Umerkot are among the hardest hit in the country and in Sindh province. Here alone, over 220,000 and 31,000 houses were destroyed, as well as the crops of over 80,000 ha and 20,000 ha, respectively. Umerkot district, as one of the most affected areas, was last classified as Phase IV "Emergency" in the IPC classification. Pak Mission Society is implementing Emergency relief after floods in Umerkot and Khairpur (Drinking water and hygiene; primary health care; cash assistance) with the collaboration of Humedica international-GFFO. The interventions of the project are designed to cope-up fully or partially the basic livelihood needs of flood-affected communities in Umerkot and Khairpur districts, Sindh, in terms of water supply, sanitation, health, nutrition, and protection by April 2024 (with an extension of 3 months) till July 2024.

 

District	Tehsil	Union Council	No of villages
Khairpur	Kot Digi Khairpur Thari Mirwah Kot Digi Kharirah Shadi Shaheed Malhar Khan		18
Umerkot	Kunri	Shakh Darelo Sher Khan Chandio	30

 

1. 2.   Project Outcome

Vital needs of flood affected communities of Umerkot and Khairpur districts, Sindh concerning Water and Sanitation, Nutrition, Health and Protection have been addressed by July 2024.

 

Project Outputs

1. Basic WASH infrastructure is brought back to use.
2. Basic health and nutrition needs of most vulnerable flood affected households in the target areas are covered.
3. Vital survival and recovery needs of vulnerable flood affected households in the target areas are covered through conditional & non-conditional cash grants.

 

1. 3.     Final Evaluation

The project has been implemented for overall 13 months since its commencement in July 2023. This is a mandatory final project evaluation required by the project donor – GFFO/Humedica International – for the purposes of assessing the effectiveness, relevance, efficiency, sustainability and impact of project initiatives. 

 

3.1 Scope

The evaluation will cover the entire project duration, from July 2023 to July 2024. Within this period, the evaluation will assess project outcomes at all levels: including result based management, beneficiaries and project stakeholders, Budget, inputs and resources. The impact assessment will be a comprehensive evaluation of the project's intervention in the targeted villages. It will delve into the overall impact and relevance of the project, analysing the effectiveness of project management and activity implementation in term of budget consumption and efficiency. The evaluation will closely examine the results achieved, coordination with local line departments, and the level of involvement of stakeholders/beneficiaries in the project interventions.

 

Furthermore, the assessment will also identify challenges faced and best practices utilized for future engagement. The review will include feedback from the target group or beneficiaries, representing a diverse group including males, females, children, vulnerable individuals, minorities, transgenders and the elderly from the target areas.

 

Overall, the impact assessment will provide valuable insights into the strengths and weaknesses of the project, enabling us to make informed decisions for future interventions and ensure sustainable positive impacts on the communities.

 

3.2 Objective of the study

The objectives of the evaluation are summarized below:

a)     To evaluate the project in terms of its effectiveness, relevance, efficiency, sustainability, and impact, with a priority on assessing the project expected results and outcome.

b)     To identify key lessons and potential practices for learning.

c)     To assess the challenges, best practices and document the outcome for future processes.

d)     Recommendations for similar future interventions as well as for new interventions in the targeted areas.

 

3.3 Evaluation Questions

The evaluation will be guided by following questions by keeping DAC criteria in focus.

 

Sr.#	Evaluation Criteria	Mandatory Evaluation Questions
1	Relevance	Was the project relevant to the identified needs of the target beneficiaries and the context? Were the project activity and strategy (including its assumptions and LFA) realistic, appropriate and adequate to achieve the results Have there been changes in the planned measures? Were some measures replaced by others or did some measures have to be intensified? Has the composition or size of the target group changed? Has the project region been expanded? Reduced? Diversified? Have the changes in the planned measures had an impact on the achievement of the objectives? Has it become clear during the course of the project that objectives have to be amended because they cannot be achieved as planned in the application? Have the measures perhaps been modified based on changed objectives?
2	Effectiveness	How appropriate were the management process in supporting delivery of project expected results? Were the project activities developed as per the development standards (humanitarian) and resilience needs?   How effective were the strategies and tools used in the implementation of the project? To what extent were the intended project outcomes and outputs achieved and how? To what extent did the project reach the targeted beneficiaries at the project goal and outcome levels? How many beneficiaries have been reached? To what extent has this project generated positive (or negative) changes in beneficiaries of targeted villages?
3	Efficiency	How far the results achieved did justified the cost incurred - were the resources effectively utilized? Did project activities overlap and duplicate other similar interventions (funded nationally and /or by other donors? Are there more efficient ways and means of delivering more and better results (outputs and outcomes) with the available inputs? Could a different approach have produced better results? How efficient and timely has this project been implemented and managed in accordance with the project proposal? How efficient is the record keeping, reporting and data management of the project activities?
4	Sustainability	Are there any plans and or strategies to sustain the gains after the project interventions? How effective were the exit strategies, and approaches to phase out assistance provided by the project? What are the key factors/areas that will require addition support/attention in order to improve prospects of sustainability of the project outcomes and the potential for replication of this approach? To what extent are the existence of institutions and their ability to function ensured even after the project has been completed? How will the acquired knowledge be passed on? Who will bear the follow-up costs (for example, personnel, repairs and maintenance costs)?
5	Impact	What are the unintended consequences as a result of the project activities? Are there any significant changes in the context as a result of the project intervention? Is the institution/facility regularly used by the target group? Are the committee structures (VDCs, O&M committees etc. are well supported and encouraged in their work by their social environment? If the project aspires to achieve outcomes at the intermediate (meso) and large scale (macro levels), has this already generated additional interest and been emulated by others? What is the reaction of local authorities and other state representatives?
6	Learning and Replicability	What are some of the key lesson learned as a result of this project that can be shared and replicated? What are the recommendations for similar future interventions? What are the recommendations for future interventions (for future project)? Which interventions were particularly popular or unpopular among the target group, and which ones encountered difficulties? Were there important/less important measures or key impacts for achieving the objectives?

 

7.4  Evaluation Methodology

The evaluation will combine qualitative and quantitative data collection and evaluation techniques using the following methods:

• Desk-base review: The evaluation will review relevant project documents and content produced before and during project implementation including the project proposal, work plans, project progress reports, annual project reports and other documents produced by or associated with the project
• KII Interviews: In addition to the desk review, the evaluation will also conduct interviews of key project stakeholders using a structured methodology developed by the consultant.

ü  External Stakeholders/Communities)

ü  PMS Staff/HO Staff(if required)

ü  Beneficiaries

ü  Line departments (if any-who were actively informed in the process)

ü  Post Distribution Monitoring Tool (PDM)

 

• Focus Group Discussions (FGDs): Data will also be collected using focus groups discussion with project beneficiaries and other project stakeholders using a pre-designed focus group discussion guide.

 

7.5  Evaluation Timeline

The evaluation is expected to start from 1^st July 2024 and will last for 25 days including fieldwork/interviews and report writing and submission. (Day wise schedule will be discussed after hiring of the consultant/firm.)

 

Sr.#	Deliverables	Detail	Timeline
1	Proposed work plan and tools (End line and PDM tool)	Proposed work plan and evaluation questionnaires and PDM tool to be submitted within 02 days following the official initiation of the evaluation.
2	Fieldwork including data collection	Training of enumerators, testing of tools and final validation Briefing, deployment of enumerators, data collection and post field work debriefing
3	Presentation of Data (excel sheets)	Lead Consultant to present analysed summary of field data before the writing the first draft report. This will allow for review, questioning and field follow up and cross validation of the data.
4	Draft of Evaluation report (End-line & PDM)	The evaluator must submit draft report for review and comments by all parties involved after analysis of the field data. Both the PMS management members and other related stakeholders in the evaluation must review the draft evaluation report to ensure that the evaluation meets the required quality criteria.
5	Dissemination workshop	Sharing the preliminary finding, data validation in stakeholder workshop
6	Final Report (End-line & PDM)	Report will be submitted 4-5 days after receiving comments from the PMS management members and other related stakeholders.

 

1. 8.     Competency Criteria:

Sr. No	Criteria	Score
1	Technical proposal: Understanding and approach (understanding of objective and results required; risk awareness and mitigation approach)	30%
2	Proposed Methodology: organization of the work (planning; proposed scheduling of activities; availability of required skills and quality assurance)	20%
3	Expertise in geographical area	10%
4	Expertise in thematic area	20%
5	Financial proposal	20%

 

1. 9.     Management and logistics

The Consultant report to MEAL HOD who will not have control over the professional work of the evaluators. The Project Manager along with MEAL office will provide technical guidance (guidance to reach targeted villages, stakeholders and key informants) for the evaluation to ensure the independence of the evaluation process. Logistical arrangements will not be provided by the PMS and is included in their (consultants’) budget.

 

1. 10.  Required Qualifications and Expertise for Lead Evaluator

The Evaluator shall have the following expertise and qualification:

ü  At least Master’s degree in Public Policy, International Development, Development Economics/Planning, Economics, International Relations/ Diplomacy or any other relevant university degree.

ü  Extensive expertise, knowledge, and experience in the field of aid effectiveness and aid management process related issues.

ü  Experience of project formulation and evaluation and practical experience of end of project evaluation in Sindh or in similar context; At least 08 years of experience in working with international organizations, donors and evaluating projects.

ü  Past experience in conducting surveys especially evaluations of German Donors funded projects. (Optional)

ü  Excellent written and verbal communication skills in English.

ü  Experience in Sindhi language will be added as an advantage.

ü  Offers will be evaluated on these criteria as well as the quality of the technical offer (proposed methodology, capacity to mobilise qualified personnel in the field, etc.) and the soundness of the financial offer.

 

1. 11.  Method of Application:

Interested, qualified and experienced person, group of persons or firms must submit an Expression of Interest dossier by 21^st June 2024 in a sealed envelope, outlining availability in line with the approximate timeline. The cover letter should be addressed to: Pak Mission Society (PMS), Operations department, House no 333, Street no 29, G-14/4 Islamabad.

 

The expression of interest can also be shared on [email protected]

 

The expression of interest should contain: (a) a technical offer and (b) a financial offer, comprising:

 

1. A.    Technical offer:

• Up to date CV of the lead consultant/evaluator (showing education and expertise).
• Technical proposition detailing proposed methodology and resources needed (max 3 pages).
• An example of a report from similar work which demonstrates evidence of the skills and experience required and a list of past evaluation produced by the lead consultant/evaluator.

1. B.    Financial offer:

• A list of all expenses expected to be incurred by the consultant including a daily rate