Terms of Reference

End of project evaluation: Envisioning and Catalysing PATS Plus for SDG 6 in Pakistan

Assignment Information

1. About WaterAid 

WaterAid is an international not-for-profit, determined to make clean water, decent toilets, and good hygiene normal for everyone, everywhere within a generation. We have been working with government and non-governmental partners in Pakistan since 1992 to ensure that these essential services are available to poor and marginalized communities, especially women, persons with disabilities etc. WaterAid employs a public health and climate adaptation approach to addressing current water, sanitation and hygiene (WASH) service requirements and sustaining them for the future. Hygiene behaviour change, menstrual hygiene, rural and urban sanitation and safe water systems, WASH in institutions (healthcare facilities, schools, public places) and nature-based solutions for water conservation, groundwater recharge and waste-water treatment are some of the interventions used for achieving our objectives. We also work with sector partners to support and influence the behavioural, institutional and policy mechanisms for accelerating progress toward achieving safe and sustainable WASH access for the poor and marginalized communities.

     For more information, please visit (www.wateraid.org) 

2. Project Description

“Envisioning and catalysing PATS Plus for SDG6” is a 3-year project managed by WaterAid Pakistan in Taluka Badin, Badin District (Sindh) and implemented through a local partner, LHDP (Laar Humanitarian and Development Programme).  The project began on 1st June 2022 but has been granted a six month no-cost extension, and so is now due for completion on 30th November 2025.

This project focuses on strengthening the institutional capacity of government departments to ensure the provision of sustainable, inclusive, and climate-resilient water, sanitation, and hygiene (WASH) services in Taluka Badin, one of the most underserved and disaster-prone areas in Sindh, Pakistan. By addressing critical gaps in WASH coverage, it aims to reduce systemic barriers and contribute to achieving Sustainable Development Goal 6 (SDG 6). The project adopts a community-driven approach, focusing on improving WASH service delivery, empowering local institutions, and promoting gender-responsive practices. It works closely with government departments to integrate WASH improvements into planning and policy frameworks, ensuring long-term sustainability and scalability. 

The project adopted the Pakistan Approach to Total Sanitation (PATS), and aimed to demonstrate small scale decentralised wastewater treatment solutions as a way to move up the sanitation ladder to reach safely managed (termed as PATS Plus), The project has five outcomes, as follows:

Outcome 1:

163,500 population of Taluka Badin are living in Open Defecation Free (ODF) environment and are maintaining this through using household latrines and hygienic behaviours and practices.

Outcome 2: 

Communities and institutions in Taluka Badin benefit from safely managed inclusive WASH services by adopting approaches of PATS plus and community management models.

Outcome 3: 

Demonstrated models of community-managed, climate-resilient and sustainable safe drinking water services for replication and uptake.

Outcome 4: 

Strengthened capacity, processes, and systems of relevant government departments to plan, deliver and sustain inclusive, climate-resilient and gender-responsive WASH services. 

Outcome 5:

. Evidence base is enhanced to support and promote wider scaling of WASH models from the district 

3. Project Achievements/progress:

The project was implemented in Badin taluka, Badin District.  Project achievements are summarised as follows:

• 410 communities declared Open Defecation Free (ODF), meeting the project target for ODF status.

• 12,741 households constructed latrines, significantly improving sanitation access across the region.

• 187 Community Resource Persons (CRPs) trained in PATS and climate-resilient, gender-responsive WASH practices, meeting the training target.

• 9 quarterly WASH Forum meetings held at the district level to discuss sanitation, hygiene, and water access issues.

• 20 schools demonstrated inclusive WASH models, meeting the target for institutional WASH improvements.

• 7 healthcare facilities (HCFs) demonstrated inclusive WASH models, exceeding the target of 5.

• 40 hand pumps built or rehabilitated to improve access to safe water for local communities.

• 5 water filtration systems rehabilitated or constructed, meeting the project target for clean water supply.

• 5 Small Women Entrepreneurship Units/ Shops established to support local women’s economic empowerment.

• 268,989 community members reached through hygiene promotion activities, improving hygiene behaviours.

• 14,527 school children reached through hygiene promotion efforts within schools.

• 1,760 individuals in healthcare facilities reached with hygiene promotion activities.

• 21,601 community members reached with water access improvements.

• 10,526 students reached through water access improvements in schools (for sanitation purposes).

• 41,500 patients in healthcare facilities reached through water access improvements (for sanitation purposes).

Some activities were adjusted during the course of the project.  These changes were discussed and documented in the mid-project review (November 2024) and will be shared with the appointed consultant.

4. Scope and Purpose of the End-line Evaluation: 

The primary purpose of this assignment is to conduct a comprehensive evaluation of the "Envisioning and Catalysing PATS Plus for SDG 6 in Pakistan" project, focusing on both the achievement of its intended outcomes and its overall performance. The evaluation aims to generate findings that are grounded in robust evidence, are credible, and offer practical value for stakeholders. These insights will be instrumental in shaping future planning and decision-making processes, particularly by integrating lessons learned into the design and implementation of similar WASH projects in the future.

The evaluation will specifically:

• Evaluate the extent to which the project achieved its intended outcomes, such as sustained (ODF) status, behaviour change, and strengthen government system and the contribution of completed activities to those outcomes.”

• Evaluate the project against the criteria of relevance, efficiency, effectiveness, impact, and sustainability, taking into account the specific context and unique challenges of Taluka Badin.

• Identify and analyse the strengths and weaknesses of the project’s approach and implementation, including partnerships and how the project team adapted to changes in context, constraints and challenges faced during the implementation period,

• Evaluate and assess how successfully the project strengthened the capacity, processes, and systems of government departments to plan, deliver, and sustain inclusive, climate-resilient WASH services. This includes examining improvements in institutional coordination, resource allocation, technical competencies, and the integration of climate resilience and inclusivity into WASH planning and implementation

• Highlight lessons learned, best practices, and recommendations to inform future WASH programming, particularly in rural and semi-urban settlements, with a focus on replicating successful models across other districts in Pakistan.

5. Evaluation Criteria and Questions:

The evaluation exercise will be guided by the specified evaluation criteria and their corresponding questions. During the inception phase, the consultant may, in coordination with WaterAid Pakistan's Programme Quality team, review and revise the questions as deemed appropriate; however, the evaluation criteria themselves shall remain unchanged.

1. Contextual Relevance:

This criterion examines the appropriateness of the project’s objectives in relation to the issues it aimed to address and the context within which it operated. The following questions should be answered:

• How well does the intervention design and approach address the identified problems related to WASH in Taluka Badin?

• To what extent is the intervention tailored to the local context and the needs of the communities, including marginalized and vulnerable groups?

• To what extent are the activities and outputs aligned with the expected outcomes, particularly in terms of attaining Open Defecation Free (ODF) status, promoting hygiene behaviour change, strengthening systems, and enhancing the effectiveness of community-managed water supply models?

2. Results 

The consultant will be expected to prioritise assessment of outcomes — such as sustained changes in WASH practices and system strengthening — over basic activity/output tracking. Where feasible and relevant, review existing health data from local facilities as supportive evidence of improvements in hygiene and sanitation outcomes. The following questions should be answered:

• To what extent have the outcomes been achieved, including, Open Defecation Free (ODF) status, hygiene behaviour change, community-managed water supply, institutional capacity strengthening, and evidence generation for scaling. 

• How effectively has adaptive management been applied to ensure that the desired results were achieved?

• What were the major factors that influenced the achievement or non-achievement of the project’s outputs and outcomes?

3. Quality

This criterion assesses the extent to which the intervention meets WaterAid's quality programme standards for projects and programmes. The following questions should be answered:

• the extent to which the project aligns with WaterAid’s Quality Programme Standards (QPS), including the full set of 32 standards. While not all standards may apply, the consultant should review and analyse those that are relevant to this project’s context. How well does the intervention align with WaterAid's project quality standards and any contextual adjustments made for Taluka Badin?

• How can the quality of the intervention be improved, particularly in terms of systems strengthening and sustainability?

4. Value for money:

Value for money measures the extent to which the project’s results have been achieved in an efficient, cost-effective manner. This includes examining the economy, efficiency, and effectiveness of the intervention. The following questions should be answered:

• Was the project managed efficiently, in terms of human, financial, and other resources, relative to the results achieved in Taluka Badin?

• Did the adopted approach effectively contribute to the desired results, particularly in achieving ODF and improving WASH services in communities and institutions?

• What alternative approaches could have been employed to enhance effectiveness or reduce costs while achieving the desired impact?

5. Transformative Change

This criterion evaluates the potential for transformative change in terms of sustainability, scalability, and the catalytic effect of the intervention. The following questions should be answered:

• To what extent did the intervention contribute to achieving lasting changes in the WASH sector, particularly in the context of climate resilience and gender responsiveness?

• What were the major factors influencing the achievement or non-achievement of transformative change in the communities and institutions?

• How can the intervention’s scalability be enhanced to replicate successful models in other districts or regions?

6. Suggested Methodology: 

WA Pakistan suggests employing a mixed-methods approach to gather relevant data, combining both quantitative and qualitative techniques to evaluate the project’s performance. The consultant will finalize the methodology, which will be approved during the inception phase. This will involve field visits and interviews with key stakeholders, including beneficiaries and government officials, to gather necessary data.

1. Data Collection Methods:

The data collection will involve both quantitative and qualitative approaches to assess the project's performance. Quantitative methods will focus on measurable outputs, while qualitative methods will provide deeper insights into stakeholder experiences, ensuring a comprehensive evaluation of the project’s impact.

Quantitative Methods:

• Surveys: Administer structured surveys to beneficiaries and stakeholders to collect data on project outputs, such as ODF status, hygiene practices, and community-managed water systems. 

• Health Data Review: Gather health data from local health facilities to assess improvements in waterborne disease rates, which could be indicative of the project’s impact on sanitation and hygiene.

• Routine Monitoring Data: Use existing project monitoring data (e.g., number of households achieving ODF status, number of water systems functioning) to track progress against targets.

Note that a separate household survey will be conducted in parallel with this project evaluation.  Planning and conducting this household survey is outside the scope of work of this agreement – however, if the results are available in time, then the key findings should be incorporated within the overall project evaluation.

Qualitative Methods:

• Key Informant Interviews (KIIs): Conduct (6-10) semi-structured interviews with local government officials (including PHED, health and education departments), staff from schools and healthcare facilities, WaterAid and LHDP project staff, Women-led oversight committee, and community leaders to gather insights into the implementation process, challenges faced, and perceived project effectiveness. The specific government levels (Taluka, District, and/or Provincial) will be confirmed during the inception phase in consultation with WaterAid Pakistan’s technical team.

• Focus Group Discussions (FGDs): Hold (6-10FGDs) discussions with community members, including vulnerable groups, to understand their experiences, perceptions, and any changes in hygiene behaviour, sanitation practices, and water management.

• Document Review: Analyze project reports, government planning documents, and other relevant materials to assess the project’s alignment with local and national priorities, and the extent of its integration into government systems.

Field Observations:

• Conduct field visits to directly observe the status of infrastructure (e.g., water systems, sanitation facilities), community engagement, and operational aspects of the project. This helps confirm whether reported data aligns with on-the-ground realities.

Sampling Strategy:

• Sampling for Surveys: Use stratified random sampling to ensure a representative sample from different segments of the community (e.g., by gender, age, or geographic location) and to cover the diversity of beneficiaries and stakeholders. The sample will be designed to achieve a confidence level of 95%, with a margin of error of ±5%. A separate household endline survey is being conducted independently and is not part of this evaluation assignment. If the survey results are available in time, the consultant should review and integrate key findings into the final evaluation report.”

• Sampling for KIIs and FGDs: Employ purposive sampling to target individuals with relevant experience or insights into the project’s outcomes and challenges (e.g., project staff, local government, community leaders, and beneficiaries).

2. Data Analysis:

The proposed methodology for data analysis is as follows:

Quantitative Analysis:

• Descriptive Statistics: Calculate basic metrics (e.g., percentages, averages) for key outcomes (e.g., ODF status, improvements in hygiene, water access).

• quantitative Analysis:  Quantitative analysis will rely on on  descriptive statistics drawn from existing monitoring data, available reports, and findings from other assessments where applicable. In the absence of a formal baseline, analysis will focus on endline performance relative to project targets and expected outcomes.

Qualitative Analysis:

• Thematic Analysis: Code and categorize qualitative data from KIIs and FGDs to identify recurring themes, patterns, and insights that explain project outcomes.

• Content Analysis: Analyze documents (e.g., project reports, policy documents) to assess alignment with objectives and to track progress on institutional capacity building and climate resilience integration.

Triangulation:

• Data Triangulation: Combine data from different sources (e.g., surveys, interviews, field observations) to cross-verify and validate findings. This ensures the reliability of the conclusions drawn from the evaluation.

• Methodological Triangulation: Use both quantitative and qualitative methods to corroborate findings and gain a fuller understanding of the project’s impacts and challenges.

7. Deliverables  

The consultant is expected to produce the following:

1.  A detailed inception report outlining the proposed methodology and approach for the evaluation, including a work plan with clear timelines for each phase of the assignment. (An inception meeting will be held with WaterAid Pakistan’s project team and technical advisors to align on expectations, methodology, and scope prior to finalizing tools and beginning data collection.)

2. Customized and contextually appropriate data collection and analysis tools for the endline evaluation, including survey questionnaires, interview guides, and observation checklists, tailored to the project’s specific indicators and objectives That will be reviewed by WaterAid FCP team, Program Quality team and technical advisors.

3. Copies of all completed survey questionnaires, consent forms, and qualitative notes from focus group discussions, key informant interviews, and any other data collection activities. These should be submitted in hard copy format.

4. Raw and clean datasets in Excel format, including cleaned and anonymized data, along with necessary documentation or codebooks to assist in understanding the dataset.

5. Presentation of key findings and recommendations to WaterAid Pakistan’s Senior Management Team (SMT) and the FCP project team. This session will provide an opportunity for feedback, clarification, and alignment before finalising the evaluation report.

6. A comprehensive final evaluation report collected data, clearly presenting key findings conclusion and recommendations. A draft report must be submitted (with a two-week review period) so the WaterAid team can provide comments and feedback before it is finalised.  This report must be aligned with WaterAid’s reporting format and will require approval from WaterAid before final submission.

8. Supervision

The consultant will report to the Manager Partnerships and Program Quality, Water Aid Pakistan.

9. Roles and Responsibilities:

a) Consultant

• Lead on design and delivery of the endline evaluation.

• Finalize methodology and tools in consultation with WaterAid.

• Conduct data collection, analysis, and reporting.

• Deliver all required outputs within the agreed timeframe.

b) WaterAid Pakistan

• Provide technical oversight and quality assurance.

• Facilitate access to internal documents and contacts.

• Organize inception and findings validation meetings.

• Coordinate review of all deliverables.

c) Local Partner (LHDP)

• Facilitating community access for the consultant.

• Introduce consultant to local authorities and community structures.

• Assist in organizing FGDs and KIIs where required.

10. Ethics and risks

The consultant must ensure that all data collection activities adhere to ethical standards, including informed consent, voluntary participation, and the right to withdraw at any time. Data collected must be anonymised and securely stored. Special consideration must be given to the participation of vulnerable groups, including women, children, and persons with disabilities. Any ethical risks must be identified and mitigated in The consultant must comply with other relevant WaterAid policies, such as the Contractor Code of Conduct and Global Safeguarding policy..

11. Application /Proposal 

The proposal should contain the following:

1. An expression of interest explaining your interest and suitability for the position against each of the selection criteria. Please explain as to how your skills match the tasks detailed in the ToR.

2. Description of methodology which will be used for the Project Evaluation.

3. Your profile/portfolio. (CVs of key proposed staff)

4. Examples of similar project evaluations conducted by the proposed consultant/firm. (please attach a sample of recent research conducted on the same theme/topic)   

5. Financial proposal (In separate envelop, detailing the item wise budget needs of the assignment, including but not limited to daily rate and other costs inclusive of all taxes. The budget needs to be supplemented with budget narrative, rationalising the cost and its breakup). 

6. A workplan with deadline for each deliverable. 

12. Proposal Evaluation criteria

Candidate (firms/individual) shall submit technical and financial proposals in two separate sealed envelopes. Technical offers will first be evaluated according to the “Evaluation criteria” below and the technical qualification threshold point is “70”. 

Technical Evaluation Criteria for Shortlisting Consultancy Firms/Consultants:

Financial offer 

Please refer to clause 5 of section “Application /Proposal”. TheFinancial offer needs to

testify the fact that each deliverable is accounted for in the offer and syncs with the

workplan. All deliverables MUST be completed within the assignment duration.