TORs for Mid Term Evaluation

Pak Mission Society

3339 views

Posted date	15th November, 2024	Last date to apply	30th November, 2024
Country	Pakistan	Locations	Islamabad
Category	Consultancy
Type	Consultancy	Position	1
Experience	5 years

Attached files

52382_job_TORs for Mid Term Evaluation.pdf

TORs

Mid Term Evaluation

Project: Child Driven Climate Action /CDCA

Target Areas: Islamabad, Rawalpindi & Lahore

Implemented by: Pak Mission Society

Funded By: KNH-BMZ

1. Introduction:

The mid-term evaluation of CDCA outputs by KNH-BMZ will be commissioned by Pak Mission Society and will cover the period from January 2023 to December 2024. It is scheduled to be carried out in January to February 2025. The objective of mid-term evaluations is to assess the progress and performance of CDCA halfway through the designated period and generate comparisons based on the baseline. This evaluation also aims to promote both accountability and learning. It will be an evidence-based, assessment of the program's activities' performance, enabling Pak Mission Society to make necessary adjustments for the remainder of the project period. The target areas for this evaluation will include Islamabad, Rawalpindi, and Lahore.

2. Project Background & brief.

"Children Driven Climate Action (CDCA)" project is scheduled for implementation from 1^st January 2023 to 31^st December 2025 and is funded by KNH-BMZ. The project aims to empower youth in Lahore, Rawalpindi, and Islamabad, Pakistan to engage in climate action. Pak Mission Society (PMS) and KNH-BMZ have a sound partnership built on trust, honesty and mutual understanding that is based on professional competence and synergies in their chosen field of humanitarian significance. Both organizations have focused on environment related climate action awareness raising, adaptation and resilience outcomes.

This project is a step forward in the journey of placing children at the center, who will gain skills and knowledge for the cause and will participate in a meaningful transformational trajectory to achieve real change for the better environment for the next generation. Establish Climate Focused Libraries in 5 low-income schools in Islamabad, Rawalpindi and Lahore. Creating awareness through cycling tours, climate walks, climate caravans as well as broadened knowledge about climate shocks and resilience, global warming, GHG emissions, general human well-being and environment ecosystem etc.

The Children Driven Climate Action (CDCA) thru Green Clubs has the potential to deliver intended outcomes as the Game Changers, this project will certainly endorse child bearing signature character of loving nature, resilience and problem solving, since the childhood is inherently designed for activism and eagerness to know undisclosed truths with curiosity within their surroundings. Children have the capacity to share unbiased opinions, who can even express themselves without shyness, and by making compelling arguments to win their case. As influencers children will expedite their full potential to achieve ecological and low-carbon footprints.

Impact Matrix of the project (project achievements/ expected results)

Overall objective (impact):

The concerns and proposals of children and youth from Islamabad, Rawalpindi and Lahore for climate justice, resilience, climate adaptation and environmental protection are systematically taken into account at community level and in Pakistan's climate policy.

Project goal

(Outcome)

Indicators (possibly plus quantity structure)

Output value

(quantitative & qualitative)

Target value (target)

(quantitative & qualitative)

Young climate activists from 25 Green Clubs from Islamabad, Rawalpindi and Lahore use their new practical and theoretical knowledge to elaborate their demands and bring them to the relevant (government) bodies through better networking.

It is estimated that the majority of children and young climate activists in the three target cities lack the knowledge and practice to raise their concerns at the national level.

(No basic data available)

It is estimated that the visibility of children and young people's concerns and proposals on climate at the national level is low due to a lack of coordination and liaison between child and youth climate activists and the national Climate Action Network and government actors.

25 Green Clubs train at least 2,500 students (25*100 students) on climate and environmental protection issues. They engage in the Climate Action Network (CAN) and develop 5 concrete demands, 2 of which are submitted to political decision-makers during the project period.

Sub-targets (output)	Indicators (possibly plus quantity structure)
Sub-targets (output)	Output value (quantitative & qualitative)	Target value (target) (quantitative & qualitative)
1. 25 Green Clubs in 25 schools are established and operational. The participating schools have incorporated the topics of climate change and the environment into regular school lessons by the end of the project period.	1. 15-20% of the target schools address climate change in an informal way, e.g. through tree planting, but there are no formal courses on climate change and the environment run by the schools. Students have little knowledge on climate change and environmental protection.	1. 80% of the target schools have successfully adapted the Green Club model (theory and practice). The students show significantly improved knowledge on climate change and environmental protection compared to the baseline study.
2. 12500 children from the 25 target schools in Rawalpindi, Lahore and Islamabad are increasingly engaged on climate and waste management issues under the guidance of the Green Clubs, contributing to increased social interest on these issues by 2025.	2. approx. 5-10% of the children and young people in the target schools are concerned with climate issues and waste management. However, no waste is collected & separated in the schools and recycled. At the beginning of the project, there are no school gardens that are co-designed by children.	2. 6,250 children (50% of the target group) show responsibility by adopting methods of waste disposal, recycling, composting and growing vegetables and set an example for their environment (classmates, families, neighbours). At the end of the project, at least 100 children, 5 children per school, participate in the school gardens and the schools continue to separate waste (1,000kg/year of recyclable material collected per school). The organic waste is composted and used for the school gardens.
3. coordination of children and youth with the Climate Action Network is improved, leading to a more effective influence of the young generation on the national climate agenda.	3) The country's climate activists are not aware of children's and young people's perspectives on climate change. Only 5-10% of the activists highlight the children's perspective on climate change, the efforts of the rest of the civil society are close to zero.	3.1 60% of the climate activists of the Climate Action Network (CAN) have participated in the actions led by the Green Clubs, know the concerns of the young people and take them into account in their political work. 3.2 Interested members of the 25 Green Clubs (approx. 100 students) engage in CAN and participate in policy forums at district, provincial and national levels (approx. 6 events during the project period).
4. 100 office holders of political parties and ministries at federal and state level support political initiatives on the topic of climate in which the concerns of children and young people are taken into account.	4. 5-10% of public officials are aware of climate issues and are active in this field, but the views of children and young people are not sufficiently taken into account.	4. there are at least 2 policy initiatives that take into account the views of children and young people.

Management arrangements,

The CDCA project team in PMS-Islamabad is responsible for the overall implementation of the

project. This project is operating from 2 office such as one is in Islamabad and the second office is in Lahore. The project is managed by a project manager based in Islamabad, assisted by a Lahore’s’ team, administrative Manager and a reporting and monitoring & evaluation-CCT Manager.

Implementation arrangements

The project works closely with the beneficiaries, 12,500 targeted youth and children will actively participate in the climate resilience and GHG emissions mitigation under the guidance of Green Clubs contributing to increased social interest on these issues. 500 climate and social activists from all walks of life, i.e. civil society organizations, students, academics, professionals, media practitioners of all genders and 100 officials, civil servants and parliamentarians working for line ministries, departments and agencies in the fields of environment and climate utilize their influence and developing synergies for persuasive furtherance of Children Driven Climate Action in collaboration with existing civil society networks both at provincial and national levels and also engage political leaders in implementing climate change policies and practices in their constituencies.

The primary beneficiaries of this project are:

325 members of the Green Clubs in 25 schools in the target cities and the 12,500 students of the 25 schools form the direct target group. The Green Clubs are made up of students from grades 7 to 12 and teachers. Children from all walks of life are targeted, but the focus is particularly on children from low-income households, as they are more exposed to the impacts of climate change and less prepared due to their financial situation. The target group has a mixed faith background and partly comes from ethnic minorities. The project will promote characteristic traits of children such as closeness to nature, resilience and problem-solving skills, making them game changers. As influencers, children will realise their full potential to achieve an ecological footprint. 500 climate and social activists’ participants at a national consultation event in the federal capital to incorporate Climate Action advocacy and practices by political wings in their constituencies, impacting global warming and limiting GHG emissions and promoting eco-friendly environment in support of Clean and Green Pakistan. 100 officials, civil servants and parliamentarians working for line ministries, departments and agencies in the fields of environment and climate.

The indirect target group of the project is the population of Islamabad, Rawalpindi and Lahore, from which about 50,000 people, including 25,000 children and young people, are to be reached through the Green Clubs' activities.

3. Purpose & scope of the Mid-term Evaluation:

The purpose of this mid-term evaluation is to ensure that progress and results of the project are monitored, analysed, understood, communicated and acted upon in a timely, efficient and result-oriented manner by a comprehensive assessment of relevance, effectiveness, efficiency, impact and sustainability of the project intervention.

In particular, it services three main purpose:

1) It will assess progress towards project objectives and results, identify key challenges and any factors that may have affected the project and its implementation. It will present an objective assessment of the extent to which the project responds to the needs of the beneficiaries, internal & external stakeholders. Their commitment to the realization of the projects objectives and their capacity to deliver on these.

2) The evaluation findings and suggestions will inform the adjustments of the project’s strategy and plan in the second half of the project life, and contribute to knowledge development of PMS team and project stakeholders.

3) Suggest and draft friendly user tools to monitor both aspects of the project-Child Focused Climate Action through green clubs (children as trailblazer), awareness raising, adaptation and resilience outcomes.

It will be the responsibility of the Evaluator with the collaboration of project team to perform ongoing evaluations throughout the project lifecycle to assess actual results against the projected impacts documented in the matrix. Any changes or updates based on evaluation findings will be documented and communicated to all stakeholders.

4. Evaluation Questions:

The evaluation will examine the project implementation against the hereunder criteria by addressing the following questions. The evaluator will develop a more detailed analytical framework of questions and sub-questions to carry out the evaluation. Therefore, this questionnaire is not final and the selected evaluator should review this list with CDCA team.

Relevance and strategic fit of the project	ü To what extent the project objectives and outcomes are relevant to the needs of the beneficiaries? ü Were project methodologies and approaches in line with Stakeholders’ capacities and expectations? ü To what extent have the project’s action so far contributed to improving the coordination of existing civil society networks will gallop compelling inroads to generate grassroots support for policy reforms, make real-time intrusions as they plan public campaigns/events/exposure in pursuance of climate neutral drives. ü Should the project design and methodology be modified to improve its relevance in the second half of the project? If so, how?
Project progress and effectiveness	ü To what extent has the project been making progress towards the planned objectives and outcomes? ü What have been achieved so far? ü What are the major constraints so far? ü How can be explained the delay in delivery? ü Have the project approaches appropriately matched the capacity of the project beneficiaries/Stakeholders for their action? ü What measures are required to improve the capacity of the project beneficiaries in the second half of the project?
Efficiency of resource use	ü Have resources (funds, human resources, time, expertise, etc.) been allocated strategically to achieve outcomes?
Effectiveness of management arrangements (including monitoring and evaluation)	ü Has the project received adequate political, technical and administrative support? Was there a clear understanding of the roles and responsibilities by all parties involved? ü How effective is communication between the project team, PMS, Stakeholders and donor?
Sustainability and Impact of the project	ü How effectively has the project built necessary capacity of people and institutions? How effectively has the project-built climate resilience, ownership and capacity of beneficiaries & Stakeholders? ü How likely the project will achieve its objectives in the second half of the project and what elements need to be taken into account to ensure sustainability of outcomes?

5. Sample Size:

To calculate the sample size for a project targeting over 12825(325 members+12500 Std) youth across Islamabad, Rawalpindi, and Lahore, we can use a formula for sample size calculation in mid-term evaluation. One common formula for sample size calculation is:

Sample Size (n) = (Z^2 * p * (1-p)) / E^2

Where:

Z is the Z-score corresponding to the desired confidence level (e.g., 95% confidence level has a Z-score of 1.96).

p is the estimated population proportion (if unknown, use 0.5 for maximum variability).

E is the desired margin of error (the acceptable difference between the sample estimate and the true population value).

Given that the project targets over 12825 youth, let's assume we want a 95% confidence level, a margin of error of 5%, and maximum variability (p = 0.5) since we do not have an estimate of the population proportion. The Z-score for a 95% confidence level is 1.96.

Plugging these values into the formula:

Sample Size (n) = (1.96^2 * 0.5 * (1-0.5)) / 0.05^2 Sample Size (n) = (3.8416 * 0.25) / 0.0025 n=384.16/1384.16-1/1+12825=384.16/1.0299=373.

For a population of 12,825 with a 95% confidence level, a margin of error of 5%, and assuming a proportion of 0.5, the required sample size would be approximately 373

6. Methodology:

The methodological approach and data analysis for evaluating the mid-term progress of the "Children Driven Climate Action (CDCA)" project by Pak Mission Society (PMS) include the following components:

ü Research Design: Utilize a mixed-methods approach combining quantitative and qualitative data collection methods to provide a comprehensive evaluation of the project.

ü Data Collection Methods: Conduct surveys, interviews, focus group discussions, and document reviews to gather relevant information on the project's impact, achievements, and challenges.

ü Sampling Techniques: Employ purposive sampling to select key project stakeholders, beneficiaries, and partners for in-depth data collection.

ü Framework for Analysis: Utilize a logic model framework to guide the evaluation process, linking project inputs, activities, outputs, outcomes, and impacts.

Data Analysis Procedures:

ü Quantitative Analysis: Use statistical software to analyze quantitative data collected through surveys, such as calculating frequencies, means, and correlations to measure project outcomes.

ü Qualitative Analysis: Conduct thematic analysis of qualitative data from interviews and focus group discussions to identify key themes, patterns, and insights related to the project's effectiveness.

ü Triangulation: Compare and integrate findings from different data sources to provide a more comprehensive understanding of the project's progress and impact.

Data Interpretation:

ü Synthesize and interpret the quantitative and qualitative findings to assess the project's achievements against set objectives and its alignment with the work plan.

ü Identify strengths, weaknesses, opportunities, and threats (SWOT analysis) to inform recommendations for project improvement and sustainability.

ü Draw conclusions on the project's effectiveness in promoting children-driven climate action, addressing community waste management needs, and enhancing health and economic outcomes.

Reporting and Dissemination:

ü Compile the evaluation findings into a comprehensive report with clear presentation of data analysis results, interpretations, and actionable recommendations for project stakeholders.

ü Disseminate the evaluation report to project partners, funders, and relevant stakeholders through presentations, workshops, and online platforms to communicate the project's progress and impact.

By following a robust methodological approach and thorough data analysis procedures, the mid-term evaluation of the "Children Driven Climate Action (CDCA)" project will provide valuable insights into its effectiveness, efficiency, scalability, Impact and sustainability, informing strategic decisions for enhancing the project's outcomes and impact.

7. Expected Deliverables.

The following deliverables can be expected to be provided to the project stakeholders.

ü Develop and submit an evaluation plan outlining the methodology, data collection methods, sampling techniques, data analysis procedures, and timeline for the evaluation process.

ü Design and provide data collection tools such as survey questionnaires, interview guides, and focus group discussion protocols tailored to gather relevant information on the project's progress and outcomes. The actual values of the indicators at mid-term are to be collected and the updated impact matrix attached to the report as an annex.

ü Collect, clean, and analyze quantitative and qualitative data using appropriate tools and techniques according to the evaluation plan. Provide both hard and soft data.

ü Prepare and present interim findings report to project stakeholders, highlighting key insights, emerging trends, and preliminary conclusions from the mid-term evaluation.

ü Compile a comprehensive draft evaluation report that includes detailed analysis of data, interpretation of findings, assessment of project progress against objectives, and SWOT analysis.

ü Provide actionable recommendations based on the evaluation findings to enhance project effectiveness, address challenges, and capitalize on opportunities for improvement.

ü Develop an action plan outlining suggested strategies and steps for implementing the recommendations to drive positive project outcomes

ü Deliver a finalized evaluation report that includes executive summary, methodology, findings, conclusions, recommendations, and annexes with raw data and supporting documents.

ü Optional: Support the dissemination of the evaluation report to relevant audiences, including project partners, donors, and the wider community through various communication channels.

ü Optional: Provide post-evaluation support, if required, to address any queries, clarifications, or additional analysis needed by project stakeholders

8. Determining Roles & Responsibilities:

The roles and responsibilities can be defined as follows:

Consultant:

ü The consultant is responsible for conducting the mid-term evaluation of the CDCA project.

ü Designing the evaluation framework and methodology.

ü Collecting and analysing data on project activities, outcomes, and impacts.

ü Engaging with diverse stakeholders, including project staff, beneficiaries, partners, and donors.

ü Drawing insightful conclusions and developing actionable recommendations based on evaluation findings.

ü Documenting and presenting the evaluation results in a comprehensive report.

Project Staff:

ü Project staff members support the consultant in providing information and data related to the CDCA project.

ü Collaborating with the consultant to facilitate data collection and access to project information.

ü Participating in interviews, focus group discussions, or surveys as required.

ü Providing contextual insights and clarifications on project activities and outcomes.

ü Project partners and donors contribute to the evaluation process by sharing their perspectives and feedback.

ü Collaborating on the development and implementation of improvement strategies based on evaluation findings.

9. Competency Criteria:

Qualification of Evaluation Team/Consultant/Firm	Consultant experience in Mid-term evaluations (min 05 years). Formal qualification of the team leader (e.g. minimum of an MA/Masters-degree) and team leader’s consultancy experience (at least 5 assignments in a leading position) Experience of team leader with conducting similar assignments (at least 3 consultancies with similar objectives and complexity) Experiences with cross-cutting issues (e.g. Gender, Rights Based Approach, Child Rights Approach)
	Knowledge and background expertise in environment & climate change action projects/programmes. Climate Action and Solid Waste Management WASH related practical knowledge and nature loving practices. Capacity building training on kitchen gardening, tree plantation, protection of species and birds, promote fresh and clean air by managing solid waste management, clean environment. Climate change adaptation, Awareness etc Capacity strengthening of communities especially children/Youth. Having experience in similar assignments with Environmental and economic sustainability (EES), WASH related practical knowledge and nature loving practices. Training & development Climate change adaptation, Capacity strengthening of communities especially Children/youth Geographic, Social & cultural experience in Rawalpindi, Islamabad & Lahore.


Technical Proposal	Methodology proposed is realistic and relevant to the objective of the ToRs’ How suitable are the proposed methods and data collection instruments for meeting the objectives and answering the questions of the evaluation/study? How appropriate is the proposed scope (e.g. groups/persons, documents or secondary data to be consulted and the proposed sample sizes) for answering the questions of the evaluation/study? In case that there is a team of consultants, are the roles, responsibilities and tasks of each person in the evaluation process clearly assigned and described? Is an evaluation schedule proposed and is it reasonable with regard to the total number of days and the allocation of days to the different phases (preparation, data collection, data analysis and reporting)?
Technical Proposal
Ethical Considerations	Is ethical consideration, no harm and data protection adequately narrated in the technical proposal?
Financial Proposal	How transparent and appropriate is the financial offer: number of days, daily honorarium, travel costs, number of and fees for supporting staff (e.g. enumerators), taxes, total costs?

10. Timeframe

The consultancy shall be conducted and completed within 30 calendar days during January/February 2025, commencing not later than 1 Jan 2025. An indicative schedule is presented in the table below:

#	Tasks	Time Allocation (# of working days)
1	Briefing Meeting	1
2	Desk Review	4
3	Inception Report	4
4	Workplan Review and Approval	2
5	Evaluation Fieldwork	9
6	Draft Report preparation, including midterm update	5
7	Draft Report submission to PMS	1
8	Feedback Review with PMS	1
9	Final Report Submission to PMS	2
10	Validation Workshop	1
	Total:	30

11. Payment Schedule.

Payments will normally be made by the end of the evaluation accomplishment. We will follow the PMS policy of total payment at the end of the assignment. We can see the consultant preference too.

12. REQUIREMENTS FOR THE EVALUATION PROPOSALS:

ü Submission of a detailed technical proposal, along with firm profile, CVs, recommendation letters or proof of previously conducted similar assignments (mid-term evaluation reports of similar subject as sample).

ü A detailed financial proposal with information on the number of working days of the individual persons involved in the evaluation, on their daily fees and ancillary costs, e.g. for transport, accommodation, catering, communication, etc

ü Clear workplan based on evaluation task.

13. REQUIRED INFORMATION ON ETHICAL CONSIDERATIONS AND THE USE OF AI.

Evaluation proposals are to provide detailed information on the following aspects:

ü Rationale and Purpose: Consultants should clearly articulate the rationale and purpose for incorporating AI into the evaluation methodology, explaining how it will enhance the quality, rigor, and efficiency of the evaluation process.

ü Methodological Approach: Consultants should provide a detailed description of the specific AI techniques, tools, algorithms, or models they intend to utilize in the evaluation, including their relevance to the evaluation objectives and data sources.

ü Data Collection and Analysis: Consultants should outline the sources of data that will be used to train AI models or algorithms, as well as the methods for data collection, processing, and analysis. This should include considerations for data privacy, confidentiality, and security, as well as measures to ensure the accuracy, reliability, and validity of the data.

ü Ethical Considerations: Consultants should address ethical considerations associated with the use of AI, including potential biases, fairness, transparency, and accountability. They should describe how they will mitigate risks and ensure compliance with relevant ethical guidelines and regulations.

ü Human Oversight and Interpretation: Consultants should clarify the role of human oversight and interpretation in the use of AI, including how AI-generated insights will be validated, interpreted, and integrated with other sources of evidence or expertise.

ü Reporting and Documentation: Consultants should specify how findings, conclusions, and recommendations derived from AI-driven analysis will be documented, reported, and communicated to stakeholders in a clear, understandable, and transparent manner.

Bid Submission Address:

For any query, please send email at [email protected] and for bid submission please share your sealed envelope containing technical and financial proposal at the following address;

Procurement Manager

House # 333, Street 29, Sector G14/4 - Islamabad

051-8446166