Terms of Reference (ToR) for Final Evaluation

OBJECTIVE OF TOR

The overall objective is to engage a local and legally registered/operational evaluation firm to conduct the final evaluation to document the project achievements using OECD criteria.

OVERVIEW

Search for Common Ground: Founded in 1982, Search is a global nonprofit corporation based in Washington DC, and one of the world’s leading international peacebuilding organizations. Search’s mission is to transform the way that the world deals with conflict, away from adversarial approaches and towards collaborative solutions. Working with actors from all parties, ranging from grassroots organizations to top-level leaders, we develop locally adapted responses to a wide range of conflict issues, using a toolkit that includes capacity-building, mediation and alternative dispute resolution, community dialogue, arts-based approaches, and mass media to advance conflict prevention and resolution. We do this through an inclusive approach, engaging government, civil society, media, and traditional and community actors, with a particular focus on women and youth, to become agents of peace in their communities and in their country. Our work has been recognized at the global level, including a nomination for the 2018 Nobel Peace Prize.

The evaluation firm will develop the evaluation methodology based on OECD DAC criteria, data collection tools, engage field teams and provide them the training, collect field data by visiting selected field visits and speak with staff, beneficiaries and other stakeholders, perform data analysis, write a detailed evaluation report including case studies, employ M&E tools to report on program achievement and progress to the SFCG Country based consultants and regional M&E Expert. This will better enable the program and regional team to diagnose and resolve implementation and design bottlenecks to improve program effectiveness.

THE PROJECT TO BE EVALUATED

Search for Common Ground (Search) is implementing a project - Youth for Humanity: Engaging Youth to Promote Religious Freedom in Pakistan - that partners with local youth groups to build their capacity to work with religious communities and promote peaceful coexistence and respect, while also launching a public media campaign to further advance new narratives around religious identity and coexistence. The project focuses on leveraging the existing initiatives led by youth and elevating it to engage religious leaders, amplifying messaging using media and digital technology, and connecting youth across the country and globally to build on lessons learned and best practices for effective youth-led programming. Focusing on large urban areas, where religious diversity is high, the project is locally owned and youth-led with a focus on Search’s global best practices for engaging religious communities using dialogue, arts, and media to promote public support for peaceful coexistence and respect. The project takes advantage of the recent focus on promoting religious sites to highlight this point for commonality and collaboration, as well as an entry point for engaging religious actors and institutions.

The overall goal of the project is to increase inter-communal and intra-communal respect and peaceful coexistence among religious communities in Karachi and Lahore.

This goal is supported by the following specific objectives and corresponding expected results (ER):

Objective 1: Youth leaders promote inter-communal and intra-communal respect in their communities.

ER 1.1: Youth leaders have the capacity to constructively engage their peers, religious actors, and communities on issues affecting inter- and intra-communal dynamics.

ER 1.2: Youth networks bring together and support youth from different faiths and secular groups to establish a foundation for intergroup collaboration.

Objective 2: Youth voices champion peaceful coexistence and the rights of all people.

ER 2.1: Enhance understanding of youth on effective messaging for promoting peaceful coexistence.

ER 2.2: Increase in youth-led and designed local public messages, on and offline, promoting peaceful coexistence and the rights of all people.

Goal and Objectives of Evaluation

The main objective of the final evaluation study is to document the project performance and the shift in the attitudes of beneficiaries towards people of other faiths targeted as part of this project in Pakistan. The project will use OECD DAC evaluation criteria – relevance, effectiveness, efficiency, impact and sustainability, while considering gender, disability and ‘do no harm’ principles as additional aspects. This should include: (a) review of contextual shifts, (b) the extent to which the project has been able to achieve its objectives indicated in the project’s log frame, (c) the effectiveness of the intervention approaches and (d) the extent of sustainability of achievements. The approach to evaluation is expected to be grounded in the guiding principles of Search’s work: inclusive and effective participation, conflict sensitivity, Common Ground approaches, commitment to building capacity, critical engagement and valuing context-based knowledge. The evaluation will be coordinated by the Search Regional M&E Expert with close engagement from Search Pakistan team.

Key Questions of the Evaluation

The key evaluation questions are based on OECD DAC criteria. The evaluation specifically focuses on assessing the relevance and impact of the project intervention in the current context of Pakistan and documenting key learnings and achievements. It covers the effectiveness of the intervention and approach, including targeting, in generating results; development of the mechanism to sustain the contribution after the life of the project and drawing lessons learned, good practices and scaling up opportunities for the future. The following are the key evaluation questions for this assignment^[1]:

Relevance:

-          To what extent is the project relevant to the current conflict dynamics vis-a-vis the landscape of inter-faith harmony in Punjab and Sindh?

-          How inclusive and sensitive is the project to participants of different groups (such as age, gender, ethnicity, religion, disability or other important factors in the context)?

Effectiveness:

-          Did the activities achieve the targeted results in relation to stated objectives/results?

-          To what extent did the youth from different groups, in target communities increase their knowledge and skills in promoting inter- and intra-communal respect and relationships?

-          How effective was media programming as a tool for improving resistance (among youth) to violence, and for promoting freedom or religion and belief (FORB) practices?

-          To what extent the project was able to promote the practice of interfaith respect and collaboration amongst the youth of various religions?

-          To what extent was the project able to foster an enabling environment for youth to develop networks with youth of other religions and sects?

-          How has the project implementation modality affected the achievement of results?

Intermediate Impact:

-          What difference has the project made, especially related to communal co-existence, respecting others’ rights and inter-communal collaboration in the target communities?

-          To what extent has the project or program generated, or is expected to generate, significant positive or negative, intended or unintended, effects in the society?

-          How has the project contributed to promoting the role of youth in local peacebuilding initiatives in the target areas of Punjab and Sindh?

Sustainability:

-          Have they taken steps to ensure sustainability post-project life? If yes, what does it include? If not, why not?

-          Has the project succeeded in building the intended capacities of local partners (e.g. youth centers, media entities) to sustain and own the results of the project?

-         Is there any evidence of local youth groups taking ownership (on-going or planned) of the project initiatives and continuing in the post-project period?

Lessons learned and good practices:

-          What were the innovative approaches/ good practices used, if any, and what were the major lessons learned from this project?

-          Is there any potential of replicability and scaling-up (full or partial) of the project in the same locations or in other locations in Pakistan?

Methodology and Data Collection Tools

A mixed methods approach is suggested for this evaluation with both qualitative and quantitative methods of data collection supported by desk review of documents, where the evaluation is meant to produce information and make recommendations that are sufficiently valid and reliable based on data and analysis. The detailed methodology for this evaluation will be developed by the evaluation team in consultation with the Search Team who will provide final approval.

Search welcomes innovative evaluation methodology(ies) from the evaluation team. The team should propose a methodology that responds to the evaluation questions and objectives, detailing how results will be captured from the project beneficiaries (and how differences within this group will be captured and understood).

Based on the brief guidance presented above, the applicant is expected to provide the detailed methodology for this evaluation. While presenting the overall methodological approach, the applicant is expected to propose a minimum number of KIIs and FGDs and a sample survey like the one done during the baseline study inn order to ensure comparison of baseline and endline data.

The team is also required to integrate some of the relevant Peace Impact Framework indicators in the survey and FGDs.

Deliverables

The project deliverables should be produced in English. The deliverables are as follows:

Inception report with evaluation design and work plan completed within 15 days of the signing of the contract (or consultants may propose an alternate timeline). The inception report should have detailed literature review, including available project documentation, methodological approach, sampling method (locations and scope) and data collection tools to capture the information and data mentioned in key evaluation questions. The inception report and the data collection tools/questions need formal approval from the Search team before starting the data collection process.

Field Data Collection: Field data collection will involve visits to the project sites in both the provinces, discussions with youth, project staff and other stakeholders. The nature and quality of the field data collection will be determined based on the agreed inception report.

First draft of the evaluation report. The firm should submit the first draft of the evaluation report, analyzing and integrating both qualitative and quantitative data. The report is subject to Search’s review. The review and feedback may be of multiple rounds depending on the quality of the report. All the raw data collected, and photographs taken during the evaluation must be stored safely and should be handed over to Search at the end of the assignment.

Final evaluation report: The report should be written (and fully edited) in English (max 35 pages), and consist of:

▪      Title page and other initial pages

▪      Table of contents.

▪      Executive        summary    that    includes    context    summary,    brief    project    introduction, methodological approach, key findings and recommendations (no more than 4 pages).

▪      Current context analysis (approximately 2 pages)

▪      Project introduction including the goal, objectives, expected results, summary activities and Evaluation methodology, detailed explanation of sample size calculation and distribution across locations, gender and ethno-religious identities.

▪      Evaluation findings, analysis, and conclusions. The findings section will have sub-chapters to cover the evaluation criteria and key research questions. This is the core of the evaluation report.

▪      Strategic and activity-focused recommendations. Each recommendation should be directly linked to the evaluation findings.

▪      Appendices, which include, research instruments, list of interviewees, bibliography, and consultant (s) brief biography.

▪      One electronic copy (in MS Word format) in English.

▪      2–3-page document capturing key results of the report in English.

Search Pakistan will provide a report template with appropriate branding, color and fonts with the consultant team after signing the contract.

Remuneration

The payment modality of the selected firm is as follows:

1)  25% at the inception stage

2)  25% after data collection

3)  25% at the submission of the draft report

4)  25% at the approval of the final report.

Timeline

The evaluation assignment will be for three months from the signing of the contract.

Requirements of the Applicants

The following skills and experience are expected from the evaluation firm for this project:

▪          Registered Pakistani organization with at least five years of relevant experience since registration

▪          Experience in developing qualitative and quantitative research methods

▪          Strong understanding of the conflict dynamics, landscape of inter-faith harmony and social cohesion in Pakistan

▪          Clear understanding of OECD-DAC criteria of evaluation

▪          Experience in carrying out project evaluations in conflict transformation and peacebuilding

▪          Experience in media programming and/or evaluation

▪          Knowledge in conflict-sensitive project management

▪          Good analytical and writing skills, and the ability to clearly present findings, draw practical conclusions, and develop lessons learned

▪          Excellent skills in English required; and the team members should be fluent in relevant local languages spoken by local communities including Urdu, Punjabi and Sindhi

▪          Ability to travel to the field for data collection and cross-checking the data

▪          Good organizational and communication skills with international/ national officials and rural communities are required

In addition, the consultants are required to respect the following Ethical Principles^[2]:

• Comprehensive and systematic inquiry: Consultants should make the most of the existing information and full range of stakeholders available at the time of the review. Consultants should conduct systematic, data-based inquiries. They should communicate their methods and approaches accurately and in sufficient detail to allow others to understand, interpret and critique their work. They should make clear the limitations of the review and its results.
• Competence: Consultants should possess the abilities and skills and experience appropriate to undertake the tasks proposed and should practice within the limits of their professional training and competence.
• Honesty and integrity: Consultants should be transparent with the contractor/constituent about: any conflict of interest, any change made in the negotiated project plan and the reasons why those changes were made, and any risks that certain procedures or activities produce misleading review information.
• Respect for people: Consultants should respect the security, dignity and self-worth of respondents, and program participants. Consultants have the responsibility to be sensitive to and respect differences amongst participants in culture, religion, gender, disability, age and ethnicity.

Selection Criteria

 Consultant will be selected based on based on:

1.       The quality of the technical proposal submitted: Thematic and technical knowledge of the subject matter, relevance of proposed methodology to the goal, objectives, and research questions for the project, Quality of proposed methods, presentation of analytical approach conflict sensitivity approaches, and quality control measures, academic qualifications and professional experiences of the proposal team members, team combination (50%)

2.       Quality of the writing sample and its relevance to the assignment in concern (10%)

3.       Performance in the interview: Presentation ideas, ability to clarify questions raised from the proposal, explanation of the field data collection team formation and explanation of explanation quality assurance approach (10%).

4.       Proactive engagement of the team members during discussion (5%

5.       Commitment to flexibility and timely completion of assignment, Timeline for proposed activities (5%)

6.       Budget proposed in the financial proposal: Justification of the rate and no of team

members proposed, total amount proposed in relation to proposed methodology, deliverables and team and balance between HR budget and other field related costs (20%).