Terms of Reference (ToR) Final Evaluation

 

Contracting Organization:	Search for Common Ground (SFCG)
Project Name:	Inspiring the Next Generation of Independent Media in Pakistan.
Project location:	Khyber Pakhtunkhwa (including Newly Merged Districts) and Balochistan
Expected Period:	May-July 2023
Type of Contract:	Service / Deliverable Based Contract

 

OBJECTIVE OF TOR

The overall objective is to engage a local and legally registered/operational evaluation firm to conduct the final evaluation to document the project achievements using OECD criteria.

CONTEXT

 

Search for Common Ground: Founded in 1982, Search is a global nonprofit corporation based in Washington DC, and one of the world’s leading international peacebuilding organizations. Search’s mission is to transform the way that the world deals with conflict, away from adversarial approaches and towards collaborative solutions. Working with actors from all parties, ranging from grassroots organizations to top-level leaders, we develop locally adapted responses to a wide range of conflict issues, using a toolkit that includes capacity-building, mediation and alternative dispute resolution, community dialogue, arts-based approaches, and mass media to advance conflict prevention and resolution. We do this through an inclusive approach, engaging government, civil society, media, and traditional and community actors, with a particular focus on women and youth, to become agents of peace in their communities and in their country. Our work has been recognized at the global level, including a nomination for the 2018 Nobel Peace Prize.

The evaluation firm will develop the evaluation methodology based on OECD DAC criteria, data collection tools, engage field teams and provide them the training, collect field data by visiting selected field visits and speak with staff, beneficiaries and other stakeholders, perform data analysis, write a detailed evaluation report including case studies, employ M&E tools to report on program achievement and progress to the SFCG Country based consultants and regional M&E Expert. This will better enable the program and regional team to diagnose and resolve implementation and design bottlenecks to improve program effectiveness.

THE PROJECT TO BE EVALUATED

 

Search for Common Ground (Search) is implementing a project - Inspiring the Next Generation of Independent Media in Pakistan. - the project was launched in September 2020 in partnership with Search for Common Ground Pakistan. This project aims to support the growth of independent and diverse media in Balochistan and Khyber Pakhtunkhwa (KP), by increasing the capacity of young journalists, particularly young women, in independent and conflict-sensitive reporting; and fostering an enabling environment for the professional advancement of women in media. Working with local universities and press clubs, the project focuses on building sustainable initiatives that can be institutionalized in these bodies to support the next generation of independent journalists.

Search is working with local partners for project implementation while Search staff will provide required oversight and supervision for successful and smooth implementation.

The overall goal of the project is to advance women- and youth-led independent media in KP and Balochistan. This goal is supported by the following objectives and corresponding outcomes: 

Objective 1: Promote the practice of conflict-sensitive journalism by young journalists and university media departments in KP and Balochistan.

• Outcome 1.1: Targeted local universities institutionalize conflict-sensitive journalism in their media department curricula.
• Outcome 1.2: Young journalists in the target districts have increased access to resources and capacity building to champion conflict-sensitive journalism in their work.

Objective 2: Foster an enabling environment for the professional advancement of women in independent media KP and Balochistan

• Outcome 2.1: Targeted local independent media actors better support women in media.
• Outcome 2.2: Independent women journalists in the target districts feel more confident and safer in their workplaces.

Activities contributing to Objective 1: Promote the practice of conflict sensitive journalism by young journalists and university media departments in KP and Balochistan.

1.1.1 Consultative Workshop with Partner Universities

1.1.2 Training of Trainers (ToT) on Conflict Sensitive Journalism

1.1.3 Journalist Training Program

1.2.1 Media Mentorships

1.2.2 Media Fellowships

Activities contributing to Objective 2: Foster an enabling environment for the professional advancement of women in independent media in KP and Balochistan.

2.1.1. Gender Training for Press Clubs

2.1.2. Workshops with Media Houses

2.2.1 Press Club Reform Initiatives

2.2.2 Expanding Networks for Women in Media

GEOGRAPHIC LOCATIONS AND PARTNERS

The project coverage area is spread over the following target districts in Balochistan and KP provinces. The project will partner with three universities and work closely with press clubs and media houses working in these districts.

Target Districts by Province		Key Partners
Balochistan	Khyber Pakhtunkhwa (KP)	Universities
Quetta Qilla Saifullah Pashin Khuzdar Chaghi Sibi	Bajaur Dera Ismail Khan Khyber Kurram Mohmand Orakzai Peshawar North Waziristan South Waziristan	1. University of Peshawar 2. Balochistan University 3. Gomal University
Project Component
University
Press Club
Gender

Goal and Objectives of Evaluation

 

The main objective of the final evaluation study is to document the project performance in terms of advancing women- and youth-led independent media in KP and Balochistan through this project. in Pakistan. The project will use OECD DAC evaluation criteria – relevance, effectiveness, efficiency, impact and sustainability, while considering gender, disability and do no harm as additional aspects. This should include the extent to which the project has been able to achieve its objectives indicated in the project’s log frame. The approach to evaluation is expected to be grounded in the guiding principles of Search’s work: inclusive and effective participation, conflict sensitivity, Common Ground approaches, commitment to building capacity, critical engagement and valuing context-based knowledge. The evaluation will be coordinated by the Search Regional M&E Expert with close engagement from Search Pakistan team.

Key Questions of the Evaluation

The key evaluation questions are based on OECD DAC criteria. The evaluation specifically focuses on assessing the relevance and impact of the project intervention in the current context of Pakistan and documenting key learnings and achievements. It covers the effectiveness of the intervention and approach, including targeting, in generating results; development of the mechanism to sustain the contribution after the life of the project and drawing lessons learned, good practices and scaling up opportunities for the future. The following are the key evaluation questions for this assignment^[1]:

Relevance:

-          To what extent is the project relevant to the current conflict dynamics vis-a-vis the media landscape in KP and Baluchistan?

-          To what extent the project interventions met the needs of women and youth journalists in the target areas?

-          How inclusive was the selection process (from gender, ethnicity perspective) of the participating journalists?

Effectiveness:

-          Did the activities achieve targeted results in relation to the stated objectives/results according to the logframe?

-          To what extent did the project affect the resources and capacity of press clubs, journalists and universities to practice and promote conflict sensitive journalism?

-          To what extent the project was able to promote the practice of conflict-sensitive journalism by participating young journalists and university media departments?

-          To what extent was the project able to foster an environment where women journalists can practice the media profession safely and independently? Is there any evidence of women journalists being supported by mainstream media actors, including their male counterparts?

-          What role did the project play to help women journalists feel more confident and safer in their work?

Intermediate Impact:

-          To what extent has the project or program generated, or is expected to generate, positive or negative, intended or unintended, high-level effects in the professional lives of women and youth journalists?

-          In what ways, if at all, has the project contributed to promoting the role of youth and women in the media in KP and Balochistan?

-            Is there any evidence of integration of conflict sensitivity in the journalism landscape in the target regions?

Sustainability:

-          Have they taken steps to ensure sustainability of the results post-project? If yes, what does it include? If not, why not?

-          What are the signs that the program or its components will continue after external funding ends?

-          Is there any evidence of local partners, universities, press clubs or government actors taking ownership (on-going or planned) of the project initiatives and continuing in the post-project period?

-          Is there any evidence of institutionalization (formally or informally) of conflict sensitive journalism in the curriculum by the departments or professors in the target universities?

Lessons learned and good practices

-          Were there any significant changes in the project design or the implementation context? What were the reasons for these?

-          What were the major lessons learned from this project?

-          What were the innovative approaches/good practices used, If any?

-          Is there any potential of replicability and scaling-up (full or partial) of the project in the same locations or in other locations in Pakistan?

Methodology and Data Collection Tools

A mixed methods approach is suggested for this evaluation with both qualitative and quantitative methods of data collection supported by desk review of documents, where the evaluation is meant to produce information and make recommendations that are sufficiently valid and reliable based on data and analysis. The detailed methodology for this evaluation will be developed by the evaluation team in consultation with the Search Team who will provide final approval.

Search welcomes innovative evaluation methodology(ies) from the evaluation team. The team should propose a methodology that responds to the evaluation questions and objectives, detailing how results will be captured from the project beneficiaries (and how differences within this group will be captured and understood).

Based on the brief guidance presented above, the applicant is expected to provide the detailed methodology for this evaluation. While presenting the overall methodological approach, the applicant is expected to propose a minimum number of KIIs and FGDs and a sample survey like the one done during the baseline study.

The team is also required to integrate some of the relevant Peace Impact Framework indicators in the survey and FGDs.

Deliverables

 

The project deliverables should be written in English. The deliverables are as follows:

Inception report with evaluation design and work plan completed within 15 days of the signing of the contract. The inception report should have detailed literature review, including available project documentation, methodological approach, sampling method (locations and scope) and data collection tools to capture the information and data mentioned in key evaluation questions. The inception report and the data collection tools/questions need formal approval from Search team before starting the data collection process. 

Field Data Collection: Field data collection will involve visits to the project sites in both the provinces, discussions with beneficiaries including young and women journalists, project staff and other stakeholders. The nature and quality of the field data collection will be determined based on the agreed inception report.

First draft of the evaluation report. The firm submits the first draft of the evaluation report analyzing and integrating both qualitative and quantitative data. The report is subject to Search’s review. The review and feedback may be of multiple rounds depending on the quality of the report. All the raw data collected, and photographs taken during the evaluation must be stored safely and should be handed over to Search at the end of the assignment.

Final evaluation report: The report should be written (and fully edited) in English (max 35 pages), and consist of:

▪          Title page and other initial pages

▪          Table of contents.

▪          Executive summary that  includes context summary, brief project introduction, methodological approach, key findings and recommendations (no more than 3 pages).

▪          Current context analysis (approximately 2 pages)

▪          Project introduction including the goal, objectives, expected results, summary activities and Evaluation methodology, detailed explanation of sample size calculation and distribution across locations, gender and ethno-religious identity (maximum 3 pages)

▪          Evaluation findings, analysis, and conclusions. The findings section will have sub-chapters to cover the evaluation criteria and key research questions. This is the core of the evaluation report. (Maximum 23 pages)

▪          Lessons learned and good practices (2 pages)

▪          Conclusions and Recommendations Strategic and activity-focused recommendations. Each recommendation should be directly linked to the evaluation findings (2 pages) Targeted recommendations for (a. targeted communities, b. Search and implementing partners, c. government and d. donors) for improvements based on findings during the evaluation process (e.g., for sustainability, future program design).

▪          Appendices, which include, research instruments, list of interviewees, bibliography, and consultant (s) brief biography.

▪          One electronic copy (in MS Word document) in English.

▪          2–3-page document capturing key results of the report in English.

Search Pakistan will provide a report template with appropriate branding, color and fonts with the consultant team after signing the contract.

Remuneration

 

The payment modality of the selected firm is as follows:

1)  25% at the inception stage

2)  25% after data collection

3)  25% at the submission of the draft report

4)  25% at the approval of the final report.

Timeline

 

The evaluation assignment will be for two months from the signing of the contract.

Requirements of the Applicants

 

The following skills and experience are expected from the evaluation firm for this project:

▪          Registered Pakistani organization with at least five years of relevant experience since registration

▪          Experience in developing qualitative and quantitative research methods

▪          Strong understanding of the conflict dynamics, media landscape in Pakistan and conflict sensitive journalism

▪          Clear understanding of OECD-DAC criteria of evaluation

▪          Experience in carrying out project evaluations in conflict transformation and peacebuilding

▪          Experience in media programming and/or evaluation

▪          Knowledge in conflict-sensitive project management

▪          Good analytical and writing skills, and the ability to clearly present findings, draw practical conclusions, and develop lessons learned

▪          Excellent skills in English required; and the team members should be fluent in relevant local languages spoken by local communities including Urdu, Pashto, Balochi, and Saraiki. Fluency in other regional languages of KP and Balochistan is a plus.

▪          Ability to travel to the field for data collection and cross-checking the data

▪          Good organizational and communication skills with international/ national officials and rural communities are required

In addition, the consultant is required to respect the following Ethical Principles^[2]:

• Comprehensive and systematic inquiry: Consultants should make the most of the existing information and full range of stakeholders available at the time of the review. Consultants should conduct systematic, data-based inquiries. They should communicate their methods and approaches accurately and in sufficient detail to allow others to understand, interpret and critique their work. They should make clear the limitations of the review and its results.
• Competence: Consultants should possess the abilities and skills and experience appropriate to undertake the tasks proposed and should practice within the limits of their professional training and competence.
• Honesty and integrity: Consultants should be transparent with the contractor/constituent about: any conflict of interest, any change made in the negotiated project plan and the reasons why those changes were made, and any risks that certain procedures or activities produce misleading review information.
• Respect for people: Consultants should respect the security, dignity and self-worth of respondents, and program participants. Consultants have the responsibility to be sensitive to and respect differences amongst participants in culture, religion, gender, disability, age and ethnicity.

Selection Criteria

Consultant will be selected based on based on:

1.       The quality of the technical proposal submitted: knowhow of the subject matter, Relevance of proposed methodology to the goal, objectives, and research questions for the project, Quality of proposed methods, presentation of analytical approach conflict sensitivity approaches, and quality control measures, academic qualification and professional experiences of the proposal team members, team combination (50%)

2.       Quality of the writing sample and its relevance to the assignment in concern (10%)

3.       Performance in the interview: Presentation ideas, ability to clarify questions raised from the proposal, explanation of the field data collection team formation and explanation of explanation quality assurance approach (10%).

4.       Proactive engagement of the team members during discussion (5%

5.       Commitment to flexibility and timely completion of assignment, timeline for proposed activities (5%)

6.       Budget proposed in the financial proposal: Justification of the rate and no of team

members proposed, total amount proposed in relation to proposed methodology, deliverables and team and balance between HR budget and other field related costs (20%).