E4H Punjab TA: Capacity Building of PHCC Assessment Teams for Enhanced Consistency and Application of Minimum Service Delivery Standards

Programme

Evidence for Health (E4H) is a Foreign, Commonwealth & Development Office (FCDO)-funded programme aimed at strengthening Pakistan's healthcare system, thereby decreasing the burden of illness and saving lives. E4H (2023-2027) provides technical assistance (TA) to Punjab and also to the Federal and Khyber Pakhtunkhwa (KP) governments. The Punjab component is being implemented by Palladium along with Oxford Policy Management (OPM).

Through its flexible, embedded, and demand-driven model, E4H Punjab will support the government to achieve a resilient health system that is prepared for health emergencies, responsive to the latest evidence, and delivers equitable, quality, and efficient healthcare services. Specifically, E4H will deliver TA across three outputs:

Output 1: Strengthened integrated health security, with a focus on preparing and responding to health emergencies, including pandemics.

Output 2: Strengthened evidence-based decision-making to drive health sector performance and accountability.

Output 3: Improved implementation of Universal Health Coverage, with a focus on ending preventable deaths.

E4H-Punjab works in partnership with the Punjab Department of Health (DOH).

Terms of reference

Background

The Punjab Healthcare Commission (PHCC) is responsible for regulating healthcare establishments and promoting quality and patient safety through the implementation and assessment of Minimum Service Delivery Standards (MSDS).

In 2025, E4H supported PHCC through the Impact Assessment of MSDS Implementation in Punjab. The assessment found that, while MSDS provides an important regulatory framework, assessment practices were not always applied consistently. Key issues included variation in interpretation and scoring, ambiguity in some indicators and evidence requirements, and uneven use of standardised assessment guidance and checklists across facility categories. Healthcare establishments also reported differing interpretations of MSDS requirements by surveyors, contributing to confusion and uneven enforcement.

The assessment recommended continuous capacity building of field assessors, including structured induction, periodic refresher learning, calibration, stronger assessment quality assurance and an assessor competency framework to improve consistency and inter-assessor reliability.

Based on these findings, PHCC has requested E4H support to strengthen the capacity of its assessment teams to apply MSDS requirements more consistently, objectively and reliably across healthcare establishments. This TA is therefore a direct follow-on to the 2025 assessment and will use its findings as the starting point rather than repeat the earlier diagnostic work.

The assignment will also complement E4H's ongoing MSDS e-certification TA. While the e-certification TA strengthens healthcare providers' understanding and implementation of MSDS, this TA focuses on PHCC assessors' capacity to assess compliance consistently and reliably. The two assignments will use aligned PHCC-approved standards, content and systems where relevant and avoid parallel or duplicative approaches.

Problem Statement

Reliable MSDS enforcement depends on assessors interpreting requirements, assessing evidence, applying scoring criteria, documenting findings and communicating feedback consistently. Variation in these practices can result in different judgements being reached on comparable evidence, reducing the consistency, comparability and credibility of assessments. PHCC therefore requires a standardised, practice-based and sustainable approach to assessor competency, calibration and quality assurance that reduces avoidable inter-assessor variation and can be maintained through PHCC's routine systems.

Goal and Objective(s)

Goal

To improve the consistency, objectivity and reliability of PHCC MSDS assessments by strengthening assessor competency and institutionalising a PHCC-owned approach to assessor learning and calibration.

Strategic Objectives

1. Strengthen assessor competency and consistency in interpreting MSDS requirements, assessing evidence, applying scoring criteria and documenting assessment findings.
2. Develop and test a standardised, PHCC-approved assessor learning and calibration package linked to existing quality-assurance processes.
3. Strengthen PHCC's internal capacity to sustain the approach through assessor induction, refresher learning, periodic calibration and ongoing updating of relevant learning resources.

• Build from the 2025 E4H MSDS assessment and validate priority gaps rather than repeat a broad assessment.
• Work through PHCC's existing assessment, regulatory and quality-assurance structures, refining and consolidating existing resources before developing additional tools or materials.
• Use PHCC-approved MSDS, content and systems, aligned with the e-certification TA where relevant, with designated focal persons and a simple mechanism to record alignment issues and decisions.
• Emphasise practical application through realistic cases, evidence review, scoring practice and structured calibration rather than classroom instruction alone.
• Measure both knowledge and consistency using the same or comparable practical exercises before and after training and during follow-up where feasible.
• Document recurring ambiguity or disagreement for PHCC decision. Formal changes to MSDS standards, regulatory scoring methods or checklists remain subject to PHCC's established review and approval processes.
• Transfer ownership through joint development, PHCC internal facilitator preparation, editable resources and integration with induction, refresher learning, supervision and QA arrangements.

Scope of Work and Delivery Methodology

During inception, the TA provider will discuss and confirm the detailed scope, sequencing and implementation approach with PHCC. PHCC steering and concurrence will be documented through inception meeting minutes, agreed action points and pictorial evidence.

Phase 1: Inception and Validation of Priority Gaps

• Review the 2025 E4H MSDS assessment, current PHCC MSDS guidance and manuals, assessor manuals, assessment/scoring tools, training resources, relevant assessment evidence and relevant outputs of the e-certification TA.
• Review current PHCC arrangements for assessor induction, training, supervision, calibration, quality assurance and feedback to identify where the TA should strengthen existing systems.
• Validate the specific competency and consistency gaps to be addressed, focusing on MSDS interpretation, evidence requirements and verification, scoring, documentation, feedback, professional judgement and assessor conduct. This should refine the 2025 findings rather than repeat the earlier broad assessment.
• Identify recurring areas of ambiguity or assessment difficulty that can be used in practical learning and calibration exercises.
• Agree an assessor competency framework and learning outcomes that will guide package development, assessment and follow-up.
• Confirm the assessor cohort, facility categories, priority MSDS areas, training batches, delivery schedule, coordination with the e-certification TA and measurement design. PHCC will  designate a focal person during inception. The focal person will hold monthly coordination meetings, with additional milestone-based meetings before package validation, training delivery and final handover. A shared decision/action log will record alignment issues, agreed decisions, responsible persons and due dates, and will be reviewed at each coordination meeting and appended to relevant deliverables.
• Submit a concise inception package containing the agreed scope and methodology, workplan, risks, competency framework/learning outcomes, measurement approach, coordination mechanism, meeting minutes, action points and pictorial evidence of PHCC concurrence.

Phase 2: Package Development and Baseline Measurement

• Develop or refine a PHCC-approved competency-based curriculum and learning package covering MSDS interpretation, evidence verification, observation, interviewing, document review, scoring, documentation, feedback and follow-up.
• Develop or refine facilitator and participant materials, practical cases/facility scenarios, evidence sets, scoring exercises, scoring guidance, evidence-recording/feedback tools and concise assessor job aids.
• Develop baseline and endline knowledge and practical scoring assessments using the same or comparable cases so that both competency and inter-assessor consistency can be measured. The tools will measure both: (i) knowledge and interpretation of MSDS requirements; and (ii) practical scoring consistency/inter-assessor reliability using common, PHCC-validated cases, evidence sets and an agreed scoring rubric. During inception, PHCC and the TA team will agree a simple scoring-consistency method, including the unit of comparison, acceptable tolerance and reporting format.
• Prepare a calibration methodology and simple analysis framework covering knowledge, scoring agreement, recurring interpretation differences and issues requiring PHCC direction. Improvement will be assessed by comparing baseline and endline results from the same or equivalent standardised cases. The analysis will report the percentage of exact scoring agreement, the percentage of scores falling within the PHCC-agreed tolerance, the spread of scores for each case/indicator, and the number and type of unexplained scoring variations. Where the scoring scale and sample size permit, an appropriate inter-rater reliability statistic may also be reported. Follow-up calibration will apply the same method to assess whether gains are sustained.
• Coordinate content and terminology with the e-certification TA where relevant, validate the package with PHCC and maintain a response-to-comments/decisions record.
• Limit tool refinement under this TA to learning resources, assessor guidance and job aids. Any proposed change to formal MSDS standards, regulatory scoring methodology or official assessment checklists will be documented for PHCC consideration and will not be applied without the required PHCC approval.

Phase 3: Training, Calibration and Results Measurement 

• Deliver the agreed capacity-building programme to the confirmed assessor cohort and prepare PHCC-nominated internal facilitators through joint facilitation and coaching.
• Use realistic case-based and simulation-based exercises and field-linked exercises focused on interpretation, evidence assessment, scoring, documentation and communication of findings. Exercises will include simulated facility assessment cases and examples drawn from real assessment situations where these are available and authorised for use. All cases, evidence sets, expected interpretations and scoring keys will be technically validated and approved by PHCC before use in training or measurement. Any field-linked exercise will require prior PHCC approval and appropriate operational and confidentiality safeguards.
• Administer baseline and endline knowledge and practical scoring assessments and analyse changes in competency and scoring consistency at aggregate level.
• Conduct structured calibration using common cases/evidence sets, compare assessor judgements and document agreed interpretations, recurring variations and unresolved issues requiring PHCC decision.
• Obtain structured participant and PHCC feedback and refine the learning and calibration package accordingly.

Phase 4: Follow-up, Institutionalisation and Handover 

• Conduct one follow-up calibration exercise for a representative cohort, where feasible, and compare findings with baseline and immediate post-training results.
• Finalise a competency and calibration follow-up framework setting out priority indicators, responsible PHCC functions, suggested review points and feasible evidence sources.
• Agree a PHCC-owned approach for assessor induction, refresher learning, periodic calibration and linkage of competency/calibration findings with routine quality-assurance and supervision processes.
• Confirm PHCC ownership, internal facilitator roles, learning-record responsibilities, refresher/calibration arrangements, version control, update procedures and alignment with relevant digital/e-certification systems. During inception, PHCC will nominate the accountable unit/function and responsible focal person(s) for the learning package, calibration records, assessor learning records, refresher and calibration schedule, repository, version control and future updates. Most likely, it will be the Director of Inspection. These responsibilities, including approval authority, review triggers and handover arrangements, will be documented in the inception package and confirmed in the sustainability action plan before TA closure.
• Hand over the approved editable package, results, records, indexed repository and sustainability action plan through a formal PHCC review and acceptance process

Sustainability and Institutionalisation

The end point is a PHCC-owned assessor capacity and calibration approach, not a one-off training event. Implementation will be undertaken in close collaboration with PHCC, with progressive transfer of responsibility throughout the assignment. E4H-supported experts will initially lead and jointly deliver key activities with designated PHCC counterparts, before progressively shifting facilitation, calibration, quality-assurance and resource-management functions to PHCC. By the end of the TA, PHCC should be able to independently deliver, maintain and replicate the assessor capacity-building and calibration approach. Sustainability will therefore be built into delivery by:

• Assigning accountable PHCC ownership through key focal points for learning resources, assessment guidance, calibration, learning records and updates. The specific accountable unit/function and named role-holders will be agreed with PHCC during inception and recorded in the ownership matrix and sustainability action plan.
• Preparing PHCC internal facilitators to support future induction, refresher learning and calibration
• Integrating the approach with existing assessor supervision and QA arrangements rather than establishing a parallel training system
• Establishing feasible competency and consistency indicators for periodic PHCC management review
• Maintaining editable resources through an indexed repository, version control and review triggers following approved MSDS changes
• Aligning with relevant PHCC digital and e-certification systems where appropriate, while avoiding duplicate platforms or content. 

Deliverables

1. Inception and Priority-Gap Validation Package: Focused validation of 2025 findings; review of current PHCC assessor learning/QA arrangements; agreed competency framework and learning outcomes; confirmed scale, sequencing, coordination and measurement plan; workplan and risks.    
2. PHCC Assessor Learning and Calibration Package: PHCC-approved curriculum; facilitator and participant materials; practical cases/evidence sets; scoring guidance/job aids; baseline/endline tools; calibration methodology and analysis framework.
3. Training, Calibration and Results Package: Delivery records for the agreed cohort; baseline/endline competency and scoring-consistency analysis; calibration outputs; internal facilitator preparation; participant feedback; unresolved interpretation issues requiring PHCC direction.
4. Institutionalisation and Handover Package: Final editable resources; follow-up calibration findings where feasible; competency/calibration follow-up framework; ownership and facilitator roles; induction/refresher/calibration arrangements; QA linkage; version control; indexed repository and sustainability action plan. 

Position Title:

Learning, Assessment & Calibration Specialist (Mid National)

Duty Station: Lahore (with travel as required)

Duration / LOE: October 2026 – March 2027 (LOE 40 days – subject to change until the start of the TA)

Reporting To: E4H Programme Team / MSDS Assessment Capacity-Building Team Lead

Role Purpose

Lead the design and implementation of competency-based assessor learning, calibration methodologies, and practical assessment tools to improve consistency and reliability of PHCC MSDS assessments.

Key Roles & Responsibilities

• Develop competency-based learning materials and assessment curriculum.
• Design practical cases, evidence sets, job aids, and calibration exercises.
• Develop baseline and endline competency assessment tools.
• Support delivery of assessor training and facilitator preparation.
• Analyse learning outcomes and inter-assessor consistency.
• Refine learning materials based on participant and PHCC feedback.
• Support institutionalisation of calibration and refresher learning.
• Undertake other TA-related tasks assigned by the Team Lead.

Requirements

Technical Expertise

• Master's degree in Education, Health Professions Education, Public Health, Organisational Development, Assessment, Psychology, or a related field.
• Minimum 10 years of overall professional experience.
• Experience in competency-based curriculum development, adult learning, practical assessment, scoring methodologies, calibration analysis, and facilitator development.

Core Competencies

• Curriculum design
• Competency assessment
• Adult learning
• Case development
• Calibration methodology
• Learning measurement
• Facilitation
• Documentation