REF Pakistan Handbook

This draft is an outline and will be further developed after the outcomes of the pilot are evaluated.

Developed by the REF Pakistan Project Team

Introduction

This Research Excellence Framework (REF) Pakistan University Handbook consolidates the operational, methodological, and procedural guidance for universities who are engaging with the new national research excellence evaluation system, the Research Excellence Framework (REF) for Pakistan. Developed collaboratively by Pakistani and international partners, on behalf of the Higher Education Commission (HEC) of Pakistan, it adapts best practices from the UK Research Excellence Framework (REF), ERA Australia, Italy VRQ, Brazil Qualis, South Africa’s NRF and others to create a model that reflects Pakistan’s higher education landscape and aspirations.

The HEC has commissioned the development of the REF for Pakistan as a national system to assess the quality, impact and integrity of research across Pakistan’s higher education sector. Its purpose is to strengthen the country’s research ecosystem, support evidence-based funding decisions, enhance institutional capacity, and align Pakistan with international standards of research evaluation. REF Pakistan therefore supports the ambitions of HEC Vision 2025, which emphasises Access, Quality, Relevance, Research Excellence and Governance as core pillars of national transformation.

The proposed framework is intended to encourage impactful, context-sensitive research; promote disciplinary and regional inclusivity; and to align with national priorities including the Sustainable Development Goals (SDGs), URAAN Pakistan, and sectoral strategies in health, agriculture, digital transformation and climate resilience

REF Pakistan is built around three interconnected pillars: Research Outputs (60%), Research Environment (25%), and Research Impact (15%). This 60/25/15 weighting recognises that excellence extends beyond publications, incorporating how research is enabled and how it benefits society. Each component is governed by detailed rubrics and evidence descriptors to ensure comparability, fairness, and transparency.

The Framework Lifecycle

Once fully operational, the REF for Pakistan will run on a multi-year cycle. Each cycle will include:

A planning and guidance phase
A period for institutional submission
Assessment by expert panels
Publication of results

The framework’s lifecycle covers every stage of evaluation: from institutional submission to reviewer assessment, scoring, moderation, and benchmarking of
evidence.

Oversight and administration are led by the REF Pakistan Secretariat housed within the HEC, supported by the Advisory Committee to uphold independence and integrity.

The Rationale for the Framework

The model is designed to reward excellence beyond publications by valuing the conditions that enable high-quality work and the pathways through which research delivers benefits beyond academia. By embedding transparency, proportionality, and evidence-based decision-making, REF Pakistan represents a
significant step toward a sustainable, high-integrity national research assessment model. It will foster continuous improvement, fair recognition of excellence, and informed investment in research capacity across Pakistan’s higher education sector.

As mentioned above the REF Pakistan framework evaluates research excellence through three interdependent pillars — Research Outputs (60%), Research Environment (25%), and Research Impact (15%). Each pillar reflects a critical dimension of a strong and internationally credible research ecosystem.
Research Outputs (Pillar A) measure what knowledge Pakistan produces — its originality, rigour, and scholarly contribution.
Research Environment (Pillar B) assesses how that knowledge is generated — the leadership, culture, and infrastructure that sustain excellence.
Research Impact (Pillar C) captures why the research matters — its tangible benefits for society, policy, and the economy.

Together, these pillars provide a balanced picture of both academic quality and national relevance. Each pillar is rated on a 1-to-4* scale, calibrated against international standards, and weighted (60: 25:15) to produce a single overall score out of 4, determining an institution’s final standing.

Over time, it will help align Pakistan’s research with international benchmarks, attract global collaboration, and demonstrate the country’s contribution to innovation and development.

How the Process Works Step-by-Step

The REF4Pakistan research evaluation process by following a PDCA (Plan–Do–Check–Act) cycle to ensure fairness, learning, and transparency.

Universities choose which research to submit and prepare short Context Statements explaining what it is, why it matters, and who did it. They also describe their research environment and impact stories.

Research is uploaded to the REF4Pakistan platform. Each item is reviewed by two experts (one from Pakistan and one from abroad) using simple rubrics to assess originality, significance, and rigour.

Reviewers compare results in group meetings to ensure fairness. Moderators check for consistency. Metrics like citations are used for context only — not automatic scoring.

Scores are combined to create a quality profile for each subject area. HEC and panels provide feedback to help universities learn and improve. Results also help decide fair research funding.

The PDCA Cycle - Simple Flow

Purpose of the Handbook

This handbook serves four key purposes:

To promote national consistency in evaluating research excellence.
To strengthen the capacity of universities and reviewers through training and transparent tools.
To provide a fair and auditable process for assessing outputs,
environments, and impacts.
To align Pakistan’s research assessment with global standards, enhancing international credibility.
Together, the sections of the handbook present an end-to-end operational
ecosystem for REF Pakistan:

Section 1 Submission Guidance supports institutions in assembling compliant submissions.
Section 2 Rubric defines the 60/25/15 scoring model and standards of judgement
Section 3 Evidence Tables translates those standards into practical indicators.
Section 4 Case Studies which demonstrate how quantitative and qualitative evidence can align with ratings.

Section 1 Submission Guidance

1.1 Introduction

This section of the University Handbook will in final copy have an introduction section explaining:

A brief overview of how the REF operates in Pakistan
How universities will engage and be eligible for submission
How the Compliance Checklist will be used to determine eligibility for
participation
Technical support

For the piloting process this is described separately

Using This Handbook

This handbook provides a comprehensive step-by-step guide for universities and research institutions preparing submissions to the REF Pakistan assessment. Every Higher Education Institution (HEI) participating must submit three document bundles, one per pillar of the REF Pakistan framework for each assessment submission.

The handbook explains how to select the best research material for each assessment submission, what to submit for each pillar (Outputs, Environment, Impact), how to prepare evidence, why each element matters, and what reviewers will be looking for. Following these instructions will ensure completeness, comparability, and a fair and equitable assessment process.

1.2. Understanding the REF Pakistan Model

Pillar	Weight	What It Measures	Why It Matters
Research Outputs	60%	Quality, originality, and rigour of research.	Demonstrates Pakistan’s scholarly excellence.
Research Environment	25%	Institutional systems and leadership that sustain research.	Shows capacity, culture, and sustainability.
Impact	15%	Reach and significance of benefits beyond academia.	Proves real-world value of research.

1.3. Pre-Submission

The REF Pakistan submission process requires universities to assemble a comprehensive, evidence-led portfolio demonstrating the quality, relevance, and contribution of their research. Each participating institution prepares a structured package consisting of research outputs, an environment narrative, impact case studies, and the accompanying evidence required for the full assessment submission. This handbook provides clear guidance so that all universities provide information in a consistent and comparable manner.

It is anticipated that the REF submission process will begin many months before the submission deadline, as institutions must identify which staff are eligible, which outputs should be submitted, and which impact cases show independently evidenced influence beyond academia. Universities will typically establish an internal Ref Pakistan steering committee to coordinate data gathering, verify evidence, and ensure that claims made in the narrative are matched by documentary support. Drafts of the outputs, environment statement, and impact case studies can then be reviewed internally before final submission to refine clarity, accuracy, and alignment with the star-level descriptors for each pillar.

Universities will need to designate a REF Pakistan Coordinator, who is familiar with data management, so most likely to be an existing ORIC or QEC co-ordinator. They will need to coordinate with colleagues to put together the qualitative evidence, the research impact, the case study etc. A vice chancellor or pro-vice chancellor may want to be nominal lead. HEC expect universities to choose the approach that best matches current internal processes.

Panel Structure and Selecting Assessment Submission Themes

Each participating institution submits research classified into specific disciplinary categories, which will then form the basis of assessment submissions to the thematic reviewer panels, with sub-disciplines. Each assessment submission is evaluated by an expert sub-panel, overseen by a main panel (A–E) as outlined below. The lead Vice Chancellor selects which of the disciplines/sub-disciplines might provide the strongest evidence. This may depend on which researchers or faculties or departments are producing the highest quality outputs. Each chosen sub-discipline then forms individual assessment submission bundles. An assessment submission comprises a complete set of data about staff, outputs, impact and the environment returned by the university in any of the chosen sub- disciplines.

The REF Pakistan Coordinator prepares a structured package for each assessment submission, consisting of research outputs, an environment narrative and up to two impact case studies for each submission bundle. The number of outputs for each assessment submissions is limited to ten in total. Each selected researcher usually contributes several outputs, internally selected for:

Originality
Significance
Rigour

Whilst the REF Pakistan Coordinator will be the primary operator collecting the submission together this will require collaboration with academic staff. There is accompanying evidence on submission presentation further in the Handbook. This provides clear guidance so that all universities provide information in a consistent and comparable manner. Offering clear guidance on how the submitted materials will be organised for assessment across Panels A–E. Universities should draw from quantitative and qualitative sources to ensure their submission is comprehensive. A robust submission requires triangulated evidence that proves the university’s research was the material and verifiable driver of change. Quantitative evidence includes metrics like policy adoption figures, economic indicators, health outcomes, and participation numbers.

Qualitative evidence includes policy documents, independent testimonials, legislative references, and media coverage from external beneficiaries. This evidence must meet REF Pakistan’s standards for independence, credibility, and traceability.

Selecting Impact Case Studies is a major strategic task that must begin early, as institutions are limited to submitting up to two case studies per submission. This restriction requires universities to prioritise cases demonstrating sustained, significant, and well-evidenced impact. Universities should maximise performance by conducting internal review cycles to assess potential cases against star descriptors, evidence strength, and alignment with national priorities. Only the strongest examples of societal, economic, cultural, or policy impact should proceed to final submission.

The Reviewer Panel Themes

Panel A – Medicine, Health & Life Sciences

Scope: Clinical medicine, biomedical and biological sciences, public health, nursing, pharmacy and allied health, including work on infectious and non- communicable diseases, mental health and ageing.

Sub-disciplines

A1 – Clinical Medicine
A2 – Public Health, Health Services & Primary Care
A3 – Allied Health Professions, Dentistry, Nursing & Pharmacy
A4 – Psychology, Psychiatry & Neuroscience
A5 – Biological Sciences
A6 – Agriculture, Food & Veterinary Sciences

Panel B – Physical Sciences, Engineering and Mathematics

Scope: Physical sciences, engineering and mathematics, including physics, chemistry, earth sciences, materials, mechanical/electrical engineering, computer science and applied mathematics, with a strong link to energy, climate, infrastructure and digital transformation.

Sub-disciplines

B1 – Earth Systems & Environmental Sciences
B2 – Chemistry
B3 – Physics
B4 – Mathematical Sciences
B5 – Computer Science & Informatics
B6 – Engineering

Panel C – Social Sciences

Scope: Economics, sociology, political science, human geography, education, psychology, anthropology, communication and related fields, with a focus on governance, social justice, development and policy impact.

Sub-disciplines

C1 – Architecture, Built Environment & Planning
C2 – Geography & Environmental Studies
C3 – Archaeology
C4 – Economics & Econometrics
C5 – Business & Management Studies
C6 – Law
C7 – Politics & International Studies
C8 – Social Work & Social Policy
C9 – Sociology
C10 – Anthropology & Development Studies
C11 – Education
C12 – Sport & Exercise Sciences, Leisure & Tourism

Panel D – Arts & Humanities

Scope: History, literature, philosophy, languages, cultural and religious studies, archaeology and creative arts, including practice-based work and public/heritage engagement.

Sub-disciplines

D1 – Area Studies
D2 – Modern Languages & Linguistics
D3 – English Language & Literature
D4 – History
D5 – Classics
D6 – Philosophy
D7 – Theology & Religious Studies
D8 – Art & Design: History, Practice & Theory
D9 – Music, Drama, Dance, Performing Arts, Film & Screen Studies
D10 – Communication, Cultural & Media Studies, Library & Information Management

Panel-E: Interdisciplinary & Cross-Cutting Research

Scope: Panel-E covers research that spans two or more main panels (A–D) and is organised around Pakistan’s national priority themes rather than a single discipline.

Sub-disciplines

E1 – Water, Agriculture & Climate Resilience (WACR)
E2 – Energy, Environment & Sustainable Cities (EESC)
E3 – Digital Futures, AI & Data for Society (DADS)
E4 – Peace, Security, Governance & Justice (PSGJ)
E5 – Gender, Inclusion & Social Innovation (GISI)

Submission Checklist

It will take some time to put together evidence, and the following checklist may help the REF Pakistan Coordinator in preparation. Before beginning detailed drafting, each institution should confirm that basic governance, data, and evidence arrangements are in place to support a credible, verifiable submission. The checklist is designed to help REF Pakistan Coordinators and steering committees ensure that these essentials are in hand well in advance of the formal submission deadline.

Category	Item	Why It is needed	Status
Governance	Vice-Chancellor approval	Confirms institutional accountability	☐
Governance	Named REF Pakistan contact	For all correspondence	☐
Outputs	10 selected outputs	Focus on quality, not volume	☐
Outputs	DOIs or digital copies	Allows verification	☐
Environment	Funding and staffing data (3 years)	Supports scoring	☐
Environment	Policies for mentoring and EDI	Shows inclusivity and culture	☐
Impact	At least one verified case study	Needed for 15% weighting	☐
Technical	Metadata form and correct filenames	Avoids upload errors	☐

1.4 Managing Submissions Internally

Universities will need to designate a REF Pakistan Coordinator, who is familiar with data management. They will need to coordinate with colleagues to put together the qualitative evidence, the research impact, the case study etc. We see this as the collaborative team. HEC expect universities to choose the approach that best matches current internal processes and structures.

The likely roles within the collaborative team:

Pakistan Academic Lead – lead of the overall piloting submission A vice chancellor or pro-vice chancellor may want to be nominal lead.
A designated REF for Pakistan Coordinator, someone who is experience in assembling data and evidence – so most likely to be an existing ORIC or QEC co-ordinator. They will be responsible for ensuring the accuracy, authenticity, and traceability of all quantitative and qualitative data submitted
Departmental Coordinator. If the research outputs are selected from specific sub-disciplines someone with an academic background will need to coordinate the internal selection of outputs.
Impact Coordinator who will select impact cases, gather strong stakeholder evidence and draft the verified impact case study This may be the same as the Departmental Coordinator
Administrative Support collaborating with the REF for Pakistan Coordinator and dedicated to managing the logistical, documentary, and coordination tasks necessary to centralise the submission workflow.
Volunteer peer reviewers rom Research Committee who will review the chosen outputs.

Internal Workflow for Universities

Stage	Activity	Responsible Office	Deliverable
1	Appoint REF Pakistan Coordinator and Steering Group	Vice-Chancellor / Research Office	Governance approval
2	Collect and shortlist outputs	Faculty / Department Coordinators	Draft output list
3	Internal peer review	Research Committee	Validated outputs
4	Draft Environment Statement	Research Office	Version 1 complete
5	Develop Impact Cases	Impact coordinator / PI	Verified evidence files
6	Cross-check with rubrics	Internal REF Pakistan Panel	Final review complete
7	Upload and submit	REF Pakistan Coordinator	ZIP package and confirmation

1.5. Detailed Instructions for Each Pillar

A. Research Outputs (60%)

Purpose: Demonstrate the intellectual quality and contribution of the university’s strongest research.

Method: Submit up to 10 outputs (articles, books, datasets, patents, or creative works). See Appendix A For the pilot this is 3-5 Outputs

All outputs that are listed in submissions will be made available to the relevant sub-panel. Reviewers look for originality, transparency, rigour, and peer validation. Select distinct and verifiable examples rather than many similar Each output requires a 150-word summary explaining originality, rigour, and contribution.
The HEI must provide verification for each output. For journal articles and conference proceedings, the submission must include a Digital Object Identifier (DOI) wherever available, to enable the REF Secretariat to source these outputs from the publishers. Where it is not possible to source particular outputs from the publishers, the HEI must provide an electronic Where supplementary information has been published for the output, the submission must include a DOI (or other URL, if no DOI is available) for this, where it differs to the DOI of the submitted output.
For all other types of output, the HEI must provide an electronic copy wherever this is available; or where it is not, a physical output or appropriate evidence of the (DOI, ISBN, acceptance letter, or URL).
The output must have been deposited as the author’s accepted manuscript. Where the published ‘version of record’ is available for deposit within the required timeframe, and where the journal or conference publisher permits it, the ‘version of record’ may be deposited instead of the accepted manuscript. Outputs that have been provisionally accepted for publication, under the condition that the author revises the manuscript that result from peer review, are not considered as the final text.
An underpinning principle of the REF or Pakistan is that all types of research and all forms of research output across all disciplines shall be assessed on a fair and equal basis, including interdisciplinary and collaborative Panel E covers research that spans two or more main panels (A–D) and is organised around Pakistan’s national priority themes rather than a single discipline.
It is important to ensure consistency in author names, affiliations, and acknowledgements.

B. Research Environment (25%)

Purpose: This shows how research capacity, culture, and sustainability have been nurtured within the HEI.

Method: Submit a 10-page Environment Statement (See Appendix B template) covering:

Research Vision and The context and mission of the HEI with an overview of the size, structure and mission of the institution. The institution’s strategy for research and enabling impact (including integrity, open research, and structures to support interdisciplinary research). Provide evidence of policies and outcomes, not aspirations.
People and Culture (mentoring, equality, inclusivity) The institution’s staffing strategy, support and training of research students, and evidence about how equality and diversity in research careers is supported and promoted across the institution. Include examples of implemented initiatives such as mentoring systems or internal grant schemes. Where a submission includes staff from distinct faculties, departments or other organisational units, the submission should explain this and any distinctive aspects of the research environments of these organisational units, within each section of the environment template. The environment element of a submission will not usually relate only relate to a single department or coherent organisational unit, but that additional information adds context. If the chosen sub-disciplines are housed in one faculty or department the same statement may be duplicated.
Infrastructure and Funding (including 3-year data tables) Income, infrastructure and facilities: the institutional-level resources and facilities available to support research, including mechanisms for supporting the reproducibility of research as appropriate to the research focus of the HEI, and to facilitate its impact
Sustainability and Future Plans – Include data tables on research income, staff, and PhD completions.
Partnerships and Collaboration

C. Impact (15%)

Purpose: For the purposes of the REF for Pakistan, impact is defined as an effect on, change or benefit to the economy, society, culture, public policy or services, health, the environment or quality of life, beyond academia.

Method: – Submit one or two impact case studies (max 5 pages each). Each case study must include at least one independently verifiable piece of evidence.

Reviewers will verify all the impacts claimed.(See Appendix C template) For the pilot only one is required

Each must include:

a.) Underlying research summary (with DOI)

b.) Pathway to impact (how change occurred)

c.) Evidence of reach and significance

d.) Independent verification (letters, citations, evaluation data) Each case study must include at least one independently verifiable piece of evidence. Reviewers will verify all the impacts claimed.

e.) Provide quantitative (numbers, reach) and qualitative (stories, testimonials) evidence.

2. Impact includes, but is not limited to an effect on, change or benefit to:

a.) the activity, attitude, awareness, behaviour, capacity, opportunity, performance, policy, practice, process or understanding. This include the reduction or prevention of harm, risk, cost or other negative effects expressed as a positive. Examples:

i. Policy changes

ii. Clinical guidelines

iii. Economic or environmental improvements

iv. Cultural enrichment

b.) This may be for an audience, beneficiary, community, constituency, organisation or individuals

c.) in any geographic location whether locally, regionally, nationally or internationally. An institution may make a strong local impact. Impacts will be assessed in terms of their ‘reach’ and ‘significance’ regardless of the geographic location. Many impacts will contribute to the economy, society and culture within Pakistan, but international or other country impact is equally valued.

3. Academic impacts on research or the advancement of academic knowledge (whether int he UK or internationally) are assessed within the ‘outputs’ and ‘environment’ elements of REF for Pakistan.

4. Impacts on students, teaching or other activities both within and/or beyond the submitting HEI are included.

Common Pitfalls in Creating an Impact Case Study

Over-statement: Avoid making grand claims that cannot be traced back to the research.
Academic Impact Only: Ensure the impact is “beyond academia.” Citations in other journals are counted in Pillar A (Outputs), not Pillar C (Impact).
Missing Evidence: A case study with no independent verification will likely be rated as Unclassified (U). Start collecting testimonial letters and policy reports early in the pilot cycle (Stage 2).

Summary Table for Evidence through Pillars

Pillar	Template	Key Content	Supporting Evidence
Research Outputs (60%)	Output Context Statement (≈150 words per output)	Explains originality, significance and rigour of the output. Must clarify authorship/contribution, use of data, ethics.	Authorship/contribution statement; reproducible data/code where applicable; peer validation or contextual indicators.
Research Environment (25%)	Environment Statement	3-year overview of strategy, EDI data, open science, partnerships, sustainability.	Quantitative and qualitative data drawn from ORIC self- assessment, HR records, funding data, partnerships, doctoral pipeline, etc.
Research Impact (15%)	Impact Case Study (1,500 – 2,000 words)	150-word summary, underpinning research and evidence, pathways to impact, beneficiaries, outcomes, supporting evidence list, equality and ethics considerations.	Reports, evaluations, beneficiary data, testimonies, government/policy documents, etc.

1.6 Additional Required / Referenced Documents

In addition to the 3 pillars, there are two required annexes (Inclusivity Plan + Responsible Metrics) and three optional/conditional evidence files (authorship, reproducibility, validation).

Inclusivity Plan

This will be included in a later version Not required for piloting

Responsible Metrics Annex

The Responsible Metrics Annex is referenced to guide contextual use of indicators (citations, downloads, etc.). The draft also references supporting documents such as authorship/contribution statements and reproducible data/code where applicable. This shows how they are used “responsibly. This confirms how any data has been informed by the research environment.

Document	Purpose / Source	How it Links
Inclusivity Plan	Required at the PLAN stage of the PDCA cycle. Describes how EDI, ECR, gender, disability and regional participation are ensured in submissions and institutional processes.	Forms part of Environment evidence and informs cross-cutting rubric (Inclusivity & Responsible Metrics).
Responsible Metrics Annex	Details field-normalised indicators (citations, downloads, etc.) and shows how they are used “responsibly.”	Attached to both Output and Impact submissions where quantitative data appear; assessed under the cross-cutting rubric.
Authorship / Contribution Statement	Confirms roles for multi-author works.	Uploaded alongside each output file or artefact.
Reproducible Data / Code (if applicable)	Demonstrates research integrity and transparency.	Either linked repository URL or uploaded supplementary file.
Peer Validation and Contextual Indicators	Independent verification or benchmarking.	Optional but encouraged for 4* or 3* outputs.

1.7 Formatting Standards

Templates will be provided to institutions in Word. Completed templates and case studies must be submitted as PDF documents for the assessment. PDF documents must be accessible to screen reading technology. A Word version of the templates and case studies will also be required. (rather than scanned documents). Completed templates must adhere to the following:

Arial font, 11 points (minimum)
single line spacing (minimum)
2 cm margins (minimum)

Avoid marketing language or imagery. REF Pakistan is an evidence-based exercise focusing on clarity and traceability. The following table gives some guidance:

Section	File Type	Max Length	Key Format Rules
Outputs List	Word / Excel	10 entries	Title, authors, DOI, 150-word summary
Environment Statement	Word / PDF	10 pages	Use headings and data tables
Impact Case Study	Word / PDF	5 pages each	Include timeline and verification evidence
Supporting Evidence	PDF	5 MB total	Attach verified documents only
All Files	–	–	Arial 11pt, 1.15 spacing, UK English

Evidence Quality Expectations

Triangulation: combine metrics/outcome data with corroborating third-party documentation.

Independence: evidence should be external to the institution or clearly verifiable.

Traceability: documents should be dated, attributable, and linked directly to specific claims.

Common Pitfalls in Submission and How to Avoid Them

Problem	Why It Happens	How to Prevent
Overly descriptive narrative	Lack of academic input	Pair administrators with research leads.
Missing verification links	Late-stage quality control	Schedule a link-checking session pre-submission.
Weak impact evidence	Beneficiary contact delayed	Collect testimonials during project delivery.
Overlap between pillars	Same data reused	Maintain separate folders per pillar.
Late submission	Poor coordination	Set internal deadline two weeks early.

Suggested Quality Assurance Process

Method: It is recommended that a senior academic unconnected to the writing team complete a final read-through using these questions:

Are all outputs peer-reviewed or quality-assured?
Does the Environment Statement include evidence and data, not aspirations?
Is each impact claim backed by independent proof?
Are all figures accurate and dated?
Are file names and formats compliant?
Has internal peer review been completed?

1.8. Final Submission Steps

Compile folder structure (Outputs, Environment, Impact, Annexes, Admin).
Check all version numbers (‘Final v1’ only).
Zip and name file as REF Pakistan_[University Name]_Panel_Sub- [Discipline]_[ date].zip.
Upload to secure REF Pakistan portal (link via Secretariat).
Save confirmation email and retain for
Respond to any reviewer clarification within 10 working

Section 2. Evaluation Rubric and Weighting Rationale

2.1 Introduction

Following consultation across the Research Excellence Framework Steering Group and international comparators (UK REF, ERA Australia, Italy VRQ, Brazil Qualis, South Africa’s NRF), REF Pakistan adopts the weighting model 60:25:15

Pillar	Weight	Description
Research Outputs	60%	Primary measure of originality, rigor, and contribution to knowledge.
Research Environment	25%	Assesses institutional capacity, culture, leadership, and sustainability.
Impact	15%	Evaluates the reach and significance of research benefits beyond academia.

Dimensions used to guide judgement include:

Outputs (indicative dimensions)

The primary goal is to elevate Pakistan’s global research standing by incentivising original inquiry, strong methods, and internationally visible publications. It seeks to move institutions from quantity-driven output models to quality-led contributions — where research is not only produced but also cited, replicated, and embedded in international debates.

Typical evidence markers

Originality and contribution: what is new, and why it advances the field (discipline-appropriate).
Rigour: transparent methods and validation; ethics where relevant; reproducibility/data provenance where applicable.
Recognition (contextual): peer-reviewed dissemination and appropriate indicators of visibility/uptake (not automatic scoring).

Assessment Dimensions:

Originality (20%)
Rigour and methodological quality (20%)
Contribution to knowledge and field (10%)
Dissemination and recognition (10%)

Environment (indicative dimensions)

This pillar assesses the institutional systems, culture, and leadership that enable high-quality research to thrive. It examines how universities organise, support, and sustain research activity through strategic direction, policies, and resourcing. The focus is not only on infrastructure — such as labs and funding — but also on people, culture, and inclusivity. Strong environments foster mentoring, gender equality, interdisciplinary collaboration, and effective governance. This pillar aims to improve long-term research capacity and culture in Pakistan. It encourages leadership accountability, continuity of funding, and institutional alignment with national research priorities

Typical evidence markers

Strategy and governance: clear research direction and delivery evidence (KPIs, structures, integrity/open research).
People and culture: mentoring and doctoral support; EDI actions with outcomes; internal peer review and capability-building.
Infrastructure and sustainability: facilities and digital systems; diverse income; three-year staffing and doctoral pipeline evidence.

Assessment Dimensions:

Strategic leadership and alignment (10%)
Research culture, support, and collaboration (10%)
Infrastructure and sustainability (5%)

Impact (indicative dimensions)

This pillar measures the real-world influence and benefit of research beyond academic circles. It captures how knowledge generated in Pakistani universities contributes to social, economic, policy, environmental, or cultural change.

The focus is on reach (how widely research benefits are felt) and significance (how deeply they improve people’s lives, policies, or practices). It values both quantitative measures — such as policy adoption or economic outcomes — and qualitative evidence like testimonials, case studies, and community engagement.

The intent is to bridge the gap between research and national development goals, encouraging universities to work more closely with government, industry, and civil society. Through this pillar, REF Pakistan aims to foster a culture where knowledge is mobilised to solve local and global challenges — demonstrating that Pakistani research is impactful, relevant, and transformative.

Typical evidence markers

Reach: who benefited and at what scale (local to international), supported by credible documentation.
Significance: depth of change (policy/practice/outcomes), ideally with pre/post indicators or independent evaluation.
Verification: at least one independent, dated source that corroborates the key claims.

Assessment Dimensions:

– Reach (7.5%)

– Significance (7.5%)

2.3. Pillar Weighting and Scoring

The scoring process for the REF Pakistan submission involves two main steps: first, the expert sub-panel assigns a star rating (4* to U) to each of the three pillars (Outputs, Environment, Impact) based on the evidence provided and the rubrics. Second, the final overall score is calculated by taking the weighted average of these three pillar ratings, using the ratio 60%:25%:15%. For example, a 4* rating for Research Outputs (4 points) contributes 4 x 0.60 = 2.40 to the total score. The final score, out of a maximum of 4.00, determines the institution’s overall standing.

The final weighted average score, calculated out of 4.00, translates directly into an overall quality profile for the assessment submission. This profile provides a national benchmark for excellence and will at the appropriate time be used for evidence-based funding and strategic planning. The interpretation table below provides the full scale for translating the numerical score into a descriptor of excellence.

Rating	Descriptor	Typical Evidence
4* (Outstanding)	Exceptional reach and significance, with clear transformative outcomes beyond academia.	Policy adoption, social/economic benefits, measurable cultural or practice change.
3* (Very considerable)	Strong, well-evidenced impacts on defined user groups or sectors.	Documented use, citations in policy, public engagement outcomes.
2* (Considerable)	Positive but localised or emerging impact.	Regional or pilot-scale outcomes; qualitative indicators.
1* (Modest)	Limited reach or early- stage impact.	Indirect or anecdotal evidence.
Unclassified (U)	No demonstrable impact.	No evidence or link to research.

Section 3. Evidence and Rating

Research Outputs (60%)

Research Outputs, weighted at 60%, are the primary measure of the knowledge produced by Pakistani universities. The assessment is holistic, evaluating outputs based on four dimensions: Originality, Rigour, Contribution to Knowledge, and Dissemination. This approach moves beyond simple citation counting to ensure that all forms of research, including locally relevant, non-English, and interdisciplinary work, are fairly assessed for their scholarly merit and quality.

Evidence Type

Originality

Limited conceptual innovation, derivative or descriptive work.

Why: Shows awareness of field but little advancement of theory or method.

Some new data or interpretation but within established frameworks.

Why: Adds modest novelty to known debates.

Presents new theoretical or methodological insights that challenge or extend existing work.

Why: Demonstrates creative advancement recognised by peers internationally.

Defines or redefines paradigms or methods; transformative originality acknowledged by global leaders.

Why: Sets new agenda for the discipline.

Rigour & Methodological Quality

Weak design, limited data, or unclear analysis.

Why: Results are not reproducible or robust

Adequate methodology, some limitations in scope or validation.

Why: Sufficient

quality but lacks refinement.

Strong design, transparent methods, and reproducible outcomes.

Why: Evidence international research standards

Methodologically exemplary, peer- validated and highly cited.

Why: Demonstrates best-practice rigour influencing future studies.

Contribution to Knowledge

Adds little to the field, mostly confirmatory.

Why: Limited reach or narrow audience.

Extends existing literature; small but recognisable advancement.

Why: Shows incremental growth in field knowledge.

Significant conceptual or empirical contribution that influences others’ work.

Why: Cited or discussed internationally.

Ground-breaking or field-defining contribution.

Why: Changes how the discipline understands key phenomena.

B. Research Environment (25%)

The Research Environment pillar (25%) is designed to assess the institutional infrastructure and culture that enables excellent research. By focusing on Strategic Leadership, Research Culture, and Infrastructure/Sustainability, the REF Pakistan framework ensures that universities are rewarded for developing long-term, supportive, and inclusive ecosystems that nurture talent and sustain high-quality work over time.

Evidence Type

Leadership & Strategy

No coherent research vision or leadership structure.

Why: Activities fragmented, reactive, or informal.

Emerging research direction, partial alignment to institutional goals.

Why: Demonstrates early strategy but weak implementation.

Clear vision, well- articulated strategy linked to national priorities.

Why: Demonstrates sustained leadership and goal setting.

Internationally benchmarked strategic framework with measurable outcomes.

Why: Provides model of excellence for others.

Research Culture & Support

Minimal mentoring or researcher development.

Why: Isolated individuals, no structured support.

Basic research training; uneven mentoring provision.

Why: Demonstrates intent to improve culture.

Active mentoring, funding access, internal review processes.

Why: Embedded research support evident in staff outputs

Highly inclusive, diverse, and equitable research culture.

Why: Recognised model supporting sustained

excellence and wellbeing

Infrastructure & Sustainability

Poor access to facilities; no sustainability plan.

Why: Dependent on ad hoc resources.

Some facilities and partnerships; weak long-term sustainability.

Why: Basic physical or digital infrastructure emerging.

Modern facilities, managed resources, partnerships with external funders.

Why: Sustainable systems for growth.

State-of-the- art infrastructure integrated with global networks.

Why: Ensures institutional resilience and knowledge transfer capacity.

Purpose:

To evaluate the institutional and disciplinary context that sustains research quality, integrity and inclusivity.

The HEC Office of Research, Innovation and Commercialisation (ORIC) Self- Assessment Scorecard provides an existing evidence base for evaluating the Environment pillar. Its indicators on governance, funding diversification, research capacity and EDI closely mirror the REF4Pakistan Environment dimensions of strategy, people and culture, infrastructure and sustainability.

A detailed mapping of the ORIC categories against the REF4Pakistan pillars is presented in para 5.2, demonstrating how institutional self-assessment data can serve as baseline evidence for Environment scoring and continuous improvement.

The rubric is summarised below.

Dimension	High	Medium	Low
Strategy & Vision	Clear mission linked to SDGs, interdisciplinarity and national needs.	Partial planning or limited alignment with goals	No coherent strategy or weak links to priorities.
People & Culture (EDI, ECR, Integrity)	Strong mentoring, inclusive recruitment, integrity framework and doctoral pipeline.	Some support systems but uneven practice.	Minimal or absent support; exclusionary culture.
Infrastructure & Open Science	Functional repositories, open data policies, reproducibility systems.	Limited but operational infrastructure.	No open-science systems; weak digital access.
Partnerships & Knowledge Exchange	Active collaborations and community/industry engagement.	Ad hoc or limited partnerships.	Minimal or no external collaboration.
Sustainability & Funding	Diverse, long-term funding and planning.	Emerging diversification but fragile.	Reliant on single or unstable funding source.

Impact (15%)

Impact, weighted at 15%, captures the real-world value of research by measuring its ‘reach and significance’ beyond academia. The goal is to incentivise research that addresses national priorities and contributes measurable benefits to society, the economy, culture, or public policy. Assessment relies on independently verifiable evidence that demonstrates a clear causal link between the university’s research and the resulting change.

Evidence Type

Reach

Impact confined to small or

Internal groups.

Why: Limited dissemination beyond project partners.

Regional or sectoral reach;

recognised within limited user communities.

Why: Early adoption but not scaled.

Broad national or multi-sectoral

adoption; external bodies use outputs.

Why: Demonstrates strong external uptake.

International reach: policy, industry, or

societal transformation evidenced.

Why: Sustained and wide-ranging influence on practice or outcomes.

Significance

Minor or anecdotal changes in practice.

Why: Lacks robust evidence or evaluation.

Tangible improvements in processes, awareness, or decision- making.

Why: Evidence indicates positive change but moderate magnitude.

Substantial and measurable benefit to users or society.

Why: Evidence of lasting improvement with causal link to research.

Transformative benefits with long-term, systemic change.

Why: Research fundamentally alters understanding, policy, or practice globally.

Evidence Quality

Weak documentation; no independent verification.

Why: Insufficient to substantiate claims.

Basic evidence (letters, testimonials, attendance data).

Why: Shows engagement but limited validation.

Independent evaluations, usage data, and policy citations.

Why: Verifiable and credible.

Triangulated, peer-validated, and longitudinal impact data.

Why: Meets international standards of impact assessment.

Section 4. Example Case Studies

This section provides five example case studies from Pakistani universities to illustrate how the 60/25/15 weighting model operates across quantitative, qualitative, and mixed-method research contexts. Each case includes evidence, assessment per pillar, and rationale for star ratings.

Case Study 1: Engineering – Renewable Energy Efficiency in Rural Sindh

Type: Quantitative (STEM)

Institution: Mehran University of Engineering & Technology (MUET)

A five-year applied engineering study developed low-cost solar hybrid systems for rural electrification in Sindh. Published in IEEE and Renewable Energy journals (Scopus Q1).

Quantitative Evidence

7 peer-reviewed articles (2 Q1)
80+ citations, 4 prototypes evaluated
Collaboration with National Rural Electrification Programme

Qualitative Evidence

Local manufacturing enabled in Hyderabad
Embedded in national climate action plan (SDG7)
Training for 30 engineers

Assessment by Pillar

Pillar	Rating	Evidence	Why
Outputs	4*	High-impact publications, validated models	Rigorous design and reproducible results
Environment	3*	Active renewable energy lab, limited by doctoral pipeline	Good leadership, partial sustainability
Impact	3*	Adopted in provincial energy plan, 3 villages electrified	Very considerable reach, emerging national scale

Weighted Score: 3.55 / 4 → Internationally excellent approaching world-leading Showing the process in detail:

Step 1: Assign Ratings per Pillar

Pillar	Rating (*)	Evidence Summary	Rationale (Why)
Outputs	4*	High-impact peer- reviewed publications; validated models; reproducible design.	Demonstrates world-leading originality and methodological rigour.
Environment	3*	Active renewable energy lab; good leadership; limited doctoral pipeline.	Strong institutional support, though partial sustainability and EDI frameworks still emerging.
Impact	3*	Research adopted in provincial energy plan; three villages electrified.	Demonstrates significant reach and growing national influence.

Step 2: Apply Weightings (60 / 25 / 15)

Pillar	Rating (*)	Weight	Weighted Contribution	Calculation
Outputs	4*	0.60	2.40	4 × 0.60
Environment	3*	0.25	0.75	3 × 0.25
Impact	3*	0.15	0.45	3 × 0.15
Total			3.60 / 4.00	Sum of weighted scores

(In this case, the moderation panel rounded to 3.55 to account for minor qualitative limitations in sustainability and scope of impact.)

Step 3: Interpret Final Score

Weighted Average (out of 4)

Descriptor

Interpretation

3.55 / 4.00

Internationally Excellent (approaching World- Leading)

The research demonstrates outstanding originality and rigour, strong institutional environment, and measurable social benefit — positioning it just below world- leading level.

Step 4: Key Learning for Institutions

Outputs drive the score — quality and verifiable originality have the highest influence (60%).
Environment must show structured leadership and sustainability to maintain consistency
Impact rewards evidence of adoption and change, even at emerging

Combined, the process ensures balance between scholarly excellence, institutional readiness, and real-world relevance.

Case Study 2: Education – Girls’ STEM Participation in Balochistan

Type: Mixed Methods (Quantitative + Qualitative) Institution: University of Turbat

Three-year longitudinal study tracking female participation in STEM secondary education, combining survey data (n=2,100) and qualitative teacher interviews.

Quantitative Evidence

2 national reports, 1 journal article (HEC Y-category)
12% increase in STEM enrolment post-intervention

Qualitative Evidence

Insights from 45 teachers
Community co-designed gender-inclusive curricula

Assessment by Pillar

Pillar	Rating	Evidence	Why
Outputs	3*	Strong design, moderate journal visibility	Robust but lacks global reach
Environment	3*	Strong community engagement, weak lab capacity	Building from low baseline
Impact	4*	Integrated into provincial Education Strategy 2023–27	Outstanding societal and policy impact

Weighted Score: 3.15 / 4 → Internationally excellent

Case Study 3: Public Health – Mobile Vaccination Uptake Analytics

Type: Quantitative (Data Science) Institution: Aga Khan University (AKU)

Big-data analytics project modelling COVID-19 vaccination uptake using mobile network data across Karachi and Lahore.

Quantitative Evidence

Published in The Lancet Regional Health (Asia)
2M anonymised data points analysed
Model accuracy 91%

Qualitative Evidence

Policy dialogue with Ministry of Health
Ethical data governance model adopted by NGOs

Assessment by Pillar

Pillar	Rating	Evidence	Why
Outputs	4*	High-impact journal, large dataset	World-leading methodological rigour
Environment	4*	Strong collaborations, ethical oversight	World-leading infrastructure and leadership
Impact	4*	Informed national vaccination outreach	Outstanding reach and significance

Weighted Score: 4.00 / 4 → World-leading

Case Study 4: Social Sciences – Urban Informality and Street Vendors in Lahore

Type: Qualitative

Institution: Punjab University – Department of Sociology

Ethnographic study of street vendors’ resilience and governance in Lahore’s informal economy. Funded by HEC.

Quantitative Evidence

1 monograph, 2 journal articles (HEC X-category)
6 workshops, 3 local consultancies

Qualitative Evidence

Interviews with 120 vendors
Informed inclusive planning policies

Assessment by Pillar

Pillar	Rating	Evidence	Why
Outputs	3*	Thematically rich, moderate visibility	Internationally excellent qualitative scholarship
Environment	2*	Limited interdisciplinary linkages	Developing but uneven culture
Impact	3*	Informed Lahore City Council policy pilot	Very considerable local reach

Weighted Score: 2.85 / 4 → Recognised internationally

Case Study 5: Humanities – Preservation of Indigenous Languages in Gilgit-Baltistan

Type: Qualitative (Cultural/Linguistic)

Institution: Karakoram International University (KIU)

Four-year ethnolinguistic project documenting endangered languages (Burushaski, Wakhi). Combined oral histories, mapping, and digital archive development.

Quantitative Evidence

1 digital archive (150h recordings)
1 edited book (OUP Pakistan)

Qualitative Evidence

Community-led preservation
Workshops in 5 valleys
Curriculum integration in regional education policy

Assessment by Pillar

Pillar	Rating	Evidence	Why
Outputs	3*	Robust qualitative methodology	Internationally excellent for regional relevance
Environment	3*	Supported by linguistic societies	Good sustainability, limited funding
Impact	4*	Policy uptake and preservation outcomes	Outstanding cultural impact

Weighted Score: 3.15 / 4 → Internationally excellent

Summary Table of All Five Case Studies

Case	Discipline	Method Type	Weighted Score	Overall Rating
1	Engineering (Renewable Energy)	Quantitative	3.55	4–3 Borderline
2	Education (Girls’ STEM)	Mixed	3.15	3*
3	Public Health (Mobile Data)	Quantitative	4.00	4*
4	Sociology (Urban Informality)	Qualitative	2.85	2–3
5	Humanities (Linguistics)	Qualitative	3.15	3*