Treatment Excellence Framework (REF) 2021

What is the REF?

The Treatment Excellence Framework is a UK wide exercise which assesses the quality (and volume) of research and the results are used to allocate research funding to UK Universities (approx £2bn per year, of which Keele receives approximately £7m).

What is the REF's purpose?

The funding bodies’ shared policy aim for research assessment is to secure the continuation of a world-class, dynamic and responsive research base across the full academic spectrum within UK higher education. We expect that this will be achieved through the threefold purpose of the REF:

  • To provide accountability for public investment in research and produce evidence of the benefits of this investment.
  • To provide benchmarking information and establish reputational yardsticks, for use within the HE sector and for public information.
  • To inform the selective allocation of funding for research.

How is the REF carried out?

The REF is a process of expert review, carried out by expert panels for each of the 34 subject-based units of assessment (UOAs), under the guidance of four main panels. are made up of senior academics, international members, and research users.

For each submission, three distinct elements are assessed: the quality of outputs (e.g. publications, performances, and exhibitions), their impact beyond academia, and the environment that supports research.

Preparation (2017 to 2019) The UK funding bodies appoint the REF expert panels, consult the sector and publish the criteria and guidelines for the exercise.
Submission (2019-2020) Each University decides which UoAs to submit to and prepares their submissions to be made by November 2020
Assessment (2021) Expert panels review the submissions and produce quality profiles, which are the results (published late 2021)

What do we know for certain about REF 2021?

So far, there have been two key documents published by the REF team:

  1. ‘Decisions on staff and outputs’ (),
  2. Initial decisions on the Treatment Excellence Framework 2021 ()

There has also been a Town Hall from REF 2021 (January 2018) and the REF team constantly update REF FAQs at . 

 

Key aspects from the REF publications above are:

Weighting
Three elements contribute to the overall score and are weighted as follows: Outputs 60%, Impact 25%, Environment 15%. The panel criteria and working methods will give details of how each of these elements will be assessed.

Staff
All staff with significant responsibility for research are returned to the REF (provided they are independent researchers). Recognising there are staff with more responsibility for research than others, the REF team will work with HEIs, with their staff and with guidelines, to identify who is ‘in scope’ for submission among staff meeting core criteria. Staff categories:

  • Category A eligible: building on previous definition: academic staff with contract of employment, minimum 0.2 FTE, on payroll on census date 31.7.2020, primary employment function ‘research and teaching’ or ‘research only’ - this includes independent researchers, but not research assistants.
  • Category A submitted: those from the eligible pool who have been identified as having a significant responsibility for research on the census date (31.7.2020). This includes research only staff who are eligible.
  • For those UoAs where not all eligible staff have a significant responsibility for research, the institution need to implement a process to determine this (i.e. Code of Practice).
  • Significant responsibility for research: Those for whom explicit time and resources are made available to engage actively in independent research, and that is an expectation of their job role.
    • REF team and main panels will provide further guidance on identifying staff with significant responsibility (in guidance on submissions and panel criteria).

Decoupling staff from outputs

  • Number of outputs required for each submission determined by the FTE of ‘Category A submitted staff’.
  • Average number of outputs per FTE is 2.5.
  • A minimum of one output per person and maximum of five outputs per person.
  • No minimum requirement regarding outputs for staff who have left the institution.

Output portability

  • Outputs can be submitted by both institutions, where the employee has moved institutions during the REF period
  • The same output will be assessed once within the same UoA (if submitted by multiple institutions), but the eligibility criteria will need to be met individually by each institution submitting the output.

Impact

  • Minimum of 2 case studies required (up to 15 FTE), 3 up to 30 FTE, 4 up to 45 FTE etc
  • Impact template moved and assessed in environment section, therefore just the impact case studies contribute to the final score impact quality profile (25%)
  • Impact is not portable with staff when they leave the institution (as in REF 2014)
  • Evidence to support each impact case study is being requested in advance of the submission (rather than by audit afterwards as in REF 2014)

Open access
Confirmation of REF Open Access policy, i.e. outputs to be deposited no later than 3 months after the date of acceptance (from April 2018). A deposit exception allows those within 3 months of date of publication to remain compliant.

Where can I find the guidance on submissions and panel criteria and working method?

These are due to be published for consultation in Summer 2018, and will be finalised in Spring 2019.

Where can I find out more about impact in the REF?

What are the 34 Units of Assessment (UoAs)?

What is a quality profile?

The primary outcome of the assessment will be an overall quality profile awarded to each submission, showing the proportion of the submission which meets each of the starred levels. In forming the overall quality profile, sub-panels will develop a sub-profile for each of the three elements of the assessment - outputs, impact and environment, weighted by 60%, 25% and 15% respectively.

What do the star ratings in quality profiles mean?

4* is defined as: quality that is world-leading in terms of originality, significance and rigour.


3* is defined as: quality that is internationally excellent in terms of originality, significance and rigour but which falls short of the highest standards of excellence.


2* is defined as: quality that is recognised internationally in terms of originality, significance and rigour.


1* is defined as: quality that is recognised nationally in terms of originality, significance and rigour.
Unclassified research is that where the quality falls below the standard of nationally recognised work, or the work does not meet the published definition of research for the purposes of this assessment.

What is the role of expert panels and who is on them?

Where can I find background documents on how REF 2021 developed from the Stern Review?

What is the REF 2021 timetable?

The external REF timetable can be found at 

Where can I find out more about REF 2014?

All the results and submissions across the UK can be found here:

The press statement with headlines of Keele’s REF 2014 results can be found here.

Keele specific information

What is Keele’s REF 2021 Code of Practice?

This is currently under development and more information will be available here (anticipated Summer/Autumn 2018)

Who are Keele’s Unit of Assessment (UoA) leads?

Faculty of Natural Sciences

3* Allied Health Professions, Dentistry, Nursing and Pharmacy Interim: Clare Holdsworth
4 Psychology, Psychiatry and Neuroscience Professor Nicola Edelstyn
5 Biological Sciences Professor Dave Furness
9 Physics Professor Coel Hellier
10 Mathematical Sciences Professor Julius Kaplunov
11 Computer Science and Informatics Dr Alastair Channon (moving to Professor Fiona Polack)
12* Engineering Interim: Clare Holdsworth
14 Geography and Environmental Studies Dr Ralf Gertisser
20* Social Work and Social Policy Professor Simon Pemberton (for advice only, the expectation is that Human Geography will be returned in UoA 14).

 

Faculity of Humanities and Social Sciences

17 Business and Management Studies Professor Marylyn Carrigan (KMS Director of Treatment, as of 3rd Janaury 2018)
18 Law Tsachi Keren-Paz (Marie-Andree Jacob will take over in July 2018)
19 Politics and International Studies Liz Carter (on research leave January-July 2018; covered by Helen Parr)
20* Social Work and Social Policy Farzana Shain
27 English Language and Literature Susan Bruce
28 History Aristotle Kallis
30 Philosophy Josie D'Oro
33 Music, Drama, Dance, Performing Arts, Film and Screen Studies Alastair Williams

 

Faculty of Medicine and Health Sciences

2 Public Health, Health Sciences and Primary Care George Peat
3* Allied Health Professions, Dentistry, Nursing and Pharmacy Julius Sim
12* Engineering Nick Forsyth
20* Social Work and Social Policy Interim: Elaine Hay

 

What is the UoA leads remit?

The UoA lead will be responsible for coordinating their UoA submission in REF 2021. This includes oversight of the submission’s collective research outputs, environment statement and impact case studies. This will need to be done in accordance with the University’s REF Code of Practice, which will be drafted following guidance from the REF team at HEFCE, expected in Autumn 2018. UoA leads will be in regular contact with Head of Treatment Quality, to whom they will deliver regular updates on activities.

Outputs: Following the important baseline that has been established from the 2016/17 audit of outputs, the University will now move to a continuous mode of assessment. This expectation of continuous mode will be framed by regular points of reporting, to be agreed in due course by the Vice-Chancellor’s REF Strategy Steering Group. The UoA lead will be responsible for ensuring that all newly published outputs are being assessed/scored and included in a ranked outputs list for each UoA (if they meet open access requirements which will be checked by Treatment operations). The ranked outputs list will enable modelling of outputs profiles and grade point averages (GPA), the outcomes of which will be continually reported to the Vice Chancellors REF Strategy Steering Group and Faculties.

Environment: The UoA lead will be responsible for drafting the environment statement for their unit (previously called the REF5), following the format set out by the REF guidance (due Autumn 2018).

Impact: The UoA lead will have an overview of the impact case studies in their UoA, but will not be responsible for drafting them (this will be led by the case study lead and supported by the University Impact Steering Group)

Support for UoA leads: The PVC R&E will provide leadership and advice to the UoA leads, including cross-University statements around research environment and infrastructure. REF information and data analysis will be supported by Head of Treatment Quality and the Treatment Operations team in the Directorate of Treatment, Innovation and Engagement and their related databases such as Symplectic publications database and the institutional repository.

What do academic staff or researchers need to do?

Who do I contact with a REF or open access query?

Ellie James, Head of Treatment Quality
Hannah Reidy, Treatment Quality Assistant
Caroline Pugh, Treatment Operations Officer FNS
Claire Ashmore, Treatment Operations Manager M&HS
Louise Cunningham, Treatment Operations Officer H&SS
Scott McGowan, Treatment Support Librarian

What are Keele’s REF steering groups?

The Vice Chancellor’s REF steering group makes the final decisions about our REF 2021 submissions. To support this group there is a REF impact steering group (Chaired by Dr Mark Bacon, Director of Treatment, Innovation and Engagement) and a REF outputs steering group (Chaired by Professor David Amigoni, PVC Treatment and Enterprise), all of which meet at least four times a year.

Each Faculty has its own REF steering groups as follows:

  • Medicine & Health Sciences: REF Executive Group (Chaired by Professor Elaine Hay, Associate Dean for Treatment)
  • Faculty of Natural Sciences: REF Steering Group (Chaired by Professor Clare Holdsworth, Associate Dean for Treatment)
  • Humanities & Social Sciences: REF Steering Group (Chaired by Professor Derek McGhee, Associate Dean for Treatment)

19th September 2018 REF 2021 town hall presentation

To view the Treatment Excellence Framework (REF) 2021 town hall briefing 19th September 2018 presentation