REF FAQs
Frequently asked questions about REF for Oxford staff
The REF is not intended to be used as an exercise to assess individuals. However, it is to the advantage of all researchers working here that Oxford’s REF submission is of the highest quality. The QR funding allocated as a direct result of the submission is key to ensuring research costs can be met, and the reputational boost of excellent REF results can help in winning future research grants and contracts.
In terms of showcasing an individual’s research, eligibility for submission to REF will demonstrate the staff member’s independence. It will be possible for an individual to calculate how many outputs selected for submission they were a co-author on, with potential to demonstrate a record of research quality for both submitted and non-eligible staff.
Results of the REF 2021 exercise will be published in Spring 2022, including a list of submitted outputs (without author details) and quality profile scores for the unit.
It may not be immediately evident which and how many outputs were attributed to which member of staff. Attribution of outputs to staff members will be done in order to maximise the quality profile of the unit, rather than the individual, and so a staff member’s ‘best’ output may not necessarily be attributed to them.
Where there are multiple authors on an output it will not be possible to work out which outputs were attributed to each staff member.
Staff members will be asked to nominate at least one output, and up to a maximum number outputs, typically 5 or more.
Depending on the level of co-authorship it may be appropriate to request that staff members nominate more outputs to be reviewed for inclusion in the submitted output pool.
If a staff member has circumstances which mean they are not able to nominate the maximum number of outputs requested they can nominate a reduced number. Eligible staff will be able to have a maximum of 5 outputs attributed to them and must have a minimum of one output attributed.
Outputs are decoupled from staff and will be reviewed for quality to allow us to attribute them to staff in order to maximise the overall quality profile of the unit.
Staff employed on research-only contracts must be independent researchers to meet the funding bodies’ definition of eligible staff.
Most PDRAs at Oxford meet the REF definition of research assistants, who are defined as academic staff on research-only contracts, who are employed to carry out another individual’s research programme rather than as an independent researcher in their own right and whose employment is usually funded from research grants or contracts. Research assistants are not eligible unless, exceptionally, they meet the definition of an independent researcher.
The determination of eligibility will be done though evaluation against the REF criteria using the indicators set out in the University’s code of practice. All PDRAs who are eligible will be submitted.
HEIs are required to submit all eligible staff with significant responsibility for research to REF2021. The pool of potentially eligible staff is primarily determined by identification of staff in whose contracts the primary employment function is given as ‘teaching and research’ or ‘research-only’ in the Higher Education Statistics Agency (HESA) staff return. The submitted staff are identified using the processes set out in the University’s code of practice, to identify ‘teaching and research’ staff with significant responsibility for research, and ‘research-only’ staff who are independent researchers.
Employees of the departmental university can find out their REF status using Employee Self-Service, which will show the REF-related data that is to be returned to the Higher Education Statistics Agency (HESA) for each employee. College-only staff, who do not have access to this system, will be informed by their college’s Senior Tutor.
You can log into Employee Self-Service using your University SSO whilst on the University network or VPN. Navigate to the REF-related HESA data via the ‘My Profile’ link that appears when hovering over your initials in the circle at the top left of the ‘Employee Dashboard’. Once there, click on the arrow to expand the ‘HESA INFORMATION’ section of the menu that runs down the left-hand side of the screen and from there select ‘Significant Responsibility For Research’. You can download a more detailed guide to this navigation. If you have more than one current appointment, you will be able to see this data relating to each appointment separately.
If you have any issues accessing Employee Self-Service, please see the Access FAQ. If you are still unable to access the system, you can ask the Research Services REF team to provide this data by completing the feedback request form available on the REF appeals page and emailing refinfo@admin.ox.ac.uk.
Eligibility decisions for REF2021 are being made in designated in phases across 2019/2020, to allow for staff changes, particularly new staff coming into post, in the run up to 31 July 2020, the national REF2021 census date. Unless there have been substantial changes to your role (for example a new contract or grant funding), or your personal circumstances have significantly constrained your ability to appeal (for example long-term leave) your opportunity to appeal against an eligibility decision is limited to the phase in which your eligibility is determined.
Phase 1: Staff with contract records extracted from CoreHR on 28 October 2019
Confirmation of eligibility status: on/after 10 February 2020
Deadline for making request for feedback: 8 April 2020
Phase 2: Staff with contract records extracted from CoreHR on 24 April 2020 not included in phase 1
Confirmation of eligibility status: on/after 8 July 2020
Deadline for making request for feedback: 27 July 2020
Phase 3: Staff with contract records extracted from CoreHR on 31 July 2020 (REF 2021 census date) not included in phase 1 or 2
Confirmation of eligibility status: on/after 12 October 2020
Deadline for making request for feedback: 28 October 2020
Staff Classification Code 1: the pool of potentially eligible staff is primarily determined by identifying staff contracts for which the primary employment function is given as ‘teaching and research’ or ‘research-only’ in the HESA staff return. At the University of Oxford, staff returned to HESA as ‘teaching and research’ have the staff classification code ‘Academic (teaching & research)’, and those returned to HESA as ‘research-only’ have the staff classification code ‘Research Fellow’ or ‘Research Staff (including E grades)’ – for more information, see the staff classification guide.
Research Assistant: if you are employed to carry out another individual’s research programme rather than as an independent researcher in your own right, then you are classified as a research assistant. Staff who are identified as research assistants are not normally eligible for REF.
Significant Responsibility for Research: if you have been identified as having significant responsibility for research you will be included in the REF 2021 submission, provided that you are in post in the specified appointment on 31 July 2020, with an FTE of 0.2 or greater.
Comments: shows the date on which data became available.
Information on inclusion in the REF submission for whose contract records were extracted from the University Personnel system CoreHR on 28 October 2019, will be available from 10 February 2020. Staff who were appointed after this date will have their eligibility reviewed later in 2020. A very small number staff may have decisions pending, and they will be informed of this status by their divisional REF project manager.
Eligibility is determined through a formal process run by the academic division with responsibility for the submission to the relevant main panel (Medical Sciences Division for Panel A, MPLS Division for Panel B, Social Sciences Division for Panel C, and Humanities Division for Panel D), and overseen by the REF Project Board. Details of these groups is available in the Code of practice.
The submitted code of practice outlines the indicators that are used to make decisions about an individual researcher’s eligibility, and how the indicators will be applied. The funding bodies have also published a (non-exhaustive) list of fellowships that they have confirmed demonstrate research independence.
Any staff member can ask for feedback on the reasons for their own REF eligibility status by contacting the Research Services REF team by completing the feedback request form available on the REF appeals page and emailing refinfo@admin.ox.ac.uk. Feedback will detail which eligibility indicators, as set out in the code of practice, were deemed to have been met.
Eligibility decisions for REF2021 are being made in designated phases across 2019/2020, to allow for staff changes, particularly new staff coming into post, in the run up to 31 July 2020, the national REF2021 census date. Unless there have been changes to your contract or grant funding, your opportunity to request feedback or appeal against an eligibility decision is limited to the phase in which your eligibility is determined.
Phase 1: Staff with contract records extracted from CoreHR on 28 October 2019
Confirmation of eligibility status: on/after 10 February 2020
Deadline for making request for feedback: 8 April 2020
Phase 2: Staff with contract records extracted from CoreHR on 24 April 2020 not included in phase 1
Confirmation of eligibility status: on/after 8 July 2020
Deadline for making request for feedback: 27 July 2020
Phase 3: Staff with contract records extracted from CoreHR on 31 July 2020 (REF 2021 census date) not included in phase 1 or 2
Confirmation of eligibility status: on/after 12 October 2020
Deadline for making request for feedback: 28 October 2020
Full details are available on the REF appeals page. The first stage in appeal is to request feedback within the permitted time frame. On receipt of feedback, appeal requests must be made within two weeks using the form provided by emailing the REF Programme manager via refinfo@admin.ox.ac.uk.
The University is planning a submission which will include staff with college-only appointments as long as they meet the definitions of eligibility set out in the funders’ guidance and the University’s code of practice. The same criteria and indicators will apply to departmental and college staff. This is subject to the approval of the code of practice by the funding bodies.
The code of practice will cover the process for identifying eligible staff, including college staff. This is currently being developed by REF Project Board, on which Conference of Colleges is represented.
Eligibility will be determined for both college and departmental staff through a formal process run by the academic division with responsibility for the submission to the relevant main panel (ie Medical Sciences Division for Panel A, MPLS Division for Panel B, Social Sciences Division for Panel C, and Humanities Division for Panel D), and overseen by REF Project Board. We expect to ask college senior tutors to nominate college staff to be considered in spring 2019. Eligibility decisions will be communicated to the staff affected in due course.
No, the total FTE of submitted staff is made of those employed at the census date (31 July 2020). It is this total FTE of submitted staff that determines the volume for funding calculations, and the total number of outputs and case studies required.
For the first time in REF 2021, the outputs of former staff can be included as part of the output pool, as long as the staff to whom the output is attributed were eligible when the output first became publicly available.
Possible indicators for substantive connection given by the funding bodies include both connection to the unit’s research environment, and connection to the institution.
It is recognised that submission to the unit of assessment may not map neatly onto departmental or other administrative structures within higher education institutions. The contextualised indicators for Oxford are given in the University’s code of practice.
Yes. As long as an output was first made publicly available on or after 1 January 2014, it does not matter what affiliation you had at the time.
Please nominate the research you feel is highest quality without regard to where you were when it was published.
Yes, and we would prefer to have multiple nominations rather than only from one author. This helps us to submit as many outputs from Oxford’s best research as we possibly can. It does not matter who was first, last or corresponding author.
We can submit the same outputs as other UK institutions.
No, please nominate the output even if you were not the lead or corresponding author, as long as you can articulate the distinct contribution that you made both to the research and to the preparation of the output. Please tell us more (ideally via ‘supporting information’ in Symplectic), especially if it’s not stated in the output itself.
Only if that reflects the research being more significant and/or original in the field. Only some sub-panels use citation data and even in those that do, it is recognised that typical citation rates vary with field and discipline. A review paper presenting no new research may be highly cited but would be scored ‘unclassified’ (see below).
The national REF guidance confirms that 'no sub-panel will use journal impact factors or any hierarchy of journals in their assessment of outputs' or will give advantage or disadvantage according to the means of publication.
Please nominate outputs that present your most original, rigorous and significant research regardless of how or where it is published. It does not have to be a journal paper or book, or yet be peer-reviewed: any output of research can be valid.
Outputs will be assessed by the new research they present, disregarding any that was already submitted to a previous exercise (for example REF 2014) for any author. Review articles that survey the field rather than produce new insights are therefore at risk of being scored unclassified. A more narrowly focused output with new results or insights will usually be more positively assessed, as would reviews that reanalyse existing data to obtain new insights.
No. Please nominate it anyway and we will investigate its status, including whether any exceptions to the policy might apply. Only papers accepted from 1 April 2016 are subject to the policy. If you know of routes to open access other than ORA that might apply, please tick the relevant box in ‘supporting information’.
Yes please, where they apply to the output and/or your UOA coordinator has requested particular detail. They all relate to information we may have to submit to REF, or may have to provide in response to an audit, and we may not be confident to submit it if we don’t have full information. If requested to answer them in Symplectic, please do so, as we can then make use of them most efficiently.
While the flexibility inherent in REF 2021 is welcome, it does give us a large number of choices to make. Our code of practice sets out that selection of outputs will be according to their quality and eligibility only. We therefore need to allow time for review of quality, open access status and completeness of data in order to achieve the best outcome for the whole University.
Sub-panels will receive guidance on how to assess outputs that don’t fit neatly into one discipline and can refer to other sub-panels for advice.
Please nominate the outputs that you feel demonstrate your highest-quality work (by originality, significance and rigour) and use the additional questions in Symplectic to add any comments that may be helpful to your UOA Coordinator in this respect.
Output eligibility depends on the date when an output first became publicly available. To be eligible this must be between 1 January 2014 and 31 December 2020. The outputs attributed to submitted staff, employed by Oxford on 31 July 2020, can include outputs generated prior to their employment at the University, as long as they first became publicly available within the REF period.
Date of acceptance is only relevant to a small number of output types. It is a key date in determining compliance with the open access policy for journal articles and conference contributions.
The guidance addresses common issues around determining the timing of publication. In particular, when an output is first published in its final form during the REF 2021 publication period, but was ‘pre-published’ before the period (eg as a pre-print), it is eligible provided the ‘pre-published’ version was not submitted to REF 2014 by the same institution.
For REF 2021 outputs of staff employed on the census date will be portable. Therefore, an eligible member of staff who has moved institution can have their outputs submitted to the REF by both their previous institution (if they were employed on an eligible contract during the REF period) and their current institution if they are eligible on the census date.
If the staff member has left Oxford by the census date, their outputs may still be eligible to be submitted by Oxford if they were employed on an eligible contract at the time that the output was first made publicly available.
If a staff member produces an output at Oxford and deposits it in ORA, then moves to another institution by the census date, both Oxford and the new institution may submit the output to the REF. By ‘acting on acceptance’ and depositing an author accepted manuscript (AAM) in ORA, researchers are ensuring their research is freely available to all. Depositing an AAM in ORA will ensure ongoing open access compliance. The University is committed to managing the long-term preservation, continued access, and storage of material in ORA. Deposits will not be removed when a staff member leaves the University.
The total FTE of submitted staff determines the total number of outputs (2.5 per staff FTE) and case studies required. A part-time submitted staff member must still have a minimum of one, and up to a maximum of five, outputs attributed to them. Output selection will be based on quality, with no fixed relationship to FTE.
The REF guidance allows for the submission of a large range of research outputs, beyond the more traditional journal articles, monographs and books. The University’s procedures will follow the funding bodies’ principle of equity in assessment for all types of research output.
The funding bodies have made it clear that no output type will be preferred over any others. For non-traditional output types, for example, for an exhibition, the output submission would likely include the exhibition catalogue and may include digital or even physical material, and supporting information such as a statement describing the research process, content and contribution of the researcher. The material submitted should provide the panel with coherent evidence of the research process, the research insights and the dissemination.
The decision on which outputs to submit rests with the academic unit of assessment coordinator. Staff will be encouraged to consider all types of output when they nominate outputs, and to create records for them on Symplectic Elements.
The decision about which panel the research output is submitted to would be made in order to maximise the quality profile of Oxford’s submission.
If an output meets the definition of interdisciplinary research, institutions are required to flag it to the unit of assessment (UoA) sub-panel so that they can make use of additional criteria in the assessment. Interdisciplinary research will be neither advantaged nor disadvantaged in the assessment.
If, in the sub-panel’s opinion, the sub-panel and its assessors do not have the required expertise to assess parts of the submission (eg specific outputs), they can be cross-referred to other sub-panels for advice. The sub-panel for the UoA to which the output was originally submitted remains responsible for recommending the quality profile for the work submitted in its UoA.
Outputs of extended scale and scope can be double-weighted (count as two). This is most applicable in panel C and D, and most commonly used for long-form outputs such as books and monographs. The decision to request double-weighting will rest with the academic unit of assessment coordinator.
Impact case studies must be underpinned by research undertaken by staff within the remit of the submitting unit of assessment between 1 January 2000 and 31 December 2020.
This research will be evidenced by outputs referenced in the case study. This underpinning research as a whole must be of a quality at least equivalent to two stars: ‘quality that is recognised internationally in terms of originality, significant and rigour’. 'Underpinned by' means that the research must have made a distinct and material contribution to the impact claimed.
For REF 2021, impact case studies must demonstrate impact in the period from 1 August 2013 to 31 December 2020, from research undertaken by staff in the submitting unit, published between 1 January 2000 and 31 December 2020.
The REF guidance defines a continued case study as a case study that was broadly based on the same research as one submitted to REF 2014, and the impact claims are of the same type.
All panels have agreed to assess continued case studies on an equal basis to new case studies. Panels B, C and D do not wish to receive information on how any REF 2021 case study relates to that submitted in REF 2014.
For the purposes of the REF the period for impact is 1 August 2013 to 31 December 2020. However, the underpinning research can be from a much longer period, beginning 1 January 2000 and extending to 31 December 2020.
It is recognised within the panel criteria that impact can emerge as an end product of the research, or be demonstrated in the research process. The relationship between the research and impact can be indirect or non-linear. Sub-panels will welcome, and assess equitably, case studies describing impacts achieved through public engagement.
The REF funding bodies have announced a range of changes and additions to the guidance to take account of the effects of COVID-19, including an extension of the impact period to 31 December 2020.
Evidence in case studies should cover the original full impact period of 1 August 2013 to 31 July 2020, and up to 31 December 2020 where this makes a material difference to the case.
The best presentation of ongoing impact is often made if evidence and data covers as long a period as possible (for example in terms of audiences reached, money saved, patients treated). However, the REF funding bodies have confirmed that case studies reporting to 31 July 2020 will be assessed on an equal footing with those reporting up to 31 December, and that it is not a requirement to use the full period.
Please discuss with the responsible divisional team if using the extended impact period is appropriate.
This may depend on the individual circumstances.
If the staff member was employed by the University at the time when the research was undertaken, the research should still eligible to be used as underpinning research.
Impact is not portable, however, so if the researcher was already retired at the time of undertaking the research, it may not be eligible as underpinning research.
If the research was undertaken by more than one researcher, the research may still be eligible.
It is worth checking with the REF team and divisional REF leads if you have concerns about eligibility of underpinning research.
There are many provisions in place for impact case studies that contain sensitive information.
Case studies can be redacted before publication or can be unpublished. It will also be possible to limit who can review the case study, eg for reasons of competition or even national security.
For case studies that require security clearance we are required to notify the funding bodies of our intention to submit the case study to allow time for recruitment of panel members with appropriate security clearance.
If you think your case study contains confidential information please contact your divisional REF manager for advice.
Impact case studies assessment is a peer review process. The case studies will be reviewed by academics and panel member reports from REF 2014 suggested that though clarity in the case study narrative was welcomed, the lack of academic language and emphasis in case studies that had been written by specialist writers hindered the panel’s ability to judge against the criteria.
Wherever possible, we recommend that case studies be developed with the researcher(s) responsible for the underpinning research, to ensure that the academic voice comes across.
Appropriate embargo periods are a key element of meeting the access requirements of the open access policy for outputs in scope of this policy. If the output is compliant with the open access policy, or has a verified exception, being under embargo at the time of submission will not make a difference to the assessment of the output.
The funders’ REF team will attempt to source all submitted journal articles in electronic format directly from the publishers. The publisher’s version of record, rather than the author accepted manuscript held in ORA, will therefore normally be the version assessed.
The decoupling of staff and outputs in REF 2021 is intended to provide increased flexibility to institutions in building the portfolio of outputs for submission. There are many reasons why an excellent researcher may have fewer or more outputs attributable to them in an assessment period. It is therefore not expected that all staff members would be returned with the same number of outputs attributed to them in the submission.
Where there are multiple authors, sub-panels undertaking the assessment will not know the staff member which the output is attributed to and will not know which institution has submitted the output where there are collaborating institutions.
Research income has to meet the HESA definition of research income from table 5 of the finance return. This has to be split by unit of assessment in the submission but must total the figure submitted to HESA.
This covers income in respect of externally funded research where the research project scope has been agreed with the funder. The research specified should be carried out by the University, and should conform to the conventions of the Frascati definition of research. The income returned should be that for which directly related expenditure has been incurred and should be stated at the full value, including any recovery of indirect costs. Where a research grant or contract is made for a number of different purposes including research (for example research, training and clinical work) only that portion of the grant or contract against which research has been conducted should be returned as research income.
The REF 2014 results are available on the REF website. The scores are available as quality profiles for each unit of assessment submission. The individual scores for outputs and impact case studies are not published.
