TEF and quality assessment, everything but the kitchen sink

The blog is back now from an extended summer break to look into the current state of HE Quality and what the future might hold.

Last week saw the publication of the specification for year two of the Teaching Excellence Framework (TEF), the revised National Student Survey (NSS) questions and a QAA consultation on ‘Quality Review Visit’: one of HEFCE’s tenders of work for gateway entry to the sector.

TEF2, along with the HEFCE Revised Operating Model for Quality Assessment, will change the quality assurance landscape immeasurably. There will be an end to large scale institutional review, which many will instinctly welcome, however there are problems with the Government’s claim that “Quality assessment and the TEF form a coherent system” (DfE, 6) with a “move to risk-based regulation which will reduce the regulatory burden across the sector” (BIS, 18).

The Government has included in the TEF2 specification a diagram to illustrate the relationship between the TEF and ‘Quality’ (DfE, 8), with the TEF atop quality assessment, the latter now designated as a ‘baseline standard’. The detail, which is emerging in the documentation now being published, reveals a more complicated relationship between the two processes with cyclical references, overlapping data requirements and asynchronous submission deadlines. The timeline below applies to 2016/17, a pilot year, and it shows that institutions will have to switch from one process to another during the course of the year. Caveat – the detailed specifications of elements of HEFCE’s Revised Operating Model are yet to be published.

Timings for TEF & HEFCE’s QA model, 2016/17

TEF/HEFCE-QA timelines

There are clearly some challenges in meeting these requirements over the next few months. The genesis of the two systems has no doubt led to this state, as HEFCE announced their review of quality arrangements before details of the TEF were known and we’re now in a situation whereby some attempts are being made to retro-fit to two together. Quality assessment, as the baseline standard, is a pre-requisite to enter the TEF (DfE, 7); meanwhile, Annual Provider Review (APR), the principal feature of the HEFCE Revised Operating Model, will use TEF outcomes as part of the data set (HEFCE, 27). Both processes appear to draw upon similar data, such as NSS outcomes, degree outcomes, non-continuation rates and employment outcomes, all benchmarked and analysed for trends. In the case of APR any discussion about the context behind the data takes place at an annual assurance visit; whereas for the TEF institutions have to explain away in their 15-page written submission (which, as it happens, may include excerpts from the quality assessment review findings). All very intricate and potentially confusing.

The written submission for the TEF provides a unique challenge. No evidence or links to evidence may be provided; the document must stand-alone and may include the following:

  • additional context about the institution, other than the contextual data provided to assessment panels;
  • explanations for the metrics, e.g. where performance is below benchmark, including both core and split (i.e. presented by specific student groups);
  • provide evidence against the assessment criteria.

NB: one element has been omitted from the earlier TEF2 consultation and doesn’t now need addressing in the written submission, namely commendations for good practice.

Table 8 on page 38 of the TEF year 2 specification lists 27 items of possible evidence that may be included in written submissions. The specification says that the list is not prescriptive but it is hard to imagine that they will not be cross-checked (and does this really move very far away from the box-ticking criticism of QAA institutional review?). The list covers a wide range of areas and the items under ‘Learning Environment’ and ‘Student Outcomes and Learning Gain’ include some areas not previously touched in institutional review, such as ‘proportional investment in teaching and learning infrastructure’. There are also items that reflect more recent developments such as learner analytics, grade point average (GPA) and learning gain. The aim of the Government here is to influence institutional behaviour and incentivise participation in initiatives such as GPA. which previously haven’t been taken-up by the sector as a whole, by wrapping them into the TEF.

A final thought on the different approach being taken to the role of students in these processes. Despite some welcome improvements to the NSS – link to new question set – the approach to student engagement seems to have gone backwards with some conservative approaches to involving students in the TEF and quality assessment. In the TEF, “providers are encouraged to show how they have involved students in preparing the (written) submission” (DfE, 37), which may include drafting a section of the provider submission. Meanwhile APR will include ways of capturing student views and there are a few methods due to be piloted this year including web-based gathering of student views, structured meetings and annual written submissions. We’ll see where this leads.

References

Higher Education Funding Council of England (HEFCE) (2016) Revised operating model for quality assessment [online]. Available at: http://www.hefce.ac.uk/media/HEFCE,2014/Content/Pubs/2016/201603/HEFCE2016_03.pdf [Accessed 4 October 2016]

Department for Education (DfE) (2016) Teaching Excellence Framework: year two specification [online] Available at: https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/556355/TEF_Year_2_specification.pdf [Accessed 4 October 2016]

Department for Business, Innovation & Skills (BIS) (2016) Success as a Knowledge Economy: Teaching Excellence, Social Mobility and Student Choice [online] Available at: https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/523396/bis-16-265-success-as-a-knowledge-economy.pdf  [Accessed 4 October 2016]


 

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s