The issue with devising an information model to enable data comparison across a diverse higher education sector is in its compromise towards common denominators. A landscape will normally be inconsistent in form and so it is with the information compiled with the aim of aiding student choice as part of the increased marketisation of Higher Education.
The Unistats site brings together the survey data used to devise the ubiquitous league tables (National Student Survey (NSS); Destination of Leavers), information at the course level (proportion of hours spent in class; assessment types), and contextual data at institution level (costs of fees and accommodation; information on bursaries and support). This collection of data is called the ‘Key Information Set’ (KIS) and was introduced by the Government as a means of providing young people applying to university with the information they need to make an informed decision.
You can see however, that the Unistats site prioritises headline figures in bold graphics over context and does not properly inform the reader of the inconsistencies behind the numbers. As with any comparison site there is a need to carefully apply criteria to the content, using explanatory text to help the reader to understand the context in which it was gathered. Some institutions have addressed this in a very direct way by adding their own explanations beside the data ‘widget’ embedded on course webpages (Link to Cambridge example course).
The data recycled from league tables, such as the NSS, is well-established, but there has no doubt been head-scratching as to how to establish parameters for new areas of data capture, such as average accommodation costs. Some examples of where interpretation has led to data inequality:
Examples of data issues
Average contact hours: here the data is presented in terms of a relevant weighting between types of activity across the entirety of the programme: ‘scheduled’ activity; ‘placement’; and ‘self-directed’ (i.e. private study). It does not say how many contact hours are provided, nor the context in which these contacts take place – e.g. small group seminars or lectures delivered to large numbers of students – which is likely to be the main point of interest for students. The misinterpretation of this data by interested parties outside the sector has already happened, as shown by the sensationalist article from the Daily Mail based on uncontextualised figures: Link to Daily Mail article [27 April 2013].
Assessment types: at the HESA training seminars on the KIS in 2013 delegates were given the task of matching assessment types against the three possible categories in the KIS (‘Practical’; ‘Coursework’ and ‘Written’). Not only were different answers given around the delegate tables, but in one instance no-one managed to recognise that a ‘performance showing’ should be categorised as a ‘Coursework’ type rather than ‘Practical’ assessment.
Private accommodation: there are clearly going to be data quality issues when institution use different accommodation portals, and the cost range used in the data set can only ever be a rough guide.
Aggregation: where there were not enough responses to a mandatory survey (such as the DLHE) at course level, the aggregated data at subject level will be shown instead.
These examples taken together undermine the purpose of the site as a comparability tool, which, of course it is, as shown by the ‘compare stats’ options.
Early indications of the use of the relaunched Unistats website point to the need for a significant marketing campaign if the site is to be utilised by applicants. The headline figure provided by HEFCE in a recent press release on usage statistics was over three million single page views (Link to HEFCE page ). A more telling measure might be the number of unique visitors at 171,186; and it was reported at the HESA-KIS training seminars that the majority of users were visiting the site during working hours: presumed to be Careers or HE staff checking the site for themselves.
This begs the question: is the site going to end-up as an information resource for HE users rather than its intended target?