Oxford recently made headlines by dropping to fourth place in the Times Good University Guide. Should we worry that Oxford is losing its way? A closer look at the factors defining the ranking reveals several flawed measurements, some of which are in direct contradiction with each other. More fundamentally, by combining every possible factor into an aggregate, it obscures the fact that different people value different things in a university. Rigorous measurement of universities’ performance can be very useful in holding institutions to account and helping students choose where to apply – unfortunately, this ranking does neither.
It ought to go without saying that a ranking table is only as valuable as the criteria behind it. This is no small quibble. Just look at the variation in between The Times’ UK and international rankings: this year Oxford came first in the world, but fourth at home; the LSE was top in Britain and 52nd in the world.
Let’s examine what data goes into the mix. According to The Times, there are seven umbrella categories: teaching quality, student satisfaction, graduate job prospects, research quality (all weighted at 1.5); entry standards, percentage of students getting a 1st or 2.1, completion rate (weighted at 1); ‘people and planet’ (weighted at 0.5).
There is an issue with almost all these metrics.
The first is that teaching quality is based on student surveys: there is no universal or objective standard against which it is measured. Even if we accept that graduating students are the best judges of teaching quality – and how many people have disliked a teacher, only to realise later how valuable they were? – this measure remains unreliable. Some students might judge relative to their expectations of the university. Others may be influenced by past experience: if you are lucky enough to have gone to an excellent school, you may be nonplussed by the teaching at Oxford, whereas those less fortunate may be in awe of the same teaching. I appreciate that it is difficult to design a more objective measure of quality, but this suggests a deeper flaw in a guide purporting to rank each university in minute detail.
Student satisfaction is a questionable metric. Of the top ten universities, Oxford and Cambridge come last here. Could it be a case of the highest expectations being the hardest please? Or do those universities which demand the most of their students – presumably a positive on the other criteria – have the least satisfied students?
Entry grades are a flawed measure too. They measure how academically successful the average student was prior to arriving, rather than the quality of the work they produce and the teaching they receive at university. This measure shows where the highest performing schoolchildren choose to go, which is likely based on reputation as much as reality – hardly a useful standard for a ranking designed to dispel preconceptions about universities.
Perhaps the most glaringly flawed statistic is the proportion getting a 1st or 2.1. If I were to be facetious, I would suggest the lower the percentage the better! This is simply a reward for grade inflation; it punishes the universities with the most rigorous exams – the very ones that push students to achieve their best and are most valued by employers. Similarly, dropout rates could be determined as much by the rigour of the course as by the support given by institutions.
Finally, we move to the suspiciously named “people and planet” section, on which Oxford scores particularly poorly. It turns out to be measured by peopleandplanet.org, an organisation which, amongst many activist aims, wants to abolish all national borders. Universities are ranked on factors ranging from how much waste is recycled to the nakedly political – divesting from the ‘border industry’, ending relationships between career services and the mining sector. There may be good arguments for doing these things, but people should be aware that factors unrelated to education influence the ranking. Related to this is the ‘social inclusion’ score. It includes things like state-school admissions (tellingly excluding grammar schools), and comparing drop out rates between different groups of students. These may be important considerations. The Times does not actually say how this influences the rankings, but, if it does at all, it contradicts the entry grades score: since universities cannot influence the quality of schooling, the easiest way to increase representation of disadvantaged groups is to reduce entry requirements.
For all of the complaints I have made about the metrics, each one may well be important to an individual person: some care very deeply about recycling; some will feel that being in a good research environment will be beneficial even as undergraduates; some will be worried about dropout rates. For the simple reason that everyone has different strengths and weaknesses, aspirations and preferences, not everyone will benefit from the same thing at university. For what is the purpose of the ranking? Is it to help prospective students choose where to apply? If so, to some the most important consideration might be that Imperial graduates appear to have the best job prospects, and that St Andrews has the best teaching as rated by students – not that Imperial came sixth and St Andrews second. If it is to help employers assess applicants, the rigour of exams might be the most relevant. For academics it might be the research quality, or staff satisfaction. To combine all the factors that might influence someone’s decision-making into an aggregate score – even if they directly contradict each other – obscures as much as it illuminates.
I would argue that Oxford (and Cambridge) is attractive because of the tutorial system, because of the confluence of extraordinarily talented individuals meeting outside their degree, and because of the beautiful surroundings, both architectural and natural. To others these may not appeal, regardless of The Times’ ranking. Rather than worrying about the league table, we should focus on preserving what makes Oxford so brilliant – and, not incidentally, so attractive to applicants.
