When newspapers compile league tables of universities, are they performing a public service to students and their parents, or are they damaging the reputation and fabric of higher education? Quite possibly both, according to speakers at a Universities UK conference in London. Francis Beckett reports.

League tables provide fun, humiliation, information, and benchmarking – but their current presentation and compilation are flawed, said University Vice-Chancellor Sir Alan Wilson (left), introducing the conference with a ‘constructive critique.’

While a university’s standards can be maintained from year to year, its place in the league tables can shift wildly. Even in the same year, it can be much higher in one table than in another. One of the problems, Sir Alan said, is that by attempting to rank institutions by a combination of many different factors, from library expenditure to numbers of firsts, the final figures have little real significance, like trying to ‘combine apples and oranges.’

He saw three faults with the criteria used by the main British newspaper league tables. First, ‘most league table generators divide by size.’ This penalises large institutions like Leeds. “You could take two small universities in the top 10 and find that they could both be included, grade for grade, within the University of Leeds” – which neatly finds itself ranked about 20 in league tables.

Second, they combine all sorts of different indicators. “Inevitably you are weighting them, explicitly or implicitly,” he said. This makes league tables volatile and unpredictable. “Some universities are ranked anything from 10 to 40.” This, contrary to appearances, did not necessarily indicate wildly fluctuating standards at these institutions.

The third problem with league tables was their effect. The desire to do well in them, said Sir Alan, ‘generates incentives for perverse behaviour.’ For example, the government may want to widen participation, and a university like Leeds aims to do just that – but if it succeeds, it could easily be lowering its place in the league tables. Or a university may have an overstocked, inefficient and expensive library. This will be rewarded by league tables, which judge only the spending on the library.

So what should be done? Two main league table compilers – Donald MacLeod of the Guardian, and Times Higher Education Supplement editor John O’Leary – were there to defend their work. Both insisted that league tables were likely to stay, and were intended as an aide to students and their parents. They are not published for the benefit of universities. MacLeod put it brutally: “I know who the audience is, and it isn’t you. It’s students and parents.”

Neither was concerned that their work generated incentives for perverse behaviour. O’Leary said he was not concerned about the widening participation agenda, or any other policy that happened from time to time to be adopted by government, but about what parents and students wanted to know. “We want to put together indicators of excellence.” MacLeod said: “If the library is overstocked, that is fine from the student’s point of view.”

By the book – efficient libraries like the Brotherton (below) aren't always rewarded by league tables, which focus only on expenditure

They have responded to criticism of their methodology and criteria, they said. The Times no longer includes the amount of student accommodation as a criterion. It has two weighted criteria – teaching assessment (weighted 2.5) and research assessment (weighted 1.5.) Other criteria include entry qualifications, which is controversial – what does it show about a university that it demands high A-level scores from potential students?

But in any case, are the tables much use to students and parents? Comprehensive school head teacher Kate Griffin, president of the Secondary Heads Association, said: “League tables do not help us much. We need to know the quality of teaching, the quality of support, and the drop-out rate. We need it not just institution by institution, but course by course.” They do not give the information that schools need. The only thing she could say in their defence was that they are not as damaging as league tables for schools, which ‘have had devastating effects.’

If they don’t help schools, are league tables much use to employers? Not a great deal, according to Carl Gilleard, chief executive of the Association of Graduate Recruiters. Three quarters of employers target specific institutions to recruit from, but this percentage is reducing each year. And when employers do target specific places, they rarely choose these places by referring to league tables.

However, dislike them, ignore them, resent them, there is no sign that the four newspaper league tables of British universities – produced by the Times, Guardian, Sunday Times and Financial Times – are about to disappear, because they help sell newspapers. People buy the newspaper for the league table, and the marketing people calculate that if they like the rest of the paper, they will go on buying it. It’s a straight commercial calculation.

Say it right – sixth-formers need course-specific information, available at events like the Festival of Languages (right)

And no one was arguing that it would be a good thing if they did disappear. Sir Alan himself told the conference: “I’m not against league tables. They will always be with us. But we have to stop pretending that they are, in their current form, objective or scientific.”

What’s needed are league tables run on better-thought out criteria, ideally, using the methods of cost-benefit analysis to put a value on different elements of higher education. “It is unlikely that we could define an ideal league table, but we could achieve several league tables accurately representing different attributes,” he added.

For the future, newspapers are now facing the challenge of compiling league tables without teaching assessments. Tables aimed at students and parents can do without research assessments, but take away teaching assessments and their use becomes limited. The Guardian says it will continue to include the scores – however out of date – but reduce their ‘weight’ and phase them out.

One alternative, floated by Sir Alan, is to use peer reviews, asking academics for a view on the teaching in other departments, but there’s often a good deal of resistance from academics to the idea of passing judgement on teaching and learning activities carried out by colleagues in other institutions – although they would feel more comfortable with research – especially for a league table exercise which many academics consider suspect anyway.