In the wake of the publication of Universities UK’s guide to quality and standards in UK universities, published last week, the pundits are having yet another go. Among the latest batch of comments is one in this week’s Guardian Education by Dr Terence Kealey, Vice-Chancellor of Buckingham University. And the news is: its all the QAAs fault! The Quality Assurance Agency has, of course, been the academy’s favourite whipping boy for a long time, and there are valid criticisms that, over the years, have been levelled at the Agency. But Dr Kealey, in a piece that, with all due respect, relies more on high rhetoric than evidence, doth protest too much.
A number of the criticisms in this piece reflect comments Kealey made to the Times Higher in October this year, when he criticised the QAA’s 2007 institutional audit of Buckingham for “traducing” the reputation of the University in its finding of “limited confidence” in the management of academic standards. Following Geoffrey Alderman’s highly publicised attack, in his inaugural professorial address at the university, on the impact on university standards of the “league-table culture”, it is tempting to see this all as a bit of a media counter-offensive. Nevertheless, let me focus on three of the key points made.
Dr Kealey observes:
The QAA needs to determine… that exam papers are set and marked fairly, that external examiners are empowered, that central administrators are disempowered, etc – and it should do nothing else.
Currently, the QAA acts as the general auditor for the Higher Education Funding Councils (HEFCs), and it pontificates on all of a university’s activities, ranging from the provision of careers services to staff appraisal systems.
But these are not the QAA’s proper concerns. A potential employer wants to know only one thing: is a degree from the University of X creditable? If so, how does it compare with one from the University of Y? Yet these are questions the QAA cannot answer. Let it start to address them and let it transfer its other auditing tasks to the HEFCs themselves.
In my view these interrelated observations flounder on some quite fundamental problems. First, Kealey’s view of the legitimate scope of (QAA) audit assumes a very narrow measure of ‘teaching quality’. It suggests that quality of teaching and learning can be adequately understood and evaluated whilst disregarding ways in which student learning experiences are shaped or inflenced by things that go on outside the classroom or examination hall. Does Dr Kealey really believe that, given that his institution prides itself on its outcomes in the National Student Survey and has voluntarily submitted to QAA review – or is that all a marketing ploy? I think not, especially as Kealey seems to acknowledge the need, in his vision, for the funding councils to take on an audit role of the areas not audited by a slimmed-down QAA. Hence my second concern, that Kealey’s solution could actually increase, not reduce, the burden of audit on most institutions (though presumably not Buckingham, since, as a private institution it receives no HEFCE funding, and so presumably would be exempt from funding council audit). Thirdly, while I have some (marginal) sympathies with his call in the piece for a return to actual teaching review, I also remember this process very well from being on the receiving end of the old Teaching Quality Assessment system, and from undertaking BVC monitoring visits for the Bar Council. Given the size of many faculties today, unless you engage in a lengthy and intensive review process, which is, with the best will in the world, highly stressful and time consuming for the faculty concerned, you are only going to see a snapshot of teaching. Likewise I would like to share Dr Kealey’s faith in the external examiner system, but, increasingly, I have to say that I don’t. And that’s not a criticism of the goodwill or good faith of those involved. I don’t think its a problem that can be fixed by QAA or anyone else just waving a bigger stick. This is for two reasons. (i) The current pressures on the external examiner system are deeply structural, associated with the massification of HE (ie most of us today, unlike Kealey, don’t work in institutions with only two and a bit faculties and under 1000 students), and the seeming inability of the sector to properly resource external examining in terms of time, administrative support and reward for those who take up the role. (ii) Given what we know about the difficulties of standardising and validating assessments, and the normal range of marker variation that can be expected, the real effectiveness of external examining is actually less than most of us assume. To make it even a little more effective would, I suspect, require externals to be much more embedded in the courses they examine, and probably would require course teams in many institutions to produce even more information on their assessment processes than they do now. Thirdly, I’d like to know what Dr Kealey’s mechanisms for a credible comparison of standards would be. It is, I suggest, an extremely difficult task to produce comparable, meaningful, measures across such a large and diverse sector. I also don’t see why employers’ notions of credibility should be regarded as (apparently) the only or at least primary measure. Important though they may be, employers are not the only stakeholder, but that’s another issue.
Lastly, and I appreciate this is an unfashionable thing for an academic to say, I don’t think Kealey gives the QAA, and especially its outgoing Chief Executive, Peter Williams, sufficient credit for what has changed. Light-touch audit was what the sector itself wanted. It, rightly in my view, places the onus on mature institutions to be self-monitoring and reflective about what they do. The QAA itself has been concerned that, in some institutions, an over-zealous internal audit culture has emerged, and, in my experience, I think the Agency has been working hard to get that message across. I would like to see the QAA focus moving more towards enhancement rather than audit, but it was clear from a joint QAA/HEFCE/HE Academy conference I attended in the early summer that HEFCE is reluctant to follow Scotland down the quality enhancement path.
The emerging lessons from the banking sector should, as Kealey indicates, encourage us to ask questions about how and whether light-touch regulation really works (an interesting question too, coming from a free-marketeer). We certainly should not jump to quick and easy conclusions by analogy. Regulation is a complex business; an awful lot of research (and theory) on regulation points to the difficulties involved in achieving regulatory objectives, and points to the fact that regulation nearly always has unintended consequences.
Kealey also touches on a theme raised by others: that grade inflation devalues degree currency: I’ll come back to that topic in my next post.