The QAA, quality and grade inflation


In the wake of the publication of Universities UK’s guide to quality and standards in UK universities, published last week, the pundits are having yet another go. Among the latest batch of comments is one in this week’s Guardian Education by Dr Terence Kealey, Vice-Chancellor of Buckingham University. And the news is: its all the QAAs fault! The Quality Assurance Agency has, of course, been the academy’s favourite whipping boy for a long time, and there are valid criticisms that, over the years, have been levelled at the Agency. But Dr Kealey, in a piece that, with all due respect, relies more on high rhetoric than evidence, doth protest too much.

A number of the criticisms in this piece reflect comments Kealey made to the Times Higher in October this year, when he criticised the QAA’s 2007 institutional audit of Buckingham for “traducing” the reputation of the University in its finding of “limited confidence” in the management of academic standards. Following Geoffrey Alderman’s highly publicised attack, in his inaugural professorial address at the university, on the impact on university standards of the “league-table culture”, it is tempting to see this all as a bit of a media counter-offensive. Nevertheless, let me focus on three of the key points made.

Dr Kealey observes:

The QAA needs to determine… that exam papers are set and marked fairly, that external examiners are empowered, that central administrators are disempowered, etc – and it should do nothing else.

Currently, the QAA acts as the general auditor for the Higher Education Funding Councils (HEFCs), and it pontificates on all of a university’s activities, ranging from the provision of careers services to staff appraisal systems.

But these are not the QAA’s proper concerns. A potential employer wants to know only one thing: is a degree from the University of X creditable? If so, how does it compare with one from the University of Y? Yet these are questions the QAA cannot answer. Let it start to address them and let it transfer its other auditing tasks to the HEFCs themselves.

In my view these interrelated observations flounder on some quite fundamental problems. First, Kealey’s view of the legitimate scope of (QAA) audit assumes a very narrow measure of ‘teaching quality’. It suggests that quality of teaching and learning can be adequately understood and evaluated whilst disregarding ways in which student learning experiences are shaped or inflenced by things that go on outside the classroom or examination hall.  Does Dr Kealey really believe that, given that his institution prides itself on its outcomes in the National Student Survey and has voluntarily submitted to QAA review – or is that all a marketing ploy? I think not, especially as Kealey seems to acknowledge the need, in his vision, for the funding councils to take on an audit role of the areas not audited by a slimmed-down QAA. Hence my second concern, that Kealey’s solution could actually increase, not reduce, the burden of audit on most institutions (though presumably not Buckingham, since, as a private institution it receives no HEFCE funding, and so presumably would be exempt from funding council audit). Thirdly, while I have some (marginal) sympathies with his call in the piece for a return to actual teaching review, I also remember this process very well from being on the receiving end of the old Teaching Quality Assessment system, and from undertaking BVC monitoring visits for the Bar Council. Given the size of many faculties today, unless you engage in a lengthy and intensive review process, which is, with the best will in the world, highly stressful and time consuming for the faculty concerned, you are only going to see a snapshot of teaching. Likewise I would like to share Dr Kealey’s faith in the external examiner system, but, increasingly, I have to say that  I don’t. And that’s not a criticism of the goodwill or good faith of those involved. I don’t think its a problem that can be fixed by QAA or anyone else just waving a bigger stick. This is for two reasons. (i) The current pressures on the external examiner system are deeply structural, associated with the massification of HE (ie most of us today, unlike Kealey, don’t work in institutions with only two and a bit faculties and under 1000 students), and the seeming inability of the sector to properly resource external examining in terms of time, administrative support and reward for those who take up the role. (ii) Given what we know about the difficulties of standardising and validating assessments, and the normal range of marker variation that can be expected, the real effectiveness of external examining is actually less than most of us assume. To make it even a little more effective would, I suspect, require externals to be much more embedded in the courses they examine, and probably would require course teams in many institutions to produce even more information on their assessment processes than they do now. Thirdly, I’d like to know what Dr Kealey’s mechanisms for a credible comparison of standards would be. It is, I suggest, an extremely difficult task to produce comparable, meaningful, measures across such a large and diverse sector. I also don’t see why employers’ notions of credibility should be regarded as (apparently) the only or at least primary measure. Important though they may be, employers are not the only stakeholder, but that’s another issue.

Lastly, and I appreciate this is an unfashionable thing for an academic to say, I don’t think Kealey gives the QAA, and especially its outgoing Chief Executive, Peter Williams, sufficient credit for what has changed. Light-touch audit was what the sector itself wanted. It, rightly in my view, places the onus on mature institutions to be self-monitoring and reflective about what they do. The QAA itself has been concerned that, in some institutions, an over-zealous internal audit culture has emerged, and, in my experience, I think the Agency has been working hard to get that message across. I would like to see the QAA focus moving more towards enhancement rather than audit, but it was clear from a joint QAA/HEFCE/HE Academy conference I attended in the early summer that HEFCE is reluctant to follow Scotland down the quality enhancement path.

The emerging lessons from the banking sector should, as Kealey indicates, encourage us to ask questions about how and whether light-touch regulation really works (an interesting question too, coming from a free-marketeer).   We certainly should not jump to quick and easy conclusions by analogy. Regulation is a complex business; an awful lot of research (and theory) on regulation points to the difficulties involved in achieving regulatory objectives, and points to the fact that regulation nearly always has unintended consequences.

Kealey also touches on a theme raised by others: that grade inflation devalues degree currency: I’ll come back to that topic in my next post.

Advertisements

2 responses to “The QAA, quality and grade inflation

  1. Dear Professor Webb

    May I assure you that my Buckingham inaugural lecture last June was not part of any “media counter-offensive” against the QAA? At the time I wrote the lecture I was unaware of the QAA verdict on Buckingham. That said, the verdict – and the report on which it was based – fully support my view that the QAA suffers from an obsessive-compulsive corporatism. By this I mean that it recognises only one form of highly centralised academic organisation, the overriding characteristic of which is the managerial hierarchy. This model rejects, instinctively, the notion of an academic “communitas.” When we add to this the QAA’s insistence on the standardisation of procedures [the fatally misguided belief that academic standards are best assured through the standardisation of procedures], we should not be surpised at its Buckingham verdict, which (believe me!) tells us much more about the QAA than about the University of Buckingham.

    Sincerely

    Professor Geoffrey Alderman

  2. Dear Professor Alderman

    Thank you very much for your clarification of that point. I may have made my point a little clumsily; it was simply that, to the outside observer, the fact that both you and Dr Kealey have been vocal in your criticisms of the QAA in the intervening period since its report on Buckingham was completed in November 2007 might, now, start to look like a “media offensive”. I certainly wasn’t intimating that I had evidence of such!

    I do share some of your concerns about the rise of what you call “corporatism”. Issues of “communitas” are important, undoubtedly, but cannot, I suggest, be considered independently of the steady decline in trust in traditional, collegial, institutions, such as the universities and the professions, and their rapidly changing role in the social and cultural fabric of the nation-state. That decline has, as writers like Giddens show, relatively deep roots in later modernity – roots that certainly pre-date the inception of the QAA! Indeed bodies like the QAA are surely more a product than a producer of such trends. My underlying concern is that, by focussing so much of our energy on criticising the products of the process, we perhaps risk ignoring the more fundamental matter of understanding the root causes of the loss of faith in universities as self-governing entities, and thence of identifying alternative ways forward – ones which hopefully cannot so readily be dismissed by our critics as rhetorical nostalgia or naked self-interest.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s