LETR five years on

Thanks to Jess Guth and the ALT, the LETR research team came together in Leeds (UK) on 25 June to revisit its work five years after publication of the 2013 Report.

It was an excellent and thought-provoking conference, ad interesting to see the ways in which developments in the last five years can be benchmarked against the report (or not). Paul Maharg has blogged a number of the conference sessions on his website, permalinks as follows:

Parallel session 1

Professions (plenary) panel

Parallel session 2

As part of the process of reflection we thought it would be interesting, and perhaps useful, to collate the work we have done since 2012 that directly addresses LETR, or builds on it in some way. The attached LETR research team outputs 2012-18 captures outputs primarily by Ching, Maharg and Webb. The list, I think, is interesting in showing the geographical reach of LETR, particularly in terms of regulatory engagement, which is probably a more meaningful/reliable measure of impact than the mere locations of presentations and publications. It also hopefully captures ways in which LETR clearly continues to inform the individual authors’ work and its direction. It  will hopefully be updated to include additional outputs from Avrom Sherr.

This post was updated on 26/06/18 to include the conference permalinks and to embed rather than reproduce the list of outputs.



Some thoughts on the proposed SQE and its implications for the English law degree

I highly recommend Richard Moorhead’s blog on the recently published consultation by the Solicitors’ Regulation Authority on the  standardised competence assessment for all those seeking qualification as a solicitor in England and Wales. In this post I just want to add a few thoughts of my own to the conversation. I’m also conscious that the SRA’s work is of interest to a wider audience than the UK. Here in Australia, for example, the Law Admissions Consultative Committee is continuing to follow post-LETR developments in England closely. The Hong Kong Law Society has also proposed the introduction of a Common Entrance Exam, though the scope of the HK proposal is less radical, and debate has, for now, been paused and rolled into a wider review of legal education and training that is now underway.* In this post I intend to reflect only the possible implications of the SRA proposal for the English qualifying law degree (QLD) – not least because I want avoid anything that might appear to pre-judge the Hong Kong debate.

The SRA consultation paper starts from three possible, broad, models for a potential qualification regime (para 11):

  • Option 1: Continuing to prescribe a limited number of pathways to qualification, the details of which we specify, which are aligned to the Statement of Solicitor Competence Statement, Statement of Legal Knowledge, and Threshold Standard

  • Option 2: Rather than prescribing a limited number of pathways, authorising any training pathway developed by a training provider which enables a candidate to demonstrate they can perform the activities set out in the Statement of Solicitor Competence to the standard required in the Threshold Standard.

  • Option 3: Developing a centralised assessment of competence that all candidates are required to undertake prior to qualification, again aligned to the Statement of Solicitor Competence, Statement of Legal Knowledge and Threshold Standard.

Option 1 represents largely a continuation of the status quo; option 2, I suggest, comes closer to the base position advanced in the 2013 LETR Report, while Option 3 is the SRA’s preferred approach for reasons essentially of cost, consistency and flexibility/diversity. The paper offers some compelling arguments for its preferences, and some balanced evaluation of the options, though I felt overall that it was perhaps a little more robust in critiquing options 1 and 2 than it’s preferred option 3 – though it does also, rightly, point out that none of the options are mutually exclusive (and indeed the LETR Report could be seen as recognising the value of elements of both options 2 and 3 – the latter notably in the use of standardised clients in skills assessments).

Much of the detail (and hence the devil) of the preferred approach remains to be developed. The paper is clear that the SRA anticipates a separate two-part assessment of knowledge and skills. The modularised assessment of knowledge must be completed first, and would be assessessed via computer-based objective testing. The second part would involve standardised practical exercises akin to the standardised clinical  assessments used by medical schools and in the current Qualified Lawyer Transfer Scheme. The paper is relatively open on the question whether, and if so how much, work experience should be required before the Part 2 assessments are completed.

The proposed scope of the knowledge assessments is broad. It will encompass ethics and professional conduct, wills and probate, taxation, business law and practice, property law, torts, criminal law and evidence, criminal litigation, civil litigation, contract law, trusts and equitable wrongs, constitutional law, EU law, human rights, and the English legal system (para. 41). The assessments are to be modularised, to facilitate “integration with other education and training programmes.” There is, however, no discussion at this stage of the broader assessment framework, prerequistes and sequencing of modules, or of the frequency of assessments.

Implications for the QLD

The paper is very clear that possession of a law degree should not be a basis for any exemptions from the knowledge (or skills) requirements of the SQE. In short, if this proposal goes through, the QLD as we currently think of it is dead – at least for the solicitors’ profession.

I agree with Moorhead that, if these proposals go through, we are likely to see the creation of a more divergent education and training playing field: possibly with growing differences between traditional liberal and what Moorhead calls ‘almost practice ready’ law degrees, plus a greater variety of postgraduate and, I would add, possibly non-graduate (in structure, though potentially graduate in level) options such as apprenticeships.

The consequences of all this for the law school sector as a whole are potentially substantial, and may be very serious. They include:

Recruitment: Law has undoubtedly grown as an academic subject on the back of its professional status and recognition. Even though less than 50% (and I suspect in some post-92 law schools the proportion could be 30% or even lower) actually make it to being a solicitor, between 60%-70% appear to enter law school with solicitor/barrister ambitions. Recruitment implications of the loss of QLD status are thus both potentially significant and very hard to judge. Will regulation reshape the market for training, or will the market for training adapt as minimally as possible to the regulation? There are significant vested interests involved, including the elite law schools, and the big LPC providers who will not let a multi-million pound business simply disappear overnight. Much may depend on how the elite law firms and their ‘preferred suppliers’, the elite law schools respond to these changes, and whether they (continue to) function as a congeries of reputational interests. (The reputational risks of marginalising the law degree are not addressed in the SRA paper. In the LETR research phase, by contrast,  quite a lot was made of the reputational importance of the graduate standing of the profession in the international marketplace. Whether ‘graduateness’ without the Oxbridge or Russell group badge carries the same cachet is moot). If some form of LLB + LPC remains a significant pathway, the recruitment effects may be mitigated – at least for some parts of the sector. If it doesn’t, the future becomes infinitely more difficult to predict.

Expansion of the knowledge-base – The new knowledge base essentially represents an amalgam of the knowledge requirements of the QLD and LPC. It thus reflects the continuing influence of the reserved areas of practice and thereby excludes much of what many (especially commercial) solicitors do. For those who want to develop nearly-practice ready degrees, it might not change the game that much from the current exempting degree model. However, that assessment also depends in large part on how much current flexibility over (QLD) content is reduced by the so-far unwritten assessment framework. To expect the  academic community to vote on the options in the absence of this seems rather like asking turkeys to vote for something that may or may not be Christmas; you really would like to know first.

Doubling-up of assessment burden: those who continue to do degrees are likely to be confronted with a growing assessment burden. Under the preferred model university assessments will not count for the SQE, and it would be a radical change of policy for universities to accept entirely external assessments as part of a (concurrent) degree. Moreover, the fact that the SRA currently sees SQE assessments as both pass/fail and sitting outside the Framework for Higher Education Qualifications (FHEQ) (see paras 57-58) makes any inclusion by recognition a less, not more likely, prospect. The impact, including diversity impact, of the scale, scope, timing, frequency and cost of SQE assessments on law students specifically appears not to have been addressed at this stage.

Implications for innovation and diversity of intellectual approaches: a drift towards nearly practice ready degrees may have significant ‘unintended’ (or from the SRA’s point of view, ‘none of our concern’) consequences for academic law. Joint degrees may decline because they simply cannot address enough of the SQE ‘core’ knowledge. Unless a clear secondary market develops in SQE test preparation (a matter which in itself may have some diversity implications), universities are also likely to find themselves under some pressure to teach to the test. That is likely to (further) undermine socio-legal, theoretical or other alternative intellectual approaches to doctrinal legal analysis. (Recall that the LETR Report data highlighted the limited value professionals attached to jurisprudential or socio-legal content/ approaches.)

The anxiety has begun…

*The Hong Kong Review under the auspices of the Standing Committee on Legal Education and Training is being undertaken by a panel comprising Justice KH Woo, Professor ATH (Tony) Smith and myself.


Creativity and assessment

With the demise of UKCLE, I have got drawn into a bit more teaching this term than in recent years – even with the buyout for LETR. And I have to say I’m really enjoying it, I’ve got a nice mix of undergraduate legal theory, postgrad teaching on a module we call Foundations of Socio-Legal Theory and Research, and a joint Law and Business module – Critical Issues in Law and Management (CILM) – that my colleague Grier Palmer (Warwick Business School) has developed. CILM is one of those great modules where content in a way comes second to process. It is primarily a vehicle for developing creativity and critical thinking, and so we use a lot of student-centred activity and presentation work, a bit of open-space learning, and assess through an essay, book review, and a set of reflective logs. Its quite experimental for both the students and those of us teachng on it!

One of our experiments this year, which is linked to a larger project on case-based learning that Grier and I are involved in for Warwick’s Institute for Advanced Teaching and Learning (with colleagues from education, health, and the Medical School), has been to create a case study around the notorious Bhopal gas tragedy. Twenty-seven years after the original explosion at the Union Carbide plant in Bhopal, this is still having a massive impact on the lives of its victims  As a case study it is a massively rich and powerful teaching resource. There are lots of primary and secondary materials available, and it provides a good vehcle for exploring a range of related legal, business and human rights issues.

We decided to run the Bhopal case as a student-led activity – we have a group of 10 CILM students who take the course as a 30 credit module, rather than the usual 24. So, their ‘6 CATS project’ this year was to lead the rest of the class in preparing to stage and then staging a Peoples’ Tribunal hearing on the continuing impact of the Bhopal tragedy. This work was spread over four weeks and integrated into the class time, starting with a briefing session for the 30 CATS students  in week 1, followed by a ‘book review’ session for the whole class – discussing and critically analysing a range of academic literature on the Bhopal tragedy. That in turn was followed in week 3 by a student-led session in which the 30 CATS team briefed and worked with their 24 credit peers on planning the tribunal. The tribunal itself was then held in week 4, over a two hour teaching slot.

We haven’t yet seen the 30 CATS students reflective pieces on this activity, but my initial sense is that it was reasonably successful, and something I would like to further develop. One of the great things about it was that, with a cohort of around 50 students on the module, we had two iterations of both the planning session and the Tribunal itself. This really gave the 30 CATS students an opportunity to reflect on what worked well and what didn’t, and to take action almost immediately to implement change. This worked really well in the planning session, so that the second was noticeably more structured, better managed and more dynamic – a great example of how students can really very quickly learn from experience. Interestingly, though the 30 CATS students are assessed on the whole process, not just the Tribunal, a lot of the assessment anxiety clearly homed in around the tribunal performance itself. One manifestation of this was that, though there was some really great content, the event was closely scripted and lacked a bit of the dynamism and creativity we were hoping to see. This isn’t meant as a criticism of the students; they could see it for themselves, and it one point in the second iteration, they ‘let go’ of the script and engaged in a bit of improvisation, and in that moment brought a wholly different energy to the activity .

For me it raises some interesting challenges. The assessment context clearly had a chilling effect and encouraged the students who were being assessed to play it a bit safe, even though that involved an element of discrepant reasoning, since they also knew that creativity was something they would get credit for. As a teacher, I want to maximise the opportunities for my students to think and act ‘outside of the box’, and to get credit for it; I don’t want to marginalise creativity by treating it as unassessable. I I therefore want to create a  space for enabling and encouraging risk-taking, and now I’m wondering whether that needs to be constructed as somewhere safe, or maybe it does need to be slightly edgy? I also want to get inside and disrupt that kind of intuitive cost-benefit thinking which drives them to play safe, and, I suspect, is ingrained by years of traditional assessment practices. I can already see that we may have left a gap between creativity and risk-taking, both conceptually and in terms of getting the message across about what we were looking for from this asssessment. So, a bit of work to be done!


New College of the Humanities – caveat emptor?

News that philosopher A.C. Grayling is launching a new private university for “gifted” students in London is, on present evidence, adding to the flames of the privatisation debate in England. The essence of the furore lies in the fact that Grayling is proposing to charge £18,000 a year to students to study for University of London degrees that they could obtain for considerably less elsewhere. The lure: small class sizes, an emphasis on a “responsive” learning environment, and a panoply of academic star professors, including Stephen Pinker, Sir David Cannadine, Richard Dawkins, and, in law, Ronald Dworkin, and Adrian Zuckerman. In addition to their degree subjects, students will also take three “intellectual skills” modules in science literacy, logic and critical thinking, and applied ethics. For this they will receive a Diploma of New College in addition to their BA or LLB – whether this promise of an extra workload for an additional award will be an incentive, or quite the reverse remains to be seen!

Critics have tended to focus on two issues so far. First, just how much teaching these luminaries will do is, of course, a moot point, and Dawkins’ observation in the Guardian that “Professor Grayling invited me to join the professoriate and give some lectures” does seem to suggest that he may not be rolling up to offer weekly tutorials in traditional Oxbridge fashion. I’m not exactly expecting Dworkin to be brushing up first years contract law either. But a second charge being levelled at New College, that it has been guilty of plagiarism in “ripping off” London University International Programme syllabi, does seem misconceived. The International Programme is of course the UoL’s old External Programme with a shiny new name. It has been around a long time, and has acquired some pretty impressive alumni over that time. These UoL courses are taught by colleges all around the world, and none of them have a formalised link with the University of London as such (though I know from my own experience as an external examiner on the External Law Programme – as it then was – that the University has in recent years put a lot of effort into outreach and developing support for the colleges offering its awards). Some of these external colleges are very good at what they do… and others are not. And that is a concern, it can be a bit of a lottery.

There is no formal quality assessment by the University of London of the teaching or learning resources provided by the external colleges, nor unless New College opens its doors to QAA, will it have to submit to the quality assessment regime expected of UK public universities. Some might say that’s no bad thing, but it begs the question as to what New College itself will do to assure prospective students that it will provide the elite education promised.
There is one remaining external check on standards: degree papers will be externally assessed. That separation between teaching and assessment may be good news for the professoriate, who are thereby exempted from the annoyance of the annual marking ritual, but it may be less good news for the students of New College. It can make it a very demanding way to study for a degree, and, certainly as regards the LLB, graduation rates and the proportion of good honours degrees, both tend to be lower than on the UoL’s internal programmes. This reflects a range of factors – student entry qualifications (the International Programme minimum standard is significantly lower than the grades needed to get into an internal programme), often a relative absence of formative assessment and preparation for university learning, variable access to learning resources, and variable teaching quality. An external degree requires teachers with a broad understanding of their subject, who are effective at teaching to a syllabus that is not of their own design. The separation of teaching and assessment can also encourage teachers and students to adopt a risk-averse, assessment-driven approach that can emphasise coverage over deep learning. Educationally, none of these are insurmountable, but I’m not sure its where I would want to start in developing a system of elite education. And if nothing else serves to damn the project, Boris Johnson’s endorsement in today’s Torygraph that New College “is a simply brilliant idea” for taking-on “the cream of the [Oxbridge] rejects” mightjust do the trick.

Exam grading at law school

At this time of year I think we should do our utmost to reassure students of the rigour of university assessment procedures. This short film from those nice people at LSU in the good old U.S. of A. explains all. If you are about to sit your exams, hEaD Space wishes you the very best of luck!

Degree classification and grade ‘inflation’

This summer has seen an awful lot of ink expended on issues of degree quality and ‘grade inflation’. I don’t particularly want to wade in on one side or the other, largely because the issue is incredibly complex, in a way that many of the ‘commentators’ in the media (and in Parliament) over the summer either could not grasp or chose to ignore. Let me just make a few observations:

I instinctively dislike the term grade inflation. It carries a lot of (deliberately disparaging) baggage. Inflation is not a neutral term. Inflation implies a reduction in value. By even engaging in the debate in the language of inflation we are therefore, from the outset, assuming something that needs to be proved; not just that we are awarding more Firsts and Upper Seconds, but that this is wrong and devalues the status of our degrees. This in turn also assumes that our assessment practices twenty or thirty years ago were superior to those in place today, and that assumption should certainly be challenged.

And this is where we have to come back to the problem of complexity. Whether we like it or not, we are not comparing like with like. When Norman Baird at qed law produces a table showing that most law schools are awarding substantially more First and 2:i’s in 2007 than in 1997, then we can certainly acknowledge that is interesting (though why 16 law schools were apparently awarding less in 2007 is more counter-intuitive and possibly even more interesting), but as Norman himself acknowledges, it is just a snapshot, based on some very simple percentages. It tells us nothing about which of those changes (if any) are statistically significant; it does not indicate whether either of those years chosen was atypical for that law school, so we have no sense of the outliers, and, obviously, it tells us nothing about the underlying practices that might account for that change. And there are a lot of candidates to explain the changes, mostly well known, including:

  • changing teaching and assessment methods;
  • possible changes to student motivation and prior learning;
  • availability of more student support mechanisms in HE;
  • possibly greater individual and institutional pressures on teachers to give higher marks (including the growing recognition that marks may need to be defensible if challenged)
  • maybe even the recognition in law that the practice of marking across a narrow band of marks, relative to other disciplines, is not defensible when you are dealing with what is usually one of the most able cohorts of students in the university.

This is where it starts to get a bit more contentious. We know we are no longer, as a system, dealing purely with the top 5% by educational attainment. And rightly so. But this has consequences; more work has to be done by departments and institutions to support those whose prior learning experience has not necessarily prepared them well for traditional higher education. Within the pressures of a crowded curriculum, that is difficult. This seems to me to expose the biggest elephant in the room. It is surely not sensible to graft the expectations of an old elite system, in which a relatively small number of students who, frankly, had the cultural and intellectual capital to succeed almost regardless of the education they received, onto a modern mass system.

In short, I don’t think we are facing a crisis of degree standards, but I think we do have some artificial expectations of what a mass system of HE can and should achieve, and a certain amount of political dishonesty about the consequences of massification; and universities have, to some extent, been complicit with government in that process.

If we really wanted the ‘standards’ of an elite system in a mass environment, we would need to resource the environment much better, and more logically (most degrees still operate an illogical system of resource allocation, whereby, as much by default as by design, a larger pro rata share of resources are spent on the later years, rather than on the critical first year of the programme). It would also have to become politically acceptable to have higher initial failure rates. Neither of these things are likely to happen. And I don’t actually have a lot of problems with that. But we surely do have to accept that, in a mass system, the purpose, scope, and value of an undergraduate degree is different from what it once was. That is not to say necessarily better or worse, just different. To some degree, the market is already doing this for us, though, of course, fairly arbitrarily, and without much sensible public debate.

As it is we have at best a hybrid (some might just say confused) system, and the teachers – the poor bloody infantry in all this – have to work out how to fix a machine that, ultimately, is constitutionally broken. What are the fixes? Here are a couple of starters for ten for this particular university challenge:

  • The classification system as both the Burgess Report and the QAA have pointed is well beyond its sell-by date. It should be replaced, though personally I would favour a more radical solution than Burgess’s transcript; possibly a grade point system.
  • Perhaps we should also consider an institutionalised move to marking on the ‘bell curve’ for larger modules. If this is done transparently across the sector (a job for the Committee of Heads of University law Schools perhaps?), it could act as a potential brake on creeping grade inflation, as any move of the median grade would have to be at least reported, possibly agreed, within the sector. Moreover, ‘marking to the curve’ seems a fairer way of accounting for marker variations (including teacher experience and expectations; year-on-year variations between papers, etc) than our current supposedly ‘objective’ system.