A few weeks ago, Ofqual introduced six new principles within their "General Conditions of Recognition", the rules with which awarding organisations must comply. Two of these new principles are:
- Principle 1 – An awarding organisation must act with honesty and integrity.
- Principle 5 – An awarding organisation must act in an open, transparent and co-operative manner with Ofqual and, as appropriate, with Users of qualifications.
With these in mind, may I draw attention to two statements made by Ofqual over recent years concerning arguably their statutory duty "to secure that regulated qualifications give a reliable indication of knowledge, skills and understanding" - a duty that is of particular importance given the trust we all place in exam grades, and the widespread belief, as endorsed by the final report of the Curriculum and Assessment Review, that "exams are the fairest way of assessing students".
The first concerns a 'news item', posted on the Ofqual website on 11 August 2019, which includes the words:
"On that basis, more than one grade could well be a legitimate reflection of a student's performance and they would both be a sound estimate of that student's ability at that point in time based on the available evidence from the assessment they have undertaken."
A very similar statement was made in an Ofqual podcast, dated 8 August 2023, by Ofqual's then Chief Regulator, Dr Jo Saxton, as can be verified by clicking in at about 9:19 here.
I find the admission that "more than one grade could well be a legitimate reflection of a student's performance" troubling, raising questions such as:
- If this is the case, why is only one grade shown on a certificate?
- How many other "legitimate" grades are there?
- Are any higher than the grade shown?
- How robust are decisions, such as the requirement that a student, whose certificate shows grade 3 for GCSE English Language, must re-sit?
The second relates to evidence given to the Commons Education Select Committee hearing of 2 September 2020 by Dr Michelle Meadows, Ofqual's then Executive Director of Strategy, Risk and Research, in which she said (Q997 here):
"There is a benchmark that is used in assessment evidence that any assessment should be accurate for 90% of students plus or minus one grade. That is a standard benchmark. On average, the subjects were doing much better than that. For A-level we were looking at 98%; for GCSE we were looking at 96%, so we did take some solace from that."
My understanding of that statement is that, in general, 96% of GCSE grades, and 98% of A level grades, as awarded in England, are "accurate plus or minus one grade".
That implies that 4% of GCSE grades, and 2% of A level grades are at least two grades adrift. From which I take concern, not "solace".
Furthermore, according to Ofqual's annual statistics of reviews of marking and moderation, only about 1% of grades are changed as the result of a challenge. If some 4% of GCSE, and 2% of A level grades, as awarded, are at least two grades wrong, how is it that only 1% of awarded grades are corrected?
Most importantly, though, I find the statement that "96% of GCSE grades are accurate plus or minus one grade" highly confusing. Are 95.9% of GCSE grades, as awarded, right, and 0.1% one grade wrong? Or are 0.1% of grades right and 95.9% wrong? Both comply with "96% of grades are accurate plus or minus one grade". But each is very different, with profoundly different implications.
To me, it is important to know the percentages of grades that are "accurate, full stop", not "accurate plus or minus one grade".
I am sure the statements I have quoted are true.
But I find them singularly obscure.
Why doesn't Ofqual tell us, clearly and unambiguously, what we all need to know - measures of the reliability, and trustworthiness, of GCSE, AS and A level grades, as actually awarded, and as shown on candidates' certificates?
Especially when Ofqual are placing a requirement on awarding organisations for honesty, integrity, openness and transparency.