What’s happened with the A-level results?

This year’s A-level results have been the most controversial ever, by a long way. But what exactly has happened? And what can we do about it?

How were the grades calculated?

When the Secretary of State announced on 18th March that schools would close, he also announced that exams were cancelled, but that “we will work with the sector and Ofqual [the exams regulator] to ensure that children get the qualifications that they need.” Detailed guidance followed.

Teachers were asked to provide a “centre assessed grade.” In the Ofqual guidance it says: “we asked schools and colleges to use their professional experience to make a fair and objective judgement of the grade they believed a student would have achieved had they sat their exams this year.” These grades were then moderated by the exam boards, using an algorithm designed by Ofqual, to ensure that grades in 2020 were similar (or “comparable”) to previous years.

Why were teacher recommendations so high?

Some parts of the media have accused teachers of assessing too generously, or trying to unfairly boost their own schools’ results. All of this is wrong. Firstly, no data on schools’ overall results is being collected or published this year. There are no performance tables – a welcome move, which has allowed teachers to focus on what really matters: the students and their results.

But, if teachers’ recommended grades had been accepted without moderation, nationally results would have risen: there would have been a 13% rise in A-levels awarded grade A*-B, which is an “implausibly high” increase. Why has this happened?

Put simply, teachers were asked to assess what they believed students to be capable of. Real exams assess how students actually perform on the day. If a teacher believed a student was capable of achieving an A in the summer, then they assessed that student at an A. If that student had sat the real exam, they may have achieved that A. But, if there was a particularly tricky question, or they managed their time badly, or they had a mental blank in the exam, they might not have done. They might have ended up with a B. So the teacher recommended grades were always going to be higher – that was baked into the system, and it is why some form of moderation was needed.

So how did the algorithm work?

The standardisation and moderation process is explained in Ofqual’s interim technical report, published on A-level results day. The report is 319 pages long, which gives you some idea of how complex the process is. It is called the Direct Centre Performance model (DCP). In Ofqual’s own words, it “works by predicting the distribution of grades for each individual school or college. That prediction is based on the historical performance of the school or college in that subject taking into account any changes in the prior attainment of candidates entering this year compared to previous years.”

What does this mean? If we take A-level Maths as an example, the exam board would look at what distribution of grades students from Churchill Academy & Sixth Form had achieved in A-level Maths over recent years. It adjusts that distribution based on the prior attainment (GCSE and other results) of the students taking A-level Maths at Churchill in 2020, and then makes a prediction of what grades it expects to see from Churchill based on that information. The algorithm then adjusts the teacher recommended grades from Churchill to fit the “expected” or predicted distribution of grades.

This is where one of the major problems has arisen. Whilst the algorithm is actually very sensible at a whole cohort level, it forgets that individual candidates are human beings and don’t necessarily fit the statistical prediction. They can surprise us – and, as a teacher, I know that they do, every single day. The algorithm doesn’t account for which students are really revising hard, which students have really pushed themselves, which students have suddenly found a new passion and understanding for a subject…it cannot possibly do this. So, instead, it irons out the students into the distribution that the algorithm suggests, almost completely ignoring the teacher recommended grades. The consequences are explained really well by Alex Weatherall in this thread on Twitter.

It also means that schools which have historically performed well at A-level are at an advantage over those which have not. So students that were recommended A* can end up with a C. And, even more cruelly, students that were recommended to pass an A-level can end up with a U grade – failing an exam they hadn’t even sat. Unfairness and injustice is baked into the system.

What about small groups?

An additional unfairness in the system is that statistical models can’t be applied fairly to small groups. In Ofqual’s own words:

“Where schools and colleges had a relatively small cohort for a subject – fewer than 15 students when looking across the current entry and the historical data – the standardisation model put more weight on the CAGs…there is no statistical model that can reliably predict grades for particularly small groups of students. We have therefore used the most reliable evidence available, which is the CAGs.”

From Ofqual’s Interim Report Executive Summary here.

If you happen to have taken a popular A-level which more than 15 students took at your school, you will have been subject to the algorithm. If your A-level choices were less popular, and fewer than 15 students took that subject at your school, greater emphasis was placed on the teacher recommended grades. Still more unfairness and injustice.

A particular example here is Maths (which a lot of people take) and Further Maths (which many fewer people take). This has resulted in many students nationally getting A-level Maths grades adjusted down, whilst their Further Maths grades go through as recommended, creating nonsensical combinations like a C grade for Maths and an A* for Further Maths.

A further inequality here is that in smaller sixth forms, you are more likely to have smaller cohorts of under fifteen taking subjects. Whereas in larger sixth forms – and especially in large sixth form colleges – cohorts are always larger than 15. Therefore the smaller the sixth form, the fewer adjustments have been made to the grades. So it isn’t even necessarily about which subjects you have chosen, but which school or college you happened to be studying them at.

What about appeals?

If you are unhappy with your grade, you have the option of mounting an appeal. This can be done if:

  1. There is an administrative error and the wrong grade has been put into the system. [We haven’t found a single example of this at Churchill].
  2. If your mock exam result shows that you are capable of achieving a higher grade than your final result.

At the moment, that’s it – there are no other grounds for challenging your result, unless you feel you were discriminated against. Mock exams are not the same from subject to subject, much less from school to school – they don’t always assess the full A-level content, they are much more about finding out what candidates need to focus their revision on in the run-up to the real exams than providing a solid grade. We expect mock results to be lower than final results – of course. In some cases, this route will help – but by no means in all.

The only other option open is to sit the full A-level exam in a special Autumn exam series. But who, honestly, could get a higher grade in October or November, without having been in a classroom since March? This is the longest of long shots.

So what can be done?

Currently, the government is saying nothing will change – but surely this can’t stand. The injustices are too great. I think the options are as follows:

  1. Look again at the algorithm and improve the level of “tolerance” around the grade boundaries so that it prioritises the teacher recommendation when a student is being downgraded, especially if they are being downgraded by more than one grade, or moved down from a passing grade to a U.
  2. Just scrap the whole thing and go back to the teacher recommended grades, like Scotland did. Although this would solve the human cost of all the disappointments, it would devalue the 2020 grades compared to previous and following years. An A grade from 2020 would simply not be worth the same as an A grade from another year. As Ofqual said themselves, the teacher recommendations on their own are “implausibly high” for all the reasons outlined above. It would solve the immediate problem – but create another one for the future.
  3. Open up an additional appeals route for candidates who feel an injustice has been done, but whose mocks don’t help them. Again, a tempting route, but what evidence could be used to support such an appeal? In the end, it comes back to the teacher recommendation, and this route very quickly ends up the same as option 2.

My feeling is that Ofqual need to go back and look again at the algorithm, and account for the human cost of squeezing individual candidates into a statistical model that does not account for their unpredictability, their uniqueness, and their actual performance to date. They might have time to do this ahead of GCSE results next week. But, for some A-level candidates, it is already too late – their university places have gone on the basis of results from exams they didn’t even sit.

Who is to blame?

Fundamentally, this is a government decision. As Laura McInerney said in her column for the Guardian today:

“Ultimately, young people have been caught in a farce presided over by an education secretary who let an obviously problematic results day go ahead with no clear plan and no appeals process. How did that happen? Civil servants busy on Brexit? On holiday? Did the exams watchdog not have the bottle to flag problems? I can’t fathom it.

But none of these questions help the Lilys, Matts, or Aatiyahs, or any one of thousands of young people, to understand how a baffling set of grades tanked their future and they weren’t given a clear way to challenge it.”

Laura McInerney, writing in the Guardian here.

I feel deeply aggrieved for those individuals whose futures have been decided not by their own work ethic, revision, effort and learning, but by an algorithm. We will continue to make the case that what has happened is wrong, unfair, and unjust – and hope that the government listens.

How do the new GCSE grades work: 2018 update

Last April I wrote “How do the new GCSE grades work” to explain about the introduction of 9-1 grades for GCSE Maths and English. This year, 9-1 grades will be used in awarding a much wider range of GCSEs, with only a few remaining on the A*-G system. This blog provides an update on the new grading system for the class of 2018.

To help people understand the grading system, Ofqual (the exams regulator) have published this video:

The new 9-1 grades equate to the old A*-G grades as follows:

newgcsegrades

In combined Science GCSE (Double Science), candidates will get two number grades in a variety of combinations as shown below:

sciencegrades

GCSE Double Award Science grades from 2018

In other words, double Science students will get results like 7-7, 7-6, 6-6, 6-5 and so on.

How are the grades awarded?

GCSE grades are awarded after all the exam marking has taken place.

Exams and coursework are marked according to the mark schemes issued by the examination boards. These only have numerical marks on – exams and coursework aren’t graded by markers. When all the marks for everyone who has taken the subject in the country are in, then the grade boundaries are decided according to a formula, so that roughly similar proportions of students nationally get each grade in each subject each year.

In other words, your grade at GCSE in the new system doesn’t just depend on how well you have done – it depends on how well you have done relative to all the other candidates in the country taking the same GCSE as you. If you are the top 20% of candidates in the grade 7 and above group, you will be awarded a grade 9. If you are outside that, you won’t. This will not be the same each year, and will change with each new group of students taking the exams every year.

This is significant because it means that if, nationally, lots of children do very well in the exam, the grade boundaries will move up. If it is a hard exam, and students nationally do not do as well, the boundaries will move down. This makes it difficult for teachers to predict grades accurately; we have to make our best professional judgment on the information available to us.

What does this mean for students?

loverevision

The changes mean that it is impossible for teachers to say “if you do this you will definitely get a grade 5 or above,” because getting a grade 5 depends on how well everyone else in the country does relative to how well you have done. We can’t possibly know how well everyone else in the country has done or is going to do, so all we can do is teach you to get better and better at your own Maths, English, Science, History, Geography and all your other subjects, until you sit the GCSE exam. You have to keep working and pushing yourself to achieve more because what was good enough for a grade 7 last year won’t necessarily be good enough for a grade 7 this year. Don’t settle! You need to keep improving so that you go into the exam at the end of Year 11 fully prepared and confident that you are the best at each subject that you can possibly be – and then you will get the grade that you deserve.

Remember there are posts on this blog to help you to revise effectively, and you can  download our guide to helping your child revise here.

Good luck!

How do the new GCSE grades work?

In the summer of 2017, students in Year 11 will be the first to receive GCSEs under the new 9-1 grading system. They will be graded in this way for English and Maths. In summer 2018, these grades will be awarded for English, Maths, Sciences, History, Geography, French and Spanish. In summer 2019, all GCSEs will be graded this way.

In this blog I will explain what the new grades mean, and how they are awarded. It is quite complicated, but I have tried to make it as simple as possible! I finish the blog this week with “what it means for students” – and this is the most important bit! – so if you get lost, please skip to the end.

Why are the grades changing?

The Government have introduced new GCSE courses for all schools in England. The content of these courses is more challenging than the old-style GCSEs, including less coursework and focusing much more on assessment in exams at the end of the course. The new number grades will identify whether students have taken the new, more challenging GCSEs, or the old-style ones.

What do the new grades mean?

A grade 9 is the highest grade in the new system; a grade 1 is the lowest pass mark. Below a grade 1 is a fail, and will be awarded “U” for ungraded.

In the first year of each new GCSE, broadly the same proportion of students will get a grade 4 or above as would have got a grade C or above in the old system. This has been called a “standard pass” by the Department for Education. If you get a grade 4 or above in English or Maths, you won’t need to re-sit those subjects post-16. The Department for Education has called a grade 5 a “strong pass” which complicates matters. A grade 5 is equivalent (in the first year of each new GCSE) to a high C or a low B in the old system.

Broadly the same proportion of students who would have got grades A or A* in the old system will be awarded grades 7, 8 or 9 in the new system. This means that fewer grade 9s will be awarded nationally than A*s under the old system.

You can see the equivalence of new grades to old in this illustration from Ofqual, the exams and curriculum regulator:

How will the new grades be awarded?

GCSE grades are awarded after all the exam marking has taken place.

Exams and coursework are marked according to the mark schemes issued by the examination boards. These only have numerical marks on – exams and coursework aren’t graded by markers. When all the marks for everyone who has taken the subject in the country are in, then the grade boundaries are decided so that broadly the same proportion of children nationally get a grade 4 and above as would have got a grade C and above, and the same for grade 7 and above with grade A and above.

Once the candidates at grades 7 and above have been decided, a formula will be used that means that about 20% of all grades at 7 or above will be a grade 9. The grade 8 boundary will be midway between grade 9 and grade 7. The same process applies to the other grades (see Ofqual’s explanation here).

In other words, your grade at GCSE in the new system doesn’t just depend on how well you have done – it depends on how well you have done relative to all the other candidates in the country taking the same GCSE as you. If you are the top 20% of candidates in the grade 7 and above group, you will be awarded a grade 9. If you are outside that, you won’t. This will not be the same each year, and will change with each new group of students taking the exams every year.

This is very significant because it means that if, nationally, lots of children do very well in the exam, the grade boundaries will move up. If it is a hard exam, and students nationally do not do as well, the boundaries will move down. This makes it difficult for teachers to predict grades accurately; we have to make our best professional judgment on the information available to us.

What does it mean for students?

loverevision

The changes mean that it is impossible for teachers to say “if you do this you will definitely get a grade 5 or above,” because getting a grade 5 depends on how well everyone else in the country does relative to how well you have done. We can’t possibly know how well everyone else in the country has done or is going to do, so all we can do is teach you to get better and better at your own Maths, English, Science, History, Geography and all your other subjects, until you sit the GCSE exam. You have to keep working and pushing yourself to achieve more because what was good enough for a grade 7 last year won’t necessarily be good enough for a grade 7 this year. Don’t settle! You need to keep improving so that you go into the exam at the end of Year 11 fully prepared and confident that you are the best at each subject that you can possibly be – and then you will get the grade that you deserve.

Remember there are posts on this blog to help you to revise effectively, and you can download our guide for families to helping students revise.

Good luck!