Over the last few years, the world has been moving slowly towards an era where AI algorithms and mathematical models will become integral to decision making in human problems. But, the advent of COVID-19 pandemic, hastening our dependency on untested algorithms. And this what precisely what happened in the UK’s A- level exam result, leading thousands of students and parents to take their protest on to the street chanting “f**k the algorithm.” 

What happened in the A-level exam results?

The UK’s A-level exam is equivalent to India’s 12th standard exam. This year, the authorities abandoned the exams as a result of the pandemic. They further decided to use an algorithm to provide students with their final grades. 

When the final grades announced on August 13th, there was a huge disparity between the final grades and expected grades for many students. In England, Wales and Northern Ireland, nearly 40% of them were lower than teachers’ assessments. In England, 36% of entries had a lower grade than teachers recommended, and 3% were down two grades. According to reports “around 280,000 A-level entries in England being adjusted down from their predictions, with 3.5 per cent reduced by two grades or more.”

Importantly, as WIRED reported, “children from poorer backgrounds were twice as likely to have their results downgraded than those from richer areas.”

How did this happen? 

What led to this disastrous episode was the parameters considered by the algorithms and the unnecessary weightage some of these parameters were given over certain others in production the final grade—a classic case of AI-bias. 

According to Ofqual, the official exam regulator in the UK, the algorithm considered three major parameters. They were:

1. An estimated grade provided by the teacher for each pupil for every subject.

2. A ranking compared with every other pupil at the school within that same estimated grade.

3. The school’s performance in each subject over the previous three years, which is the main culprit. 

The authorised added the third and now controversial parameter to maintain in a national level continuity as well as to ally any fears over grade inflation caused by teachers assessing their own students. Hence they chose to modify the “potentially biased teacher assessments” by taking into account the school’s historical performance and other factors that may have had little to do with the individual student. As a result, 60% of final grade was determined by those additional factors and instead of teacher’s assessment. 

As a result, even if your teacher predicted you an A grade, if you were ranked tenth in your subject at your school and the tenth highest-performing person at your school for that subject received a B grade for each of the previous three years, then you would most likely be awarded a B grade, points out ——

The algorithm also downgraded 39 per cent of the A-level grades predicted by teachers in England, especially students from poorer backgrounds. According to WIRED, “this means that a bright pupil in a poorly performing school may have seen their grade lowered because last year’s cohort of pupils didn’t do well in their exams.”

The BBC points out that this method was adopted to make sure that the grades this year - even without exams - would be consistent with how schools had done in the past.

What now for the students?

Initially, the government in England has introduced what it’s calling a ‘triple lock’, which provided three stages of appeal for students, where students will effectively get to choose their grade from a teacher assessment, their mock exam results, or a resit to be taken in the autumn.

However, after the upload and widespread protest, the government has decided to ditch the algorithm based grading entirely and instead, students will receive grades based on their teachers’ estimates.

Image by Alberto G via Flickr