Technology needs humanity

The grief of those who lost loved ones in this pandemic is real and palpable. Those who had to bury their family and friends — and have not had a chance to grieve properly — have had to go through what few of us can even imagine. Those who lost their livelihoods or live in the continued fear of losing their income and homes are living in perpetual anxiety and worry. But the pandemic’s impact goes beyond the immediate loss of lives and livelihoods. The pandemic will negatively impact, at least a generation’s life, if not more. Students globally have suffered and continue to suffer. And it is not just the pandemic that is to blame here. The technology we have built to respond to the emerging challenges is just as blameworthy.

Earlier this year when schools were closed, A Level exams across the world were cancelled. Instead, students, on the cusp of starting their university careers, were told they would be assigned grades based on prior assessments, teacher reports and historical data from their schools. In other words, an algorithm would calculate the students’ grades. In a world fixated on technology to solve our problems — this sounded like a great idea.

Except it was a disaster.

The results announced earlier last week showed widespread problems with grades across the board. In the UK, the algorithm assigned grades lower than what the teachers had assigned in almost 40% of the students. The algorithm worked well for richer students in private schools and their grades were better than expected. For poorer students, the grades were much worse, including higher number of calculated fails (or U) that were a shock to teachers and students. The use of historical school data in the eyes of the computer algorithm meant that students from low-income neighbourhoods and attending schools with bad grades in the past, were penalised and those who attended affluent, successful schools were given higher grades because of their peers’ past performance. The algorithm in essence took away a students’ ability to move up and break social barriers. It rewarded a poor student with grades that showed that class mattered more than promise. As if poorer students weren’t already suffering enough in the pandemic, exam results were a perfect “kick while you are down” moment. The poor, aggrieved students may now have to pay hefty fees for appeals and will most likely not get their revised grades in time for their place at university. The generational loss would be profound, and only increase inequality.

While many who are wedded to AI may try to spin it one way or another, the reality is that our algorithms remain biased against already marginalised people. The A Level results are not an anomaly. Medical technologies including pulse oximeters are well known to underperform with patients with dark skin. Face recognition algorithms in popular apps are notorious for making repeated errors in recognising non-white races.

The typical response from those in charge (institutions and governments) is to make a statement of concern, reaffirm abstract notions of equity, and talk about how this is an unprecedented time and this is the best we could do. That is simply unacceptable. We can and must do better. There is a critical place of empathy, ethics, decency and fairness in all technology. That place cannot remain vacant. The world should not be run by algorithms that discriminate, and technologies that victimise on the basis of skin colour, or the neighbourhood one grows up in. Privilege has ruined countries and societies, we should not allow technology to further cement inequity.

We rely increasingly on technology, but our technology increasingly needs humanity.

Published in The Express Tribune, August 18^th, 2020.

Like Opinion & Editorial on Facebook, follow @ETOpEd on Twitter to receive all updates on all our daily pieces.