The myth of technology objectivity

Like people across the globe, I too have been afflicted by the football world cup fever. I have particularly enjoyed seeing people in Boston donning colourful jerseys supporting their favourite teams. One day I ran into enthusiastic Norwegian fans at a train station as they waited for their team to come out of the nearby hotel. There are plenty of people from Cabo Verde who call Boston home, and their energy has been infectious. On Fridays, I have noticed a substantial uptick of people wearing red at my mosque, supporting Morocco and praying for their success. And like people across the world, I have been disturbed by how FIFA has conducted itself, on and off the pitch.

In the last few days, I have also been wondering about the broader question of technology and decision-making. How video assisted referee (VAR) technology has been used in particularly games is being discussed in newspapers, podcasts and on social media. Supporters for more technology argue that it allows for checking things that a referee may miss, and that VAR increases the likelihood of a fairer game. They say that VAR fixes the issues of human error. There is merit in that argument. But what we have seen in the last few weeks is that while VAR may find faint connection with a strand of hair (e.g. Croatia-Portugal match), the choice about when to use VAR is entirely human. In some cases, the referees may choose to use VAR to rewind to a particular play to an extraordinary length (e.g. Egypt-Argentina match) and in another instant in the same game, VAR may not be used at all. It is hard to imagine how such subjectivity in decision-making creates a fairer game. One cannot help but recognise that technology alone cannot create systems of fairness, for as long as those using it, creating it or designing it will suffer from explicit or implicit biases. In fact, technology can further amplify the bias by cloaking it in the veil of fairness or independence, thereby making accountability even more difficult for those who are clearly wronged.

This issue, of course, goes beyond FIFA or any particular sport. Last month, at a conference in Istanbul, I listened to legal experts about the use of AI in refugee and asylum cases. While there is excitement in some circles, ethicists and many legal experts in the room were deeply concerned about the use of AI in legal matters, especially when there were marginalised groups involved. Scholars argued that there may be some merit in using AI systems to increase speed, but we cannot disregard the deep biases these systems have inherently built in. Those biases are in there because the systems are imperfect, are designed by people who carry those biases while creating technology, and in the hands of people who may have biases these technologies exacerbate the risks to vulnerable groups. For profit companies making and marketing these technologies as efficient game changers also have an incentive to disregard serious criticisms that can create an impression of partiality. As a result, communities whose life and well-being may depend on thoughtful and rigorously analysed decisions may face harm and injury because of reliance on technology that is both biased and muddies the waters of accountability.

With the rise of AI, and many of my colleagues in engineering, medicine and public health imagining newer applications in complex situations, I have been concerned about what we assume about technology and its ability to make decisions for others, about our biases and our own agency in ensuring accountability. I believe that we need to recognise two fundamental issues here. First, even technologies that may seem autonomous or 'smart' are not free from biases. And second before we apply these technologies, how would we feel if we were on the receiving end of that bias? Would we still have an absolute enthusiastic support? We cannot outsource our decisions – in sports, legal issues, public health, or in any other matter to technology and assume that it would create a fairer world. A fairer world is created when we hold ourselves accountable and confront our biases – not when we transfer them to a machine.