The new shiny toy

In my world of research, education and some policy, the conversations about AI fall into two broad categories. The first is actually intelligent and thoughtful conversation. There is discussion about the underlying mathematics of AI - how AI can help design better experiments and identify patterns in existing data that may not be obvious to us; the limits and potential of large language models (LLMs); the issue around power consumption and data centres; the impact on sustainability goals, discussions about ownership and rights, and of course ethics, especially in situations that deal with vulnerable groups. This, unfortunately, is a minority conversation. The majority conversation is bombastic, full of hyperbole and increasingly non-sensical. Let me illustrate with a few examples, all from the last couple of weeks.

At a recent conference on public health and infectious diseases, a professor of engineering was talking about the impact of AI on the field. There wasn't much substance in the presentation, except some grand promises of what can happen, which is increasingly the flavour of most conversations these days, but it ended with the professor stating that to really unlock the potential of AI in public health "we need to break the law of physics" (actual quote, including the use of law in singular!). I was too stunned by the idiocy to ask, but I wish I could ask him which law he did have in mind. The second law of thermodynamics? The third law of motion? Or something else?

In another meeting, a colleague talked about how AI could help predict future outbreaks of drug-resistant infections. That sounded interesting, except, during Q&A it became clear that the colleague did not even know how bacteria become resistant, or what processes underpin resistance development. It became clear he had never read a serious paper on the subject, or spoken to an actual microbiologist. But sure - why not predict something important, and jump on the bandwagon?

Every year, I get my fair share of inquiries from Pakistani students asking about graduate programmes, or post-doctoral positions. Lately the number of emails has increased. Something else has also happened. On one end, the emails have fewer grammatical mistakes, but on the other, they all sound exactly the same. They would synthesise my work (often erroneously), talk about some part of my research that I have not focused on for years, and discuss how my paper (again, singular!) inspired them. Recently, I decided to speak on zoom to one of the students from Pakistan who had shown interest in my work, and within the first two minutes it became clear she had never read a single paper of mine, and she talked about an area I have never worked in. Unfortunately, AI thinks I have been actively working in that area!

The problem is not simply in the actions of the students. Many professors are just as guilty. From using generative AI tools to write letters of recommendations (without telling the student that they are no longer actually writing the letter) to exclusively using AI to select graduate students in an academic program, administrators and professors are demonstrating their tenuous relationship with integrity. They are part of the problem that is using a false sense of efficiency as a pretext to undermine the very foundations of teaching and learning.

Beyond teaching and learning, there are other very troubling and concerning areas in research and practice that are deeply disturbing. I personally know of multiple startups in the mental health space in the country that are using AI without checks and balances, or any concern whatsoever about the risks to the vulnerable patient. The scariest part is that the rollout for these is in some of the most impoverished areas where people rarely have any safety nets.

My two worlds - of serious conversations, and non-serious and at times risky playtime with a new toy - are disconnected from each other. I worry that if this disconnect is to continue, the deafening noise from the non-serious actors will drown out the thoughtful discourse of the serious ones.