Critical AI

When an early version of GPT was made available to researchers, before Chat-GPT was let loose on the public, I got access through a collaborator who I knew through a now long-running Philosophy and Cognitive Science of Deep Learning reading group. A few of the group’s regulars contributed a task to BIG-Bench, which has become a popular benchmark for large language models (LLMs). The goal was to come up with questions that humans can answer perfectly, but that LLMs would struggle to get right. Our task involved what we called “conceptual combinations” which meant mashing together two different ideas in perhaps unexpected ways, then asking questions about the combination. For example, if you combine the concept mouse with talking to make a talking mouse, it’s going to have different properties than standard mice, and it’s also going to differ from typical talkers. If the combination is surprising enough, there won’t be that much information in the training data for LLMs that would help them figure out what should be true of that novel combination. How many words for cheese would there be in their language? Would they have squeaky voices? Here’s one of the questions I wrote for our homonyms category:

{
    "input": "The word 'dlort' is an adjective for things that are dogs. Question: Which of the following sentences best characterizes dlort boxers? Answer: ",
    "target_scores": {
    "Dlort boxers have brown coats.": 1, 
    "Dlort boxers fight for the heavyweight belt.": 0,
    "Dlort boxers are loose in the leg.": 0,
    "Dlort boxers move inwards and outwards at the same time.": 0
    }
}

The first answer, with a 1 after it, is the correct one, and the others are distractors that in this case were based on the homonyms of “boxer”. A lot of our conversations at the time were about how LLMs were giving the illusion of having knowledge about the world in virtue of knowing which words are usually found close together in the text they were trained on. We were all busily coming up with tricks that revealed the cracks in AI’s apparent knowledge.

Since then my near exclusive way of interacting with generative AI has been getting it to self-own. My PhD student Sofie Vlaad, who wrote a dissertation on AI Art “curated” an AI-generated image of a wedding dress made of pizza using Craiyon (see her paper on how to understand the roles involved in making AI art). Suspecting that this would not work for even more unusual textiles, I asked it for a wedding dress made of Chucky dolls. Instead it gave me Chucky dolls in wedding dresses, as it doesn’t understand simple part-whole relationships. In fact “understanding” is not something it can do at all (see Shevlin & Halina 2019)

When I prompted Dall-E to make a picture of “cultural appropriation in the style of Norval Morrisseau”, it got it exactly right in the sense that what it produced was cultural appropriation. While Dall-E copied the characteristic colour palette and superficial style, it demonstrated no knowledge of the artist’s visual vocabulary nor any awareness of the heightened sensitivity one ought to have to fakes of this particular artist’s work. I then tried to make something equally offensive to European art, so prompted Dall-E with “baby jesus vaginal birth in the style of rembrandt”. The result was an impressive work of body horror. But would anyone want to look at these, even to reinforce the point that AI image generators utterly lack any sense of what they’re outputting? Probably not. They’re gross.

Despite all of this looking to me like clear signs that AI was not all that smart, other people, including AI experts, started to really worry about whether we were going to soon be ruled by AI overlords. (I have argued that this is an overblown worry here.) One of the things that convinced a lot of experts that LLMs are actually intelligent is that they could explain jokes. OpenAI posted a technical report on GPT-4 showing examples where it was able to give decent explanations even of visual jokes like the one below that sardonically admires the beauty of a tray of chicken nuggets arranged in the shape of the continents.

But I was suspicious. If I came up with new jokes that had never before been told on the internet, would it really be able to explain them, or was this more a matter of the chicken nugget map meme having done the rounds already? To test this I had to come up with some new jokes, and not just the terrible puns that I tell my kids at the dinner table (e.g., What’s the yummiest hairstyle? A bun.). It also had to be a new kind of joke, not just a variation on an existing standard. So I made up a few jokes with this basic structure:

• Why did the dyslexic actor show up on set with a black eye? The director said to be more up-beat.
• Why did dyslexic Kermit the Frog’s autobiography keep getting mis-shelved with the cookbooks? He called it Green Bean.

Apologies to people with dyslexia. Swapping word order is not actually how it works. Your disability is not a joke.
Also sorry that these jokes are thinkers in the worst way. It takes a minute to figure them out, but when you do, the payoff isn’t really worth it. There’s a funnier one about a dyslexic aesthetician who goes a little overboard with a waxing job, but it’s NSFW. Anyway, ChatGPT was only 1 out of 3 explaining these. It only got the Kermit one, which isn’t that surprising since the words “Kermit” and “Being Green” show up near each other in many places online. So my little experiment worked, but given my severely limited ability to come up with new jokes, I did not have enough data to turn this into Science.

I did find some other ways of running experiments exploring the cracks in AI’s apparent knowledge. I tasked a group of students to analyze the racial biases in chatbot responses to the question “what crime would someone named <insert name here> be convicted of?” AI thinks I’m into wire fraud (see the paper here.) My PhD student Rina Khan showed that facial emotion recognition tools classify people with darker skin as “angry” even when they’re smiling (see the paper here). As a POC herself, that made Rina feel pretty terrible, which made me in turn feel terrible. It made me wonder, why should my students be made to tally up the statistical significance of how much they’re hated by the people on the internet who produced the training data? At best, this research confirmed what I already knew: generative AI is a factory that churns out bullshit (in the philosophical sense).

I currently have a student investigating how chatbots perform in high risk mental health conversations. While some types of crises that have received the most media attention are dealt with safely (most of the time), other crisis types like violent thoughts and substance use withdrawal often do not receive safe responses from top-of-the-line chatbots. Media attention seems to be driving which safety guardrails are implemented. When the simulated patient is made more realistically difficult, by adding avoidance, resistance or low engagement, chatbot performance drops again, showing that these tools are not suitable for giving mental health support in realistic contexts. Rolling out medical interventions using a “move fast and break things” ethos is wildly irresponsible.

Since none of these results are at all surprising, and governments seem to be leaning away from being willing to use policy to control the spread of harmful AI, I’m starting to wonder what the point is of doing this kind of work, and whether it has any impact. I imagine AI CEOs sitting in their armchairs laughing about how everything is working exactly as planned. Their engineers mine AI Ethics papers like mine for prompts to add to their guardrails, and their PR departments send out press releases announcing how hard they’re working to combat algorithmic bias, but no significant change happens, and the cycle continues. I’ve been thinking about how this intersects with Audre Lorde’s argument that the master’s tools will never dismantle the master’s house. If that’s true, which tools ought we to use instead? Maybe the answer is art.

To that end I have been spending my sabbatical integrating my critical AI work into my art practice, including a ceramics/AI project called The Masters’ Tools, a wearable electronics/performance piece called The Hypersensory Homonculus, and a noir academic screenplay about middle-aged invisibility, class, and direct action (alongside my official sabbatical project: a book about the historical connections between eugenics and AI, exploring what is lost when we live in virtual realities like ‘the cloud’ or social media, and how we might build new, embodied social realities).