Social media recently blew up over a recent paper in Ethics & Information Technology called ChatGPT is Bullshit (https://link.springer.com/article/10.1007/s10676-024-09775-5). The authors, Michael Townsen Hicks, James Humphries and Joe Slater of the University of Glasgow, claim that falsehoods produced by large language models like ChatGPT are best characterized as bullshit rather than the commonly used term "hallucination." They're wrong.
I'll grant that the authors did an excellent job of producing a provocative article that received vastly more attention than anything I'll ever write. Well done on that front. They also raise some interesting points, but ultimately I think that they're flat wrong in their conclusions. As nearly as I can tell, the authors are philosophers. Since they're at the University of Glasgow, they're probably good philosophers. But they're still wrong.
The crux of the problem Hicks and his colleagues are trying to describe is the tendency of large language models like ChatGPT to produce falsehoods. LLMs will sometimes just make stuff up. The most common term for this is hallucination; the more technical term is confabulation. A confabulation is an unintentional fabrication of information typically caused by memory gaps.
Hicks and colleagues go through some pretty intricate rhetorical gyrations to support their claim that ChatGPT is bullshit. You can read the article to see their full set of arguments, but basically they're saying that ChatGPT (and LLMs) are bullshit because they're not designed to represent the truth. They're designed to give the impression that they're representing the world as it is. The authors rely on Harry Frankfurt's definition of bullshit as a "reckless disregard for the truth." Hicks and colleagues argue that LLMs have no notion of the truth, therefore are indifferent to truth and thus are bullshit. (I'm way oversimplifying here, but that's the gist of their argument.)
The authors use some interesting (and questionable) means to support their argument. The most problematic of these is that they conveniently do away with the need for intention by creating the term "soft bullshit." Hard bullshit is produced with the intention to mislead about the bullshitter's agenda. Soft bullshit is produced without that intention.
This is a pretty massive alteration of Frankfurt's idea of bullshit and I think Hicks and colleagues go too far here. They also conveniently disregard the idea of motivation. I've known my fair share of bullshitters and they always have the underlying motivation of manipulating the receiver of the bullshit. LLMs do not have such underlying motivations, so I question the conclusion that they're bullshit. The authors also criticize the term hallucination because it anthropomorphizes LLMs. This is curious given the need to anthropomorphize LLMs to call them bullshit.
The authors hint around at what is really going on, but I wish they had gone further. I'm not convinced that LLMs are bullshit, but (as the authors point out) LLMs can absolutely be used to produce bullshit. But that's not on the LLMs, that's on the user. I can also take just about any book and manipulate it to produce bullshit. That doesn't make the book bullshit. The authors do include a section titled "ChatGPT is a bullshit machine." I agree that it CAN be, but that doesn't mean it inherently IS a bullshit machine.
Ultimately, the authors just substitute one imperfect metaphor for some others (hallucination and confabulation). The problem with the bullshit metaphor is that it goes too far in putting the blame on AI. When AI produces bullshit that makes it out into the world, the blame should be placed squarely on the user, not the LLM. A user that produces bullshit with ChatGPT is either ignorant of LLMs limitations, careless/lazy, or is the real bullshitter. It's on the user, not ChatGPT.
The good news about the article is that it draws attention to the limitations of LLMs. That may, somewhat ironically, lead to less AI produced bullshit. Do you agree? Let me know at craig@AIGoesToCollege.com.
If you want more AI Goes to College, check out the podcast at https://www.aigoestocollege.com/. Co-host Dr. Robert E. Crossler (better known as Rob) and I have some great discussions about the challenges of integrating AI into higher ed.