We generally understand how LLM hallucinations work. An AI model tries to generate what seems like a plausible response to whatever you ask it, drawing on its training data to construct something that sounds right. The actual truth of the response is, at best, a secondary consideration.

snip

But over the last few months, it has occurred to me that, for all the hype about generative AI systems “hallucinating,” we pay much less attention to the fact that the current President does the same thing, nearly every day. The more you look at the way Donald Trump spews utter nonsense answers to questions, the more you begin to recognize a clear pattern — he answers questions in a manner quite similar to early versions of ChatGPT. The facts don’t matter, the language choices are a mess, but they are all designed to present a plausible-sounding answer to the question, based on no actual knowledge, nor any concern for whether or not the underlying facts are accurate.

snip

This is not the response of someone working from actual knowledge or policy understanding. Instead, it’s precisely how an LLM operates: taking a prompt (the question about job losses) and generating text based on some core parameters (the “system prompt” that requires deflecting blame and asserting greatness).

The hallmarks of AI generation are all here:
• Confident assertions without factual backing
• Meandering diversions that maintain loose semantic connection to the topic
• Pattern-matching to previous responses (“ripped off,” “billions of dollars”)
• Optimization for what sounds good rather than what’s true