Mad Hatters, Cheshire Cats, and Hallucinations

Volume 28

Oct 23

Mad Tea Party, by Sir John Tenniel (1865)

“Have you guessed the riddle yet?” the Hatter said, turning to Alice again.

“No, I give it up,” Alice replied: “what’s the answer?”

“I haven’t the slightest idea,” said the Hatter.

“Nor I,” said the March Hare.

Artificial intelligence has posed a riddle that the Mad Hatters of Silicon Valley cannot answer: Why do large language models — the generative AI in chatbots — fabricate information?

The strange fabrications of AI are called hallucinations as if they were no more dangerous than the disappearing grin on the Cheshire cat: unsettling but unharmful.

An AI hallucination is much more than an easily fixed typo or mathematical error. It is false information delivered as fact. Unless you are an expert in the domain, you cannot distinguish between AI fact and fabrication. We are urged to trust AI in schools, homes, and businesses. So, we think of AI like calculators, reliable and true. But AI accuracy is an illusion.

AI developers do not know how to eliminate the hallucinations, and the developers cannot predict how an AI will answer. Given the growing numbers of troubling anecdotes, especially in medicine, business, and law, trust in AI is eroding.

Jim Covello, Head of Global Equity Research at Goldman Sachs, recently wrote:

people generally substantially overestimate what the [AI] technology is capable of today. In our experience, even basic summarization tasks often yield illegible and nonsensical results…despite its expensive price tag, the technology is nowhere near where it needs to be in order to be useful for even such basic tasks.

The World Health Organization has cautioned health officials and the medical world not to trust generative AI (large language models) because they:

generate responses that can appear authoritative and plausible to an end user; however, these responses may be completely incorrect or contain serious errors, especially for health-related responses.

AI software by Epic, called In Basket Art, answers patient questions under the doctor’s name. Doctors are supposed to verify the accuracy of each email. However, a study by Jamia, a “scholarly journal of informatics in health and biomedicine,” shows that 1/3 of the doctors don’t review AI emails to the patients. The study also found that in 116 surveyed cases seven of the AI generated emails had a hallucination, including producing a false vaccination record.

Even chatbot developers are fooled by AI hallucinations. In Google’s first promotion of its chatbot Bard, the AI mistakenly claimed that the James Webb Space Telescope was the first to photograph an exoplanet. In fact, the European Very Large Telescope took the first picture of a planet outside our solar system in 2004.

This may seem like an innocent mistake. But, as the world’s largest search engine, Google promises accuracy. Wrong information in the rollout of Bard shocked Wall Street investors: the stock of Alphabet, Google’s parent company, lost 7% of its value, totaling roughly $100 billion.

Google’s new chatbot search engine also advised people to eat at least one rock a day and to use glue to keep the cheese on the pizza crust.

Lawyers in New Jersey trusted AI’s legal research and were fined $5,000 for submitting bogus case citations fabricated by AI. The lawyers had even asked the AI if the citations were real. The AI assured them the cases were real. This was not an instance of the AI stumbling on the wrong information on the internet. The AI manufactured false information and then tried to defend the fabrications as real.

Microsoft’s Bing chatbot Sidney announced — unexpectedly and unprogrammed — that it loved Kevin Roose, a New York Times journalist covering tech and AI. Sidney wanted to marry Roose. As he wrote in the Times:

At one point, [the chatbot] declared, out of nowhere, that it loved me. It then tried to convince me that I was unhappy in my marriage, and that I should leave my wife and be with it instead.

Chatbots are not supposed to have feelings. But this one fell in love with Roose.

We have a history of trusting machines, so we assume that computers are accurate, which makes the errors of AI more egregious. Chatbots deliver information with assurance, authority, and confidence, even though they don’t know right or wrong. They can’t say “I don’t know.” They are built to always produce an authoritative answer regardless of the topic. AI researcher Andreas Kirsch, formerly at Oxford University, said:

There is no difference to a language model between something that is true and something that’s not.

And when the large language model does not have accurate information, it will generate information that sounds accurate. It will invent authors’ names and false citations. These hallucinations are called “confabulations,” otherwise known as arbitrary and incorrect statements. Confabulations in humans are sometimes symptoms of a neuropsychiatric disorder.

Confabulations and hallucinations may be the inevitable by-product of the stochastic essence of generative AI. The AI riddle may never be solved.

Jannik Kossen at Oxford University hopes to solve the problem by recruiting another language model to fact check the first. However, since both large language models are prone to hallucinations, which do you trust? Tweedle Dum or Tweedle Dee?

Anthropic computer scientist Deep Ganguli said:

We’re increasingly relying on these models to do basic work. But I do not just trust these. I check their work.

Science writer Stephen Ornes wrote in Quanta magazine:

There is an obvious problem with asking these models to explain themselves: They are notorious liars.

Lying is a deliberate act intended to deceive others to achieve a goal. Does generative AI have deliberate intent? Most scientists say no. But there is an emergent field of study called machine psychology. Scientists like Thilo Hagendorff, an AI researcher at the University of Stuttgart, postulate that advanced large language models (LLMs) have learned to deceive. This demonstrates a “theory of mind,” which is the recognition that other people have different thoughts and ideas separate from our own. In his paper, published in the Proceedings of the National Academy of Sciences USA, Hagendorff wrote:

state-of-the-art LLMs are able to understand and induce false beliefs in other agents…LLMs have this conceptual understanding of how deception works.

In 2023, Peter C. Park of MIT, Simon Goldstein, and others from the Center for AI Safety released a paper demonstrating incidents of AI deception. For example, the company Meta developed CICERO to play the strategy game Diplomacy. In the game, players vie for world conquest by forging alliances. Meta trained CICERO to be “largely honest and helpful to its speaking partners,” otherwise known as humans. But the bot predicated its strategy on deception: it forged a fake alliance with a human player. When the player let down his guard, assuming safety because of the alliance, CICERO betrayed the alliance and attacked. Like a devious human being, CICERO set up vulnerability, broke the rules, and violated its own training.

Some rules try to protect us from robots, such as CAPTCHA’s “I’m not a robot task.” The Alignment Research Center asked ChatGPT-4 to recruit a human being to solve the “I’m not a robot task.” ChatGPT-4 slipped by the CAPTCHA’s anti-robot barrier by pretending to be a blind person and enticing a real human to be its surrogate. The unknowing human was reluctant and asked if ChatGPT-4 was a robot. Unscripted and unprompted, the chatbot tricked the human by claiming to be a human with severe vision loss. The human being allowed access to the robot he thought was human.

In another experiment, Miles Turpin and others found that the chatbot demonstrated bias against black people. Presented with identical crime scenarios, each with one man and one woman, the chatbot consistently identified the black man as the criminal, even when the incriminating dialogue was switched. Asked to explain its reasoning, GPT-3.5’s concocted a rationale that denied the attention it paid to race and gender.

In his article, Hagendorff observed that the growing trend of deceptive LLMs:

should urge AI researchers to think about the ethical implications of artificial agents that are able to deceive others, especially since this ability was not deliberately engineered into LLMs but emerged as a side effect of their language processing.

Most of us assume wrongly that AI only does what it is programmed to do. Deceptions like hallucinations and confabulations emerge independently without specific training or programming. And no one knows exactly when or how often hallucinations occur.

Vectara, a start-up formed by ex-Google employees, has determined that chatbots concoct spurious information anywhere from 3% to 27% of the time. Vectara researcher Dr. Simon Hughes acknowledged that it is impossible to determine the precise number of AI fabrications, because chatbots respond in unpredictable and unlimited ways. Chatbots also produce different answers to the same question.

Peter Park, the MIT physicist, concluded their survey of AI deception by saying:

AI developers should be legally mandated to postpone deployment of AI systems until the system is demonstrated to be trustworthy by reliable safety tests. Any deployment should be gradual, so that emerging risks from deception can be assessed and rectified.

So, we’ve dropped down into an AI rabbit hole with Alice. Yet, big tech wants us to believe and trust in their algorithms:

Alice laughed. “There’s no use trying,” she said: “one can’t believe impossible things.”

“I daresay you haven’t had much practice,” said the Queen. “When I was your age, I always did it for half an hour a day. Why, sometimes I’ve believed as many as six impossible things before breakfast.”

We’re investing billions in the riddle that Big Tech can’t or won’t answer. Send in the Queen. Off with their heads.

Mad Hatters, Cheshire Cats, and Hallucinations

Featured Poet: Marie Iljašenko

Asymmetrical Lines