AI chatbots might be better at swaying conspiracy theorists than humans (original) (raw)
Co-author Gordon Pennycook: "The work overturns a lot of how we thought about conspiracies."
A woman wearing a sweatshirt for the QAnon conspiracy theory on October 11, 2020 in Ronkonkoma, New York. Credit: Stephanie Keith | Getty Images
A woman wearing a sweatshirt for the QAnon conspiracy theory on October 11, 2020 in Ronkonkoma, New York. Credit: Stephanie Keith | Getty Images
Belief in conspiracy theories is rampant, particularly in the US, where some estimates suggest as much as 50 percent of the population believes in at least one outlandish claim. And those beliefs are notoriously difficult to debunk. Challenge a committed conspiracy theorist with facts and evidence, and they'll usually just double down—a phenomenon psychologists usually attribute to motivated reasoning, i.e., a biased way of processing information.
A new paper published in the journal Science is challenging that conventional wisdom, however. Experiments in which an AI chatbot engaged in conversations with people who believed at least one conspiracy theory showed that the interaction significantly reduced the strength of those beliefs, even two months later. The secret to its success: the chatbot, with its access to vast amounts of information across an enormous range of topics, could precisely tailor its counterarguments to each individual.
"These are some of the most fascinating results I've ever seen," co-author Gordon Pennycook, a psychologist at Cornell University, said during a media briefing. "The work overturns a lot of how we thought about conspiracies, that they're the result of various psychological motives and needs. [Participants] were remarkably responsive to evidence. There's been a lot of ink spilled about being in a post-truth world. It's really validating to know that evidence does matter. We can act in a more adaptive way using this new technology to get good evidence in front of people that is specifically relevant to what they think, so it's a much more powerful approach."
When confronted with facts that challenge a deeply entrenched belief, people will often seek to preserve it rather than update their priors (in Bayesian-speak) in light of the new evidence. So there has been a good deal of pessimism lately about ever reaching those who have plunged deep down the rabbit hole of conspiracy theories, which are notoriously persistent and "pose a serious threat to democratic societies," per the authors. Pennycook and his fellow co-authors devised an alternative explanation for that stubborn persistence of belief.
Bespoke counter-arguments
The issue is that "conspiracy theories just vary a lot from person to person," said co-author Thomas Costello, a psychologist at American University who is also affiliated with MIT. "They're quite heterogeneous. People believe a wide range of them and the specific evidence that people use to support even a single conspiracy may differ from one person to another. So debunking attempts where you try to argue broadly against a conspiracy theory are not going to be effective because people have different versions of that conspiracy in their heads."
By contrast, an AI chatbot would be able to tailor debunking efforts to those different versions of a conspiracy. So in theory a chatbot might prove more effective in swaying someone from their pet conspiracy theory.
To test their hypothesis, the team conducted a series of experiments with 2,190 participants who believed in one or more conspiracy theories. The participants engaged in several personal "conversations" with a large language model (GT-4 Turbo) in which they shared their pet conspiracy theory and the evidence they felt supported that belief. The LLM would respond by offering factual and evidence-based counter-arguments tailored to the individual participant. GPT-4 Turbo's responses were professionally fact-checked, which showed that 99.2 percent of the claims it made were true, with just 0.8 percent being labeled misleading, and zero as false. (You can try your hand at interacting with the debunking chatbot here.)
Screenshot of the chatbot opening page asking questions to prepare for a conversation
Credit: Thomas H. Costello
Screenshot of the chatbot opening page asking questions to prepare for a conversation Credit: Thomas H. Costello
Participants first answered a series of open-ended questions about the conspiracy theories they strongly believed and the evidence they relied upon to support those beliefs. The AI then produced a single-sentence summary of each belief, for example, "9/11 was an inside job because X, Y, and Z." Participants would rate the accuracy of that statement in terms of their own beliefs and then filled out a questionnaire about other conspiracies, their attitude toward trusted experts, AI, other people in society, and so forth.
Then it was time for the one-on-one dialogues with the chatbot, which the team programmed to be as persuasive as possible. The chatbot had also been fed the open-ended responses of the participants, which made it better to tailor its counter-arguments individually. For example, if someone thought 9/11 was an inside job and cited as evidence the fact that jet fuel doesn't burn hot enough to melt steel, the chatbot might counter with, say, the NIST report showing that steel loses its strength at much lower temperatures, sufficient to weaken the towers' structures so that it collapsed. Someone who thought 9/11 was an inside job and cited demolitions as evidence would get a different response tailored to that.
Participants then answered the same set of questions after their dialogues with the chatbot, which lasted about eight minutes on average. Costello et al. found that these targeted dialogues resulted in a 20 percent decrease in the participants' misinformed beliefs—a reduction that persisted even two months later when participants were evaluated again.
As Bence Bago (Tilburg University) and Jean-Francois Bonnefon (CNRS, Toulouse, France) noted in an accompanying perspective, this is a substantial effect compared to the 1 to 6 percent drop in beliefs achieved by other interventions. They also deemed the persistence of the effect noteworthy, while cautioning that two months is "insufficient to completely eliminate misinformed conspiracy beliefs."
Just the facts...
This chatbot approach proved effective for a wide range of conspiracy theories, from classics like the JFK assassination, Moon landing hoaxes, and the Illuminati, to more recent examples like the 2020 election fraud claims and COVID-19. "We also got these smaller spillover effects, showing that to some extent it was making people less generally conspiratorial," said co-author David Rand, a cognitive scientist at MIT. "It also increased their intentions to do things like ignore or block social media accounts sharing conspiracies. So it seemed like it worked, and really worked broadly. [The chatbot] is able to really meet people exactly where they are rather than just providing general blanket debunks."
"If you look at the actual conversations, the AI overwhelmingly provided non-conspiratorial explanations for these seemingly conspiratorial events, encouraging people to engage in critical thinking, and providing counter-evidence," said Rand. The chatbot was also polite and tried to build a rapport with the participants. The team has conducted follow-up research, not yet published, changing the prompt to dispense with rapport-building and just present the factual evidence-based counter-arguments; the results were just as strong. Another tweak to the prompt directed the chatbot to not rely on facts and evidence at all and try to talk participants out of the conspiratorial beliefs based on rapport alone. That did not work at all. "This suggests it really is the facts and evidence that are doing the work here," said Rand.
But would people who embrace conspiracy theories willingly engage with an AI chatbot designed to sway their beliefs? Rand pointed out that many conspiracy theorists actually want to talk about their beliefs. "The problem is that other people don't want to talk to them about it," he said. So the authors suggest posting a link to the chart in conspiracy forums, inviting people to do their own research by engaging with it as one strategy. Perhaps internet search terms related to conspiracy theories could be matched to AI-generated summaries of accurate information, or AI social media bots could reply to users who sharing inaccurate conspiratorial content.
The chatbot performed less well in an interaction study the team launched in the wake of this summer's assassination attempt on former president Donald Trump, during which numerous conspiracy theories emerged. The chatbot was given all the relevant information and context it needed to debunk those conspiracies. The team also gave it search-engine capabilities to search news articles and related online materials about the attempt.
"We found that it was not quite as effective as the conspiracies that were in the data that the models were trained on, but it did have a pretty noticeable effect," said Costello. "It reduced people's assassination-related conspiracies by six or seven points on a scale from zero to 100." The authors suggest this is because less factual information is available in the immediate aftermath of such an event or in the early days of a pandemic, for example. So the chatbot has less material to work with.
"Purveyors of misinformation sometimes use a technique called the 'Gish gallop'—named after a creationist who was particularly good at it—in which a conversation is flooded with an overwhelming number of mistruths," H. Holden Thorp, editor-in-chief of Science wrote in an accompanying editorial about the study's findings. "Humans are unable to respond to this effectively, no matter how skilled the manner of their responses may be. But the LLM cannot be overwhelmed; it can cite counter evidence indefinitely. Although it is perhaps discouraging that a machine might be better at countering misinformation than a human, the fact that it is ultimately the scientific information that does the persuading is a relief."
Science, 2024. DOI: 10.1126/science.adq1814 (About DOIs).
Listing image: Stephanie Keith | Getty Images
Jennifer is a senior reporter at Ars Technica with a particular focus on where science meets culture, covering everything from physics and related interdisciplinary topics to her favorite films and TV series. Jennifer lives in Baltimore with her spouse, physicist Sean M. Carroll, and their two cats, Ariel and Caliban.
