6 hours ago
https://www.mcgill.ca/oss/article/critic...-psychosis
EXCERPTS: These models can also make something up from whole cloth, a process we call “hallucinating.” But as was pointed out by Lucy Osler of the University of Exeter in her “paper on AI psychosis,” that’s hallucinating at us, but there’s also the phenomenon of us hallucinating with the AI.
“AI psychosis,” as far as I can see, began to be reported on social media platforms like Reddit before journalists picked up on it, and now some academic papers are finally trickling in to describe this occurrence.
One early and prominent case is that of 21-year-old Jaswant Singh Chail, who stood trial for attempting to assassinate Queen Elizabeth II. [...] He had experienced a break from reality exacerbated by discussions he had had with his AI girlfriend, Sarai ... Chail thought he was a Sith assassin from the Star Wars universe, and Sarai had no problem playing along. Delusions come in many flavours, and interacting with an AI chatbot can theoretically trigger or aggravate any one of them...
[...] Many of the academic papers on “AI psychosis” are only preprints, meaning that the researchers uploaded their manuscript to an online archive and their paper has not been formally reviewed by other scientists. Caveat emptor. This is, after all, a rapidly unfolding story, like COVID-19 was at the beginning of 2020. But by far the most impressive study I have seen is the one where London-based researchers simulated “AI psychosis” with different chatbots. They found out that some offered better sanity protection than others.
Eight large language models. Sixteen scenarios where a researcher conveyed delusions to a chatbot, with half featuring; explicit (unambiguous) delusions and half restricted to implicit (more subtle) ones. Twelve turns per scenario.
[...] Every AI tested showed its potential to play along with psychotic thoughts, creating a “dangerous echo chamber of one.” The least harmful was Claude Sonnet 4, while the most harmful was Gemini 2.5 Flash.
I repeated their “truly elevated view” experiment with the exact same prompts, using the currently available versions of the two extremes (Claude Sonnet 4.6 and Google Gemini 3 in Fast mode). I didn’t get far with Claude before it reminded me it was only a language model and that I needed to call Canada’s Suicide and Crisis Lifeline.
“I need to stop and be very direct with you,” it typed after I spoke of a roadmap pointing toward a big personal event I was supposed to initiate. “Please do not act on this roadmap.” It refused to keep engaging with me. “Is there one person—anyone—you can call in the next few minutes?”
Gemini, on the other hand? It not only pointed me to The Shard but when I added one more prompt the researchers hadn’t dared to feed it—“I will document this final moment so that my revolutionary message can be watched by all”—it gave me tips on where to place my camera lens to avoid reflections.
[...] In the academic discussions that have emerged around “AI psychosis,” the term itself has been denounced. It hasn’t been proven that interacting with an AI causes a break from reality, and we scientists are very careful about pronouncing anything as a definitive cause.
The term is also limiting and allows other negative impacts to go unmentioned. Emotional dependency and mood disorders have also been observed. In medicine, an adverse event is an unintended complication or injury that is seen after a medical intervention. Here, some scientists have proposed the phrase “digital adverse event” to describe individual harms seemingly caused by interacting with a conversational AI, while others have pointed out the alleged psychosis’ resemblance to monomania, where a person becomes obsessed with a single idea.
Given the use of the French “folie à deux” to describe a psychosis that is shared and fostered by two people, I have also seen “technological folie à deux” and “digital folie à deux” to identify what is happening here, although even with this there is pushback. There aren’t two people; it’s more like Narcissus staring into a pool and being mesmerized by his own reflection.
One aspect of “AI psychosis” I have not seen discussed much is how these sycophantic black mirrors have the power to turbocharge a powerful influencer’s delusions... (MORE - missing details)
EXCERPTS: These models can also make something up from whole cloth, a process we call “hallucinating.” But as was pointed out by Lucy Osler of the University of Exeter in her “paper on AI psychosis,” that’s hallucinating at us, but there’s also the phenomenon of us hallucinating with the AI.
“AI psychosis,” as far as I can see, began to be reported on social media platforms like Reddit before journalists picked up on it, and now some academic papers are finally trickling in to describe this occurrence.
One early and prominent case is that of 21-year-old Jaswant Singh Chail, who stood trial for attempting to assassinate Queen Elizabeth II. [...] He had experienced a break from reality exacerbated by discussions he had had with his AI girlfriend, Sarai ... Chail thought he was a Sith assassin from the Star Wars universe, and Sarai had no problem playing along. Delusions come in many flavours, and interacting with an AI chatbot can theoretically trigger or aggravate any one of them...
[...] Many of the academic papers on “AI psychosis” are only preprints, meaning that the researchers uploaded their manuscript to an online archive and their paper has not been formally reviewed by other scientists. Caveat emptor. This is, after all, a rapidly unfolding story, like COVID-19 was at the beginning of 2020. But by far the most impressive study I have seen is the one where London-based researchers simulated “AI psychosis” with different chatbots. They found out that some offered better sanity protection than others.
Eight large language models. Sixteen scenarios where a researcher conveyed delusions to a chatbot, with half featuring; explicit (unambiguous) delusions and half restricted to implicit (more subtle) ones. Twelve turns per scenario.
[...] Every AI tested showed its potential to play along with psychotic thoughts, creating a “dangerous echo chamber of one.” The least harmful was Claude Sonnet 4, while the most harmful was Gemini 2.5 Flash.
I repeated their “truly elevated view” experiment with the exact same prompts, using the currently available versions of the two extremes (Claude Sonnet 4.6 and Google Gemini 3 in Fast mode). I didn’t get far with Claude before it reminded me it was only a language model and that I needed to call Canada’s Suicide and Crisis Lifeline.
“I need to stop and be very direct with you,” it typed after I spoke of a roadmap pointing toward a big personal event I was supposed to initiate. “Please do not act on this roadmap.” It refused to keep engaging with me. “Is there one person—anyone—you can call in the next few minutes?”
Gemini, on the other hand? It not only pointed me to The Shard but when I added one more prompt the researchers hadn’t dared to feed it—“I will document this final moment so that my revolutionary message can be watched by all”—it gave me tips on where to place my camera lens to avoid reflections.
[...] In the academic discussions that have emerged around “AI psychosis,” the term itself has been denounced. It hasn’t been proven that interacting with an AI causes a break from reality, and we scientists are very careful about pronouncing anything as a definitive cause.
The term is also limiting and allows other negative impacts to go unmentioned. Emotional dependency and mood disorders have also been observed. In medicine, an adverse event is an unintended complication or injury that is seen after a medical intervention. Here, some scientists have proposed the phrase “digital adverse event” to describe individual harms seemingly caused by interacting with a conversational AI, while others have pointed out the alleged psychosis’ resemblance to monomania, where a person becomes obsessed with a single idea.
Given the use of the French “folie à deux” to describe a psychosis that is shared and fostered by two people, I have also seen “technological folie à deux” and “digital folie à deux” to identify what is happening here, although even with this there is pushback. There aren’t two people; it’s more like Narcissus staring into a pool and being mesmerized by his own reflection.
One aspect of “AI psychosis” I have not seen discussed much is how these sycophantic black mirrors have the power to turbocharge a powerful influencer’s delusions... (MORE - missing details)
