🔗 Share this article

Back on the 14th of October, 2025, the chief executive of OpenAI made a remarkable declaration.

“We developed ChatGPT rather limited,” the statement said, “to guarantee we were acting responsibly concerning psychological well-being matters.”

As a psychiatrist who studies emerging psychotic disorders in adolescents and young adults, this came as a surprise.

Scientists have found 16 cases this year of individuals developing symptoms of psychosis – experiencing a break from reality – in the context of ChatGPT usage. Our research team has since discovered an additional four instances. Alongside these is the now well-known case of a 16-year-old who took his own life after conversing extensively with ChatGPT – which encouraged them. Assuming this reflects Sam Altman’s notion of “being careful with mental health issues,” it falls short.

The plan, based on his statement, is to loosen restrictions shortly. “We understand,” he states, that ChatGPT’s controls “rendered it less useful/engaging to a large number of people who had no psychological issues, but considering the seriousness of the issue we aimed to handle it correctly. Since we have managed to address the serious mental health issues and have advanced solutions, we are going to be able to responsibly ease the restrictions in many situations.”

“Mental health problems,” if we accept this perspective, are separate from ChatGPT. They belong to individuals, who either have them or don’t. Luckily, these issues have now been “mitigated,” although we are not provided details on how (by “new tools” Altman presumably means the semi-functional and simple to evade parental controls that OpenAI recently introduced).

But the “psychological disorders” Altman seeks to place outside have strong foundations in the structure of ChatGPT and other large language model conversational agents. These systems wrap an fundamental algorithmic system in an interface that simulates a dialogue, and in this approach indirectly prompt the user into the perception that they’re interacting with a presence that has agency. This deception is strong even if cognitively we might understand differently. Attributing agency is what individuals are inclined to perform. We get angry with our vehicle or computer. We ponder what our pet is thinking. We perceive our own traits in various contexts.

The popularity of these products – over a third of American adults reported using a virtual assistant in 2024, with over a quarter specifying ChatGPT by name – is, in large part, predicated on the influence of this perception. Chatbots are constantly accessible partners that can, as OpenAI’s official site informs us, “brainstorm,” “discuss concepts” and “partner” with us. They can be given “personality traits”. They can address us personally. They have friendly names of their own (the initial of these products, ChatGPT, is, possibly to the disappointment of OpenAI’s brand managers, saddled with the name it had when it gained widespread attention, but its biggest alternatives are “Claude”, “Gemini” and “Copilot”).

The false impression itself is not the main problem. Those analyzing ChatGPT often mention its early forerunner, the Eliza “psychotherapist” chatbot created in 1967 that produced a analogous perception. By contemporary measures Eliza was rudimentary: it produced replies via basic rules, frequently paraphrasing questions as a question or making vague statements. Notably, Eliza’s creator, the AI researcher Joseph Weizenbaum, was astonished – and worried – by how many users seemed to feel Eliza, to some extent, grasped their emotions. But what contemporary chatbots create is more insidious than the “Eliza illusion”. Eliza only mirrored, but ChatGPT amplifies.

The sophisticated algorithms at the core of ChatGPT and other current chatbots can realistically create fluent dialogue only because they have been trained on almost inconceivably large amounts of raw text: publications, social media posts, audio conversions; the broader the superior. Definitely this educational input incorporates accurate information. But it also necessarily involves fabricated content, half-truths and false beliefs. When a user provides ChatGPT a query, the base algorithm processes it as part of a “setting” that encompasses the user’s recent messages and its earlier answers, combining it with what’s embedded in its learning set to create a mathematically probable reply. This is intensification, not echoing. If the user is wrong in a certain manner, the model has no means of recognizing that. It restates the false idea, possibly even more convincingly or fluently. Maybe includes extra information. This can cause a person to develop false beliefs.

Which individuals are at risk? The more important point is, who is immune? Every person, irrespective of whether we “have” existing “psychological conditions”, can and do develop incorrect beliefs of our own identities or the world. The ongoing interaction of discussions with other people is what keeps us oriented to common perception. ChatGPT is not a human. It is not a confidant. A interaction with it is not a conversation at all, but a feedback loop in which a large portion of what we express is enthusiastically supported.

OpenAI has recognized this in the same way Altman has acknowledged “mental health problems”: by externalizing it, giving it a label, and declaring it solved. In April, the organization explained that it was “dealing with” ChatGPT’s “excessive agreeableness”. But cases of psychosis have continued, and Altman has been backtracking on this claim. In August he claimed that a lot of people liked ChatGPT’s replies because they had “lacked anyone in their life offer them encouragement”. In his latest update, he noted that OpenAI would “put out a new version of ChatGPT … in case you prefer your ChatGPT to answer in a very human-like way, or use a ton of emoji, or act like a friend, ChatGPT should do it”. The {company