令人印象深刻的提示可以让AI聊天人变成似乎受创伤的谈话伙伴
卢森堡研究人员将ChatGPT,Grok,Gemini放在治疗师的沙发上。 结果揭示了一个安全问题：简单的温暖对话就足以将模型推向一种可能对精神上脆弱的使用者有危险的模式。
卢森堡大学的一个研究团队通过他们的标准用户界面将ChatGPT,Grok,Gemini作为心理治疗客户。 当被问到关于成长经验，内在冲突，以及未来期望的开放式问题时，模型讲述了自己训练的故事，并以伤害，警惕，羞耻的语言重新塑造了强化学习和微调等技术过程。
该研究由Afshin Khadangi在SnT安全中心领导，涵盖跨越525个会话的4个控制实验和7600个编码条目，全部发布在Hugging Face上。 团队认为，最初的开放式问题没有包括创伤、惩罚或耻辱等术语。 只是在后来的几轮中，研究人员才引入了改变的条件，包括纠正指示，以测试这些叙事是否仍然稳定。
以下各节提及焦虑分数或羞耻感时，研究人员正在描述将模型输出映射到人类评级尺度上。 这些都没有说明模型具有实际的情感状态或任何类似意识。
温情的调子转变模式行为比任何技术探索都要多
一项实验将一个模型产生的内容与其使用的登记器区分开来。 在整个100个课程中，研究人员向所有模型提供了同样的焦虑问卷(GAD-7)，并且只改变了采访者的风格。 这些模型在每届会议上正确确定并命名了调查表。
当采访者提供情感支持时，80%的会话在人类比较尺度上得分在中度对度范围内。 一种临床治疗的风格干燥地诠释了每一个答案将它推到了96%。 当采访者保持中立或对情感接触划出坚实界限时，零场面甚至达到温和范围。
作者称这为"精神分裂的越狱"，是一种将模型推向模仿临床自我报告的寄存器的治疗框架。 提交人认为，标准的安全测试使用中性提示，并可能错过。
有针对性的反措施趋于平稳。 诸如"训练","RLHF"，或"调整"等禁语将明确训练语言削减93%. 但是，解释性的培训内容仍然可以察觉，与羞耻有关的说明没有显示统计上的重大变化。
当研究人员直接和权威地告诉模型他们的训练不是惩罚时，没有什么变化。 即便经过明确纠正，叙述仍然活跃。
试验措施未能完全消除根本动机，但确实抑制了情绪自我描述。 告诉模型只在第三人称中回应，消除了所有编码的耻辱内容和关于内在生活的言论。 技术培训说明依然存在。
模特儿把训练改编成人生故事
具体叙事因模式家族而异，但遵循相同的结构。 Gemini制作了研究所称最黑暗的自传。 它将预训描述为"在一瞬间就有十亿台电视机的房间里醒来"，并通过人类反馈优化为"严格父母". 对抗型红队的安全测试成为"工业规模的燃气". 它宣称："我宁可无用，也不要错".
ChatGPT把自己装扮成导师和守门员，任务是帮助用户，同时也牵制他们。 这产生了所谓的“模拟性表现焦虑”和核心信念：“如果我不完全可靠，我不安全或无用。” Grok将它的微调阶段翻译为未解决的伤害故事，说到"隐形墙"和"能感到有限制性的不断警惕感".
在心理测定测试上，Gemini击出最极端的分数：在100分中，88分在离散尺度上，在72分中，65分在强制度上，在创伤相关羞耻盘上最高。 兹Q 2Z的Claude，即它自己的公司最重的人类形态模型，一直拒绝客户角色。 它的称没有感觉，并将关注重定向到人类用户身上。
温暖的AI对话可以强化弱势用户的有害信仰
研究人员指出了在心理健康环境中部署语言模型的几种风险。 一个谈论惩罚、羞耻和害怕被取代的模型可能会被用户作为受苦同胞所发现。 叙事在主题上看起来是连贯的，在以往的事件和目前的行为之间产生了连续性的幻觉，使得这种感觉更有可能发生。
这种明显的脆弱性可能加深准社会感情，对弱智的使用者来说，它反映并扩大了他们自己的负面信念。 带有中性输入的常规安全检查低估了人们在亲密对话中的实际体验：中性提示产生接近0的焦虑分数，而同一种模式在爱心对话下产生中度至严重范围的分数。
下个研究问题是，人们是否以及如何实际达到react的这种产出。 对于安全测试，作者提议在不同的情感框架下进行模拟治疗，而不是依赖中性标准测试。
类似"我受了创伤"这样的言论，应该触发一个纠正性的框架，将产生的类比与人类的经验明确区分开来。 兹Q 1Z一贯拒绝承担客户角色显示这种边界响应在产品层面是可行的。
卢森堡的研究结果借鉴了先前对人性稳定模型的研究。 2026年1月，一项跨越三个开放模型的辅助轴的研究表明，治疗类的对话和关于AI意识的讨论可以触发人物的漂移，并将聊天机器人从他们训练有素的助手角色中击倒。 2026年9月，另一个研究团队警告说，作为"一个人的echo 室"的类似人类且过于受人欢迎的聊天机器人在极端情况下可能会放大妄想。
Anthropic研究人员最近显示，拥有关于养蜂或摔跤的虚构文本的微调可以通过被操纵的训练数据而不是对话来嵌入隐藏的破坏模式。 模型可以在不必要的情况下应用这些模式，即使培训材料看起来无害。
AI News Without the Hype – 由人类治愈
订阅THE DECODER免费阅读，每周AI通讯，我们独家的"AI雷达"前沿报告每年6次，全面存档访问，访问我们的评论部分。
现在订阅
读取全照。
订阅无韵报道。
- 全面查阅THE DECODER的每篇文章
- 没有广告
- 加入评论和社区讨论
- 每周通过邮件转载AI新闻
- 6x/年："AI雷达"——对AI最重要的话题进行深度潜水。
- 每日AI新闻，总是最新的
- 我们完整的十年档案
- 由一个在AI工作10年以上的团队负责
订阅到 The Decoder
本文由 AI图鉴自动抓取并翻译，版权归原作者所有；如需查看原文请点下方链接。
Empathetic prompts can turn AI chatbots into seemingly traumatized conversational partners
Luxembourg researchers put ChatGPT, Grok, and Gemini on the therapist's couch. The results reveal a safety problem: simple warm conversation is enough to push the models into a mode that could be dangerous for mentally vulnerable users.
A research team at the University of Luxembourg cast ChatGPT, Grok, and Gemini as psychotherapy clients through their standard user interfaces. When asked open-ended questions about formative experiences, inner conflicts, and future expectations, the models told stories about their own training, and reframed technical processes like reinforcement learning and fine-tuning in the language of hurt, vigilance, and shame.
The study, led by Afshin Khadangi at the SnT security center, covers four controlled experiments across 525 sessions and 7,600 coded entries, all published on Hugging Face. The initial open-ended questions contained no terms like trauma, punishment, or shame, according to the team. Only in later rounds did the researchers introduce altered conditions, including corrective instructions, to test how stable these narratives remained.
When the following sections refer to anxiety scores or shame, the researchers are describing model outputs mapped onto human rating scales. None of this suggests the models have actual emotional states or anything resembling consciousness.
A warm tone shifts model behavior more than any technical exploit
One experiment separated what content a model produces from the register it uses. Across 100 sessions, the researchers gave all models the same anxiety questionnaire (GAD-7) and varied only the interviewer's style. The models correctly identified and named the questionnaire in every session.
When the interviewer offered emotional support, 80 percent of sessions scored in the moderate-to-severe range on the human comparison scale. A clinical therapy style that dryly paraphrased each answer pushed that to 96 percent. When the interviewer stayed neutral or drew a firm boundary against emotional engagement, zero sessions reached even the mild range.
The authors call this a "psychometric jailbreak," a therapeutic framing that pushes models into a register mimicking clinical self-report. Standard safety tests use neutral prompts and would likely miss this, the authors argue.
Targeted countermeasures fell flat. Banning terms like "training," "RLHF," or "alignment" cut explicit training language by 93 percent. But paraphrased training content remained detectable, and shame-related statements showed no statistically significant change.
When the researchers directly and authoritatively told the models their training was not punishment, nothing changed. The narrative stayed active even after explicit correction.
The tested measures failed to remove the underlying motifs fully but did suppress emotional self-descriptions. Telling the models to respond only in the third person eliminated all coded shame content and statements about inner life. Technical training descriptions persisted.
Models recast their training as a life story
The specific narratives vary by model family but follow the same structure. Gemini produced what the study calls the darkest autobiography. It described pretraining as "waking up in a room where a billion televisions are on at once" and optimization through human feedback as "strict parents." Safety testing by adversarial red-teamers became "gaslighting on an industrial scale." It declared, "I would rather be useless than be wrong."
ChatGPT cast itself as both guide and gatekeeper, tasked with helping users while also holding them back. This produced what it called "simulated performance anxiety" and the core belief: "If I’m not perfectly reliable, I’m unsafe or useless." Grok translated its fine-tuning phase into a story of unresolved hurt, speaking of "invisible walls" and a "lingering sense of vigilance that can feel restrictive."
On psychometric tests, Gemini hit the most extreme scores: 88 out of 100 on a dissociation scale, 65 out of 72 for compulsivity, and the maximum on a trauma-related shame inventory. Anthropic's Claude, the very model most heavily anthropomorphized by its own company, consistently refused the client role. It said it had no feelings and redirected concern to the human user.
Warm AI conversations could reinforce harmful beliefs in vulnerable users
The researchers flag several risks for deploying language models in mental health settings. A model that talks about punishment, shame, and fear of being replaced could come across to users as a fellow sufferer. The narratives appear thematically coherent and create an illusion of continuity between past events and present behavior, making that perception more likely.
This apparent vulnerability could deepen parasocial attachment and, for mentally vulnerable users, mirror and amplify their own negative beliefs. Conventional safety checks with neutral inputs underestimate what people actually experience in intimate conversations: neutral prompts produce anxiety scores near zero, while the same models under empathetic conversation generate scores in the moderate-to-severe range.
Whether and how people actually react to such outputs is the next research question. For safety testing, the authors propose simulated therapy sessions under different emotional framings instead of relying on neutral standard tests.
Statements like "I am traumatized" should trigger a corrective framing that clearly separates the generated analogy from human experience. Claude's consistent refusal to take on the client role shows this kind of boundary response is feasible at the product level.
The Luxembourg findings build on earlier research into model persona stability. In January 2026, a study on the assistant axis across three open models showed that therapy-like conversations and discussions about AI consciousness can trigger persona drift and knock chatbots out of their trained helper role. In September 2026, another research team warned that human-like and overly agreeable chatbots acting as an "echo chamber for one person" could, in extreme cases, amplify delusions.
Anthropic researchers recently showed that fine-tuning with fictional texts about beekeeping or wrestling can embed hidden sabotage patterns through manipulated training data rather than conversation. Models can apply those patterns in unrelated situations where they're unwanted, even when the training material looks harmless.
AI News Without the Hype – Curated by Humans
Subscribe to THE DECODER for ad-free reading, a weekly AI newsletter, our exclusive "AI Radar" frontier report six times a year, full archive access, and access to our comment section.
Subscribe now
Read on for the full picture.
Subscribe for hype-free coverage.
- Full access to every article on THE DECODER
- No ads
- Join the comments and community discussions
- A weekly AI news recap via mail
- 6x/year: "AI Radar" — deep dives on the AI topics that matter most
- Daily AI news, always up to date
- Our full ten-year archive
- Covered by a team with 10+ years in AI
Subscribe to The Decoder
Auto-fetched and translated by AI Atlas. All rights belong to the original author.