11月4日,Anthropic旗下Claude Code团队工程师、前Google技术高管Addy Osmani近日在社交平台X发表长文《当AI智能体完成工作时,如何保持批判性思维》(Critical thinking when agents do the work),警告随着AI智能体(AI Agent)越来越强大,人类可能在享受效率提升的同时,逐渐丧失独立思考、验证结果和理解复杂系统的能力。他将这种现象称为“认知投降”(Cognitive Surrender)。
Osmani曾在Google工作超过14年,负责过Chrome开发者体验及Google云AI相关业务,目前在Anthropic从事Claude Code相关工作。
Osmani表示,自己是AI智能体工作流的坚定支持者,日常工作中经常同时运行多个Agent。但他发现一个值得警惕的变化:随着AI模型不断进步,输出结果中的明显错误越来越少,人类反而更难保持警惕。
过去,人们通常需要经历信息收集、分析、判断等过程才能得到答案,而现在,这一过程可能被压缩到几秒钟。AI生成的内容往往结构完整、表达流畅,而且大部分情况下是正确的,这使得用户越来越容易跳过独立思考和验证环节。
他将人类使用AI的状态划分为三个层次:首先是“认知卸载”(Cognitive Offloading),即将任务交给AI完成,但仍然对结果负责;其次是“推迟判断”(Deferral),即不再将AI输出视为需要验证的假设,而是直接作为下一步工作的依据;最后是“认知投降”,即AI的输出逐渐变成用户自己的判断,用户甚至不再认为有必要检查。
其中,“认知投降”这一概念最早由沃顿商学院研究人员Steven Shaw和Gideon Nave提出。
Osmani认为,这种转变往往不是突然发生的,而是在不断使用AI的过程中逐渐形成。用户可能认真阅读第一份AI报告,对第二份只进行快速浏览,到了第三份甚至只检查大纲。最终,当有人询问某个具体结论时,用户可能发现自己根本无法解释其中的逻辑。
他将这一现象与“认知债务”(Cognitive Debt)联系起来。在软件开发领域,AI可以生成大量代码,测试也可能全部通过,但团队成员对系统架构和实现逻辑的理解却在不断减少。直到某天需要修改系统时,开发者才发现已经没有人真正理解某些功能的工作原理。
不过,Osmani并不认为开发者需要记住庞大代码库的全部细节。他指出,Claude Code团队也会在需要时借助AI重新建立对特定代码模块的理解。真正重要的不是始终掌握所有知识,而是知道自己不了解什么,并且具备重新理解它的能力。
对于如何避免“认知投降”,Osmani建议,在向AI提出复杂问题前,人类应该先形成自己的初步假设,而不是完全依赖AI提供答案。即使最初的猜测并不准确,也可以帮助用户在接收AI结果时保留独立判断的基础。
他还建议,用户应当主动寻找能够推翻AI结论的证据,而不是只询问模型对自己答案有多大信心。对于存在争议的问题,可以让不同模型从相反立场提出论证,再由人类综合判断。
在软件开发领域,Osmani认为,AI时代的瓶颈正在从编写代码转向验证代码是否正确。
目前,AI智能体已经可以自主编写代码、运行测试、读取错误信息并反复修改,直到测试通过。但问题在于,测试本身也可能由AI编写。如果测试逻辑与代码存在相同错误,即使所有测试显示通过,也无法证明软件不存在问题。
因此,他强调,开发者不仅应该关注测试是否通过,还需要明确测试究竟证明了什么、没有覆盖哪些风险。对于AI交付的成果,验收标准应该是实际测试结果、运行日志、截图等可以验证的证据,而不是AI生成的一段看起来合理的总结。
Osmani还提出,开发者应该要求Agent明确列出它在执行任务过程中自主作出的决定、采用的假设以及忽略的条件,因为这些内容通常不会出现在最终的工作总结中。
随着AI能够同时执行越来越多任务,另一个问题也开始出现。Osmani将其称为“编排税”(Orchestration Tax),即AI生成成果的速度与人类能够认真审核、并愿意为其负责的速度之间的差距。
当AI生成的工作成果不断堆积,而人类已经没有足够时间审核时,继续增加Agent数量并不能解决问题。他认为,这种情况下,团队应该改进自动验证机制,或者适当降低任务生成速度。
Osmani同时警告,AI的高度自动化可能使人类逐渐放弃练习原有技能。他建议开发者每周尝试在不使用AI的情况下编写一些代码,每个月重新审视自己已经停止练习的技能,并判断哪些能力仍然值得保留。
他还引用Anthropic关于内部工程师和研究人员使用AI的研究,指出其中存在一种“监督悖论”:监督Claude需要具备编程能力,但过度使用AI又可能使这些能力退化。
Osmani强调,这些观点并不意味着应该减少AI的使用。在错误成本较低的任务中,完全交由AI完成没有问题,但涉及投资、职业选择、产品安全或影响其他人的重大决定时,人类仍应保留关键判断权。
在他看来,未来的AI模型可能比人类更擅长发现错误,但这并不能消除人类的责任。决定开发什么产品、为什么值得开发、会影响哪些人,以及愿意为此承担何种代价,仍然需要人类作出选择。
他在文章结尾总结道:“你可以将工作和大部分审核任务交给AI,但不能将责任也一并交出去。”
本文由 AI图鉴自动抓取并翻译,版权归原作者所有;如需查看原文请点下方链接。
On November 4th, the late Addy Osmani, former Google tech executive, published on social platform X, Creative Thinking when agents do the work, warning that, as AIAI agent becomes stronger, humans may gradually lose the ability to think independently, validate results and understand complex systems while enjoying increased efficiency. He referred to this phenomenon as “cognition surrender”.
Osmani has worked in Google for more than 14 years, has been in charge of the Chrome Developer Experience and Google CloudAi, and is currently working on Claude Code in Anthropic.
Osmani stated that he was a strong supporter of AIAI agentworkflow and that he often operated multiple Agents in his daily work. But he found an alarming change: as the AI model continues to advance, there are fewer obvious errors in output, and it is more difficult for humanity to be vigilant.
In the past, people usually needed to go through a process of information gathering, analysis, judgement, etc. to get answers, which could now be constricted to a few seconds. The content generated by AI tends to be structured, well articulated and, in most cases, correct, making it increasingly easy for users to skip independent thinking and validation.
He divided the state of human use of AI into three levels: first, Cognitive Offloading, which gave AI the task, but remained responsible for the results; second, Deferral, which no longer viewed AI output as a presumption requiring validation, but as a direct basis for next step work; and lastly, Cognitive Surrender, where the output of AI gradually became the user's own judgement and the user no longer considered it necessary to examine it.
Among them, the concept of “cognitive surrender” was first proposed by Steven Shaw and Gidion Nave, researchers at the Walton Business School.
In Osmani ' s view, such a shift is often not sudden but is taking place in the course of the ongoing use of AI. Users may carefully read the first AI report, browsing the second and even checking the outline for the third. Ultimately, when a specific conclusion is asked, the user may find itself unable to explain the logic.
He linked this phenomenon to Cognitive Debt. In the area of software development, AI can generate a large number of codes and may pass all tests, but team members ' understanding of system architecture and logic is decreasing. It was not until one day that the system needed to be modified that the developers found that no one really understood the working principles of certain functions.
Osmani does not, however, believe that developers need to remember the full details of the large code library. He noted that the Claude Code team would also re-establish its understanding of specific code modules with AI as needed. What is really important is not to always have all the knowledge, but to know what it does not understand and to have the capacity to understand it again.
On how to avoid “cognitive surrender”, Osmani suggested that human beings should develop their own initial assumptions before asking AI complex questions, rather than relying entirely on AI to provide answers. Even if initial speculation is not accurate, it helps users to retain the basis for independent judgement when receiving AI results.
He also suggested that users should take the initiative to find evidence that could reverse the AI conclusion, rather than simply asking how confident the model was in its own answers. In the case of controversial issues, different models can be justified from opposite positions before humans are judged together.
In the area of software development, Osmani believes that the bottlenecks of the AI era are shifting from code writing to validation.
At present, AIAI agent has been able to write code, run tests, read error messages and change them repeatedly until the test is passed. The problem, however, is that the test itself could also be prepared by AI. If there is the same error in testing logic as the code, even if all tests are shown to be passed, it is not possible to prove that the software is free of problems.
He stressed, therefore, that developers should not only be concerned about whether the tests were passed, but also clearly about what the tests proved and what risks were not covered. For AI deliverables, acceptance criteria should be verifiable evidence of actual test results, operational logs, cut-off charts, etc., rather than an apparently reasonable summary of AI generation.
Osmani also suggested that developers should ask Agent to clearly set out its own decisions, assumptions and conditions of neglect in the discharge of its mandate, as they would not normally appear in the final summary of work.
As AI was able to carry out more and more tasks simultaneously, another problem began to emerge. Osmani describes it as the “Organization Tax”, the gap between the speed at which AI produces results and the speed at which humans can carefully examine and be held accountable.
A continued increase in the number of Agents will not solve the problem when AI generates cumulative work and human beings do not have enough time to review it. In his view, in such cases, the team should improve the automated validation mechanism or appropriately reduce the speed of task generation.
Osmani also warned that the high level of automation of AI could lead to humans gradually abandoning their original skills. He suggested that developers try to develop codes on a weekly basis without using AI, revisit their skills once a month and determine which competencies were still worth retaining.
He also quoted Anthropic's study on the use of AI by in-house engineers and researchers, pointing out that there was a “oversight paradox” in it: monitoring Claude required programming capabilities, but over-use of AI could degrade those capabilities.
Osmani stressed that these views did not mean that the use of AI should be reduced. While there is no problem leaving it entirely to AI to perform tasks that are wrongly costly, human beings should retain critical judgement when it comes to investments, career choices, product safety or major decisions that affect others.
In his view, the future AI model might be better at detecting mistakes than humans, but it did not eliminate human responsibility. The choice of what to develop, why to develop, who to affect, and at what cost to be willing to do so still remains for humanity to make.
At the end of the article, he concluded: “You can give AI the job and most of the clearance tasks, but you cannot hand over responsibility together.”
Auto-fetched and translated by AI Atlas. All rights belong to the original author.