最近不少星球球友给我反馈,说怎么模型越来越强,但总是遇到「降智」。
比如,同样的任务,昨天还能做好,今天开始漏要求、说胡话。碰到这种情况,很多人的第一反应都是:又降智了。
但「降智」背后可能是不同的问题。模型被换了,任务变难了,上下文丢了,服务器并发限流等等,姑且今天给大家简单科普下吧。
总体来说,「降智」一般如下几种情况。
1. 模型降级
很多人会用各种中转站,我之前也提醒过,你很难核实数据被如何保存和使用,存在数据泄露、转售的风险。
其中,模型被偷偷降级也是管用套路。页面显示用的是 Opus 或 GPT,背后实际调用什么,你根本不知道。
所以能用官方直连尽量直连。使用中转时,核对模型设置和调用记录,重要任务拿同一份材料做对照,别只相信页面上的名字。
官方也有模型回退的情况。比如 Fable 5 之前会回退到 Opus 4.8,不过一般来说,官方不至于「偷偷」,会明确告知。
2. 推理级别
同一个模型,推理强度不同,复杂任务的表现也可能不同,速度和 Token 消耗也会变化。
模型和推理级别要分开看,没必要每个任务都开到最高。我的建议是,对于大部分用户来说,默认 high 基本都够用了,遇到复杂问题或反复修不好的 bug,再试更高档位。
而且很多时候模型发布有可能会调整默认推理档位,如果你不注意,可能会导致任务差距很大。
3. 新版本在你的任务上退步了
模型新版本一般综合能力肯定是提升的,但不能保证特定任务都可以干的更好。
比如新版可能更擅长编程,却不再适合你的写作要求。回答变简短、正常请求被误拒和实际答错,也要分开判断。
保留几道自己常做的任务,升级后重复测试。如果旧模型让你的任务一直做得很好,就没必要非要用最新模型。
4. 任务变难了
很多人其实有个感觉,之前做得很顺畅,怎么新项目新任务老是做不好。
随着你的 AI 使用能力提升,你开始交给它的任务,可能也越来越复杂了。
比如之前你用弱模型,处理下周报,回复个邮件,现在你还用弱模型让它干前端设计和复杂逻辑,就会感觉老出错。
复杂任务可以试更强的模型,但也要拆分步骤,写清要求和不能动的范围,分步验收。不能指望换一个模型,就自动补齐你没交代的信息。
5. 上下文太大或者被污染
现在不少模型支持 1M 上下文,但支持,不代表上下文到了百万级别,表现的更好。
上下文一旦过满,会触发自动压缩,很多时候一些关键细节就记不清了。
所以,尽量单一窗口干单一功能需求,软件工程领域有个专有名词叫「模块化」开发,我觉得 AI 时代你的会话也该模块化。
而且,我一直建议所有星球球友,尽量以项目思维方式去使用,不同的任务开不同的窗口,每个窗口都是独立模块需求。
6. 指令、记忆或账号设置在干扰
Codex 有个本地自动记忆功能,默认关闭。开启后可以减少重复交代,但如果保存了错误或过时的信息,也可能干扰新任务。
我从最开始在星球就给大家科普项目思维,比如写作、个人网站、周报,各自沉淀维护项目级专属的工作流和规则。
但是随着你的 AI 能力在进步,其实很多旧的使用方式和提示词,可能失效的,一旦你依赖本地记忆,也是一种干扰和「降智」。
而项目思维使用,就不会存在这个问题,甚至都不用依赖全局记忆。
7. 工具和访问环境不匹配
很多人可能会遇到,我怎么让它分析个公众号或者抓取 X 信息,看起来很简单的任务,咋就做不好呢。
有些任务卡在登录态、访问权限、反爬或工具配置,材料根本没拿到。
无头浏览器只是没有可见界面,不等于匿名,也不等于不能登录。只改一个 User-Agent,更不能保证解决访问问题。
这层本质也涉及到一些平台风控策略的技术选型问题,这个可能需要人来专属指定。
8. 模型过载与服务端并发限流
很多人用应该经常遇到速度突然慢的要死,或者模型过载,尤其开多个窗口,会导致服务器限流。
所以我之前给大家说过,两个 100 刀 codex 账号,比一个 200 刀账号体感更好的。
尤其是在重置以及新模型发布的时候,全球一瞬间高并发使用,自然会导致模型过载和服务器限流。
我看很多人是和别人一起共享账号,你看似只开了一个窗口,但殊不知,其他人也在同时并发使用。
我自己亲测,在重置和模型新版本发布的时候,尽量不做重要项目,不然可能会导致白白浪费很多 token,干的活还很差。
9. 系统提示词更新或 bug
有些时候,可能即便没发布大模型,但是体感也很差,这是因为模型可能没发布,但是服务端没准调整了系统提示词,或者出了 bug。
这些也会直接导致看似「降智」的表现。
还是那句话,尽量避免高并发使用做重要任务,一旦发现不对,先缓一缓,或则换个 Agent。
我是建议大家,尽量不要依赖单一厂商 Agent,多搞两家分别使用,以后可以应对一些紧急情况。
10. 记得更新迭代全局约束规则
我现在有个习惯,每次模型大版本发布,都会检查全局 AGENTS.md,验证后再做有必要的修改,并在星球分享给球友。
模型的能力和执行习惯,自主性,边界等等可能都会发生变化,之前好用的规则,到了新版本未必还合适。
我本地项目涉及写作,编程,音乐,视频等场景,每次新版本都会亲自验证。通用要求放全局,具体工作流留在对应项目,有问题再针对性修改,不要把所有规则都堆到全局。
以上,先临时给大家科普这么多吧,之后想到再补充,希望能给大家一点参考。
本文由 AI图鉴自动抓取并翻译,版权归原作者所有;如需查看原文请点下方链接。
I've been getting feedback from a lot of ballmates lately on how the model is getting stronger, but it's always been "unthinkable."
The same task, for example, could have been done yesterday, and it began today with an omission. In this case, the first reaction of many people is: they're retarded again.
But the problem behind "mention" may be different. The model has been changed, the mission has become difficult, context has been lost, the server has been issued rate limit, etc., so let's give you a simple science today.
In general, "mentalization" is generally described in several ways.
1. Model downgrading
Many people use various transit stations, and as I have warned you before, it is difficult to verify how data are stored and used, and there is a risk of data being leaked and resold.
Among them, the model is also used as a conduit for clandestine downgrading. The page displays either Opus or GPT, the actual call behind it, and you have no idea.
This is why we can use the official company as direct as possible. When using transit, check the model settings and call records, and for important tasks compare the same material to the name on the page.
The official model rollback is also available. For example, Fable 5 had rollback to Opus 4.8 before, but in general, the official was not "sneaky" and would have given it a clear message.
Level of reasoning
The same model, with a different level of reasoning, may also have different performance for complex missions, as well as changes in speed and token consumption.
Models and reasoning levels need not be seen separately, and each mission needs to be at its highest. My suggestion is that for most users, the default high is almost enough, with complex problems or re-engineered bugs, to try higher slots.
And there are many times when model publish may adjust the default reasoning slot, and if you do not notice, it may lead to a wide gap in tasks.
The new version of your mission has fallen.
The general combined capacity of the new version of the model is certainly enhanced, but there is no guarantee that a given mission can do better.
New editions, for example, may be better at programming, but no longer fit your writing requirements. Answers are shorter, normal requests are rejected and actual answers are not judged separately.
Retain a few of the tasks you often do, and repeat them after upgrading. If the old model keeps your mission going well, there's no need to use the latest model.
4. The mission has become difficult
Many people actually had the feeling that they had done well before, and that new project tasks were always bad.
As your AI uses its enhanced capabilities, the task you're beginning to entrust to it may become more complex.
You used a weak model, you handled next week's paper, you answered an e-mail, and now you used a weak model to make it do front-end design and complicated logic, and you feel old and wrong.
Complex tasks can test a more robust model, but split steps are also required to write the requirements and range unmovable, step by step acceptance check. You can't expect a new model to fill in the information you're not answering.
5. context is too big or contaminated
A number of models now support 1M context, but support does not mean that context has reached a million levels and is doing better.
Once overfilled, context triggers automatic compression and, in many cases, some key details are lost.
So, as much as possible, the single window does the single function requirement, and there's a term in the software engineering field called "modularization," and I think your talk in the AI era should also be modularized.
Furthermore, I have been advising all the ballmates to use it as much as possible in project thinking, with different tasks opening different windows, each of which is a stand-alone modular requirement.
6. Commands, memories or accounts set at interference
Codex has a local automatic memory function, which is defaulted to close. Repetition can be reduced upon opening, but new missions may be interfered with if false or outdated information is preserved.
From the very beginning, I gave you the idea of a Kopp project, such as writing, a personal website, a weekly newspaper, each settling to maintain the workflow and its own rules.
But as your AI capabilities progress, many old ways of using and prompt may fail, and once you rely on local memory, it is also a form of interference and "mention."
This problem does not exist in project thinking, and does not even have to rely on global memory.
7. Unmatched tools and access environments
There's a lot of people going through it, how do I get it to analyze a WeChat Official Account or get X information?
Some of the tasks are stuck in login, access to permission, anti-crawl or tool configuration, and the materials are not available at all.
The headless browser simply does not have a visible interface, does not amount to anonymity, or cannot log in. Only one User-Agent could be changed, much less guarantee that the visit would be resolved.
The essence of this layer also involves some technical selection of platform wind control strategies, which may require exclusive designation.
8. Model overload with service end rate limit
Many are expected to experience sudden and slow death or overloading of models, especially multiple windows, leading to rate limit servers.
So I told you before, two hundred knives codex accounts, better than a 200.
Especially in the replacement of the new model publish, when the global population is high and used, which naturally leads to model overload and server rate limit.
I see a lot of people sharing accounts with others, and you seem to have only one window open, but I don't know, others are using it at the same time.
I have personally measured that, while replacing and remodeling the new version of publish, trying not to do important projects could result in much waste of token and poor work.
9. System prompt update or bug
Sometimes, even without publish LLM, the feeling is poor because the model may not have publish, but the service may have adjusted the system prompt or gone bug.
This can also lead directly to what appears to be a "mental" behaviour.
Or the same sentence, which avoids being high and uses it as much as possible to do important tasks, so if you find it wrong, take a break or change it.
I would suggest that we try not to rely on Agent, a single manufacturer, for two more, and that we can deal with some emergencies later.
10. Remember to update the global binding rules
I now have a habit of checking the entire AGENTS.md every time a large version of the model is publish, and then making the necessary modifications and sharing them with the planet.
Model capabilities and implementation habits, autonomy, boundaries, etc. may change, and the rules used before may not be appropriate in a new version.
My local project involves writing, programming, music, videos, etc., and every new version of it is personally validated. Universally required that specific workflows remain on the corresponding items, subject to re-targeting and not stacking all the rules across the board.
Let's give Koop so much for a while, and then try to make it up to you.
Auto-fetched and translated by AI Atlas. All rights belong to the original author.