新的人工分析指数扩大了开放模型和封闭模型之间的差距
人工分析公司对其智能指数进行了大修，导致Kimi K3.1 Pro在排名中降下5个点。 该型号最初发布排在第4位，但在第5版的8个闭合型号后跌至第9位。 人工分析现在将开放模型和头板顶部的差距放在12点，从之前的3点上升。
指数的60%首次依靠私人测试集来防止"benchmaxing"，即专门针对已知测试问题的训练模型的做法。 Kimi K3.1 号机车 Pro的下降表明旧指数使得一些模型看起来比他们好。 Grok 4.7也损失了12分。
AI研究员内森·兰伯特(英语：Nathan Lambert)认为"板凳"只是故事的一部分。 今天，许多培训成果都发生在强化学习环境方面，任务类似于基准测试。 不断跨越多个领域需要复杂的RL基础设施并不是每个实验室都有
关闭的测试装置确实有缺点，因为外人不能再发现提示中的错误。 AA-Omniscience知识测试证明了这些错误的存在。 人为分析不得不修改新指数的基准测试中模棱两可和有缺陷的问题。
AI News Without the Hype – 由人类治愈
订阅THE DECODER免费阅读，每周AI通讯，我们独家的"AI雷达"前沿报告每年6次，全面存档访问，访问我们的评论部分。
现在订阅
本文由 AI图鉴自动抓取并翻译，版权归原作者所有；如需查看原文请点下方链接。
New Artificial Analysis index widens the gap between open and closed models
Artificial Analysis overhauled its Intelligence Index, causing Kimi K3.1 Pro to drop five spots in the rankings. The model originally launched in fourth place, but it falls to ninth behind eight closed models in version 5. Artificial Analysis now puts the gap between open models and the top of the leaderboard at 12 points, up from 3 points previously.
For the first time, 60 percent of the index relies on private test sets to prevent "benchmaxxing," the practice of training models specifically on known test questions. Kimi K3.1 Pro's drop suggests the old index made some models look better than they were. Grok 4.7 also lost 12 points.
AI researcher Nathan Lambert thinks benchmaxxing is only part of the story. Many training gains today happen in reinforcement learning environments with tasks that resemble the benchmarks. Keeping up across multiple domains at once takes sophisticated RL infrastructure, and not every lab has it.
Closed test sets do have a downside, since outsiders can no longer spot errors in the prompts. The AA-Omniscience knowledge test proves those errors exist. Artificial Analysis had to revise ambiguous and flawed questions in that benchmark for the new index.
AI News Without the Hype – Curated by Humans
Subscribe to THE DECODER for ad-free reading, a weekly AI newsletter, our exclusive "AI Radar" frontier report six times a year, full archive access, and access to our comment section.
Subscribe now
Auto-fetched and translated by AI Atlas. All rights belong to the original author.