我参考了我的人工智能晶体球，问问我的首席联赛幻想队
几周前，我部署了一个小群的AI agents 来编制和管理一个实验性名册。 鉴于AI agents扩散到我们生活的几乎每个方面，我想知道一个集群的bots是否能组建一支获胜的队伍。
我首先研究了几年前在这里使用AI的潜力，它的建议非常荒谬——它没有包括现实世界的信息，比如玩家受伤或团队管理层的改变。
这一次，在去探亲的航班上，我向Codex求助。 首先，我给模型提供了一些相关的研究论文。 然后，我要求它建立一个通过各种步骤指导不同agents的指南和代码，检查最近的消息，并运行一个预测模型。
它很快形成了一个设置，似乎可以整理和分析更多的信息比我曾经有过。 但我对最初的团队建议多么愚蠢感到惊讶。 把埃林·哈兰留在凳子上？真的吗，Codex?
然后，我要求Codex建立一个团队，由agents做不同的事情：几个人阅读新闻和博客文章，其他人分析最近的结果，这似乎大大改进了建议。 我试图把agents增加到超过100个
然而，几年前的改善是惊人的。 Haifeng Xu是芝加哥大学计算机科学家，他研究AI在市场状况中的应用
徐告诉我
去年末，徐对卡尔希和保利市场等预测市场越来越受欢迎感到惊讶。 他想知道这些市场的最佳商业模式可能如何，因此他与几位同事一起开始测试他们的预测技能。
他们有模型猜测，例如谁将赢得NBA锦标赛冠军，谁与特朗普总统会面。
徐告诉我，“很快，我们开始意识到模型做得非常好，这些模型有时比一般人做得更好，没有特别的指示或指导，他称。 在测试中，GPT-5最擅长于预测，其次是Grok 4,Claude Sonnet,Gemini和Llama.
团队接着开发了先知竞技场，这是一个测量模型预测能力的基准测试。 也是哈佛大学研究人员和一家名为Arcada Labs的公司，最近给了前沿模型10 000美元，这些模型在实验中不断损失金钱，但研究人员看到这些模型在一段时间内不断改进的证据。
指出拥有独特的任务——阅读新闻、收集其他数据、分析整个系统
有趣的是，AI仍然经常落后于被称为超级forecasters的非常熟练的专家。 徐解释说，虽然人类无法消耗同样多的信息，但他们对现实世界——即新闻背后的动态——的理解却更为深刻。
这称，越来越多的证据表明，人类超级预测器的边缘正在侵蚀。 最近一场名为Metaculus杯的比赛显示，AI有时甚至可以击败最好的人类预测者。 而一个称为"预测奔驰"的基准测试可以提供进一步证据，证明当持续比赛的结果很快结束时AI可以匹配人类技能。 事实上，我认为我们可以期望大实验室将注意力转向未来探索作为一种强大的新方法来测量机器智能的进步。
所有这些预测性实验似乎都表明，像数学、编码和法学一样，基于贸易的行业可能很快被大赦国际改进。 也许最好的人贩子将会承担一种新的角色：监督和组织AI agents，能够远比他们更能支配未来。
目前，过去几周，我当然不会在自己的选择中使用它的任何建议。
但AI无疑会好起来的，也许让它太容易主宰幻想联盟很快。 但我希望不会，因为这会破坏通过自己赢得一切来赢得吹嘘权的乐趣。
这是威尔·奈特的AI Lab通讯的版本。 在这里阅读之前的通讯。
本文由 AI图鉴自动抓取并翻译，版权归原作者所有；如需查看原文请点下方链接。
I have consulted my AI-powered crystal ball, and it’s telling me that frontier models are going to get a lot better at making predictions in the near future. Just ask my Premier League fantasy team.
A few weeks ago, I deployed a small horde of AI agents to craft and manage an experimental roster. Given the proliferation of AI agents into pretty much every aspect of our lives, I wanted to know if a swarm of bots could put together a winning team.
I first looked at the potential for AI here a few years ago, playing around with a tool from the UK’s Turing Institute that selected teams after modeling previous fantasy league data. Its suggestions were remarkably bonkers—it didn’t incorporate real-world information, like a player’s injury or a team’s change in management.
This time, while on a flight to visit family, I asked Codex for help. First, I fed the model a couple of relevant research papers. Then, I asked it to build a harness—instructions and code that guides different agents through various steps—that would check recent news and run a model to make predictions.
It quickly came up with a setup that seemed to collate and analyze way more information than I ever would have. And yet I was surprised by how dumb the initial team suggestions were. Leaving Erling Haaland on the bench? Really, Codex?
I then asked Codex to create a team of agents that did different things: several that read news and blog posts; others that analyzed recent results; more that ran models; and a bunch that critiqued suggestions. This seemed to significantly improve the suggestions. I tried increasing the number of agents to more than 100, hoping this would make the system smarter still, but it didn’t change much besides running up my API bill.
Still, the improvement from a few years ago was striking. According to Haifeng Xu, a computer scientist at the University of Chicago who studies the uses of AI in market situations, newer models may be even better at making predictions because they’re able to consume and analyze huge amounts of real-world information, and they’re able to orchestrate more complex systems for their guesses.
“AI is just getting smarter in understanding the world—I feel like that's the fundamental reason they’re improving,” Xu tells me.
Late last year, Xu was marveling at the growing popularity of prediction markets like Kalshi and Polymarket. He wondered how the best commercial models might fare on those markets, so alongside several colleagues, he began testing their forecasting skills.
They had models guess, for example, who would win the NBA Championship and who President Trump would meet with.
“Very quickly we started to realize that models did really well,” Xu tells me. The models were sometimes able to do better than the average person without any special instructions or guidance, he says. In their testing, GPT-5 was the best at making predictions, followed by Grok 4, Claude Sonnet, Gemini, and Llama.
The team went on to develop Prophet Arena, a benchmark that measures a model’s ability to make predictions. It’s one of a number of benchmarks designed to test AI’s future-gazing capabilities; another, Prediction Arena, from researchers at Harvard University and a company called Arcada Labs, recently gave frontier models $10,000 to wager on prediction markets over 57 days. The models consistently lost money in their experiment, but the researchers saw evidence of them improving consistently over time.
Xu tells me that AI can most effectively make predictions using a custom harness, and notes that having agents with unique tasks—reading the news, collecting other data, and analyzing the whole system, for instance—seems to be the most effective approach.
Interestingly, AI still often lags behind very skilled experts known as superforecasters. Xu explains that, although humans cannot consume as much information, their understanding of the real world—that is, of the dynamics behind a news story—is deeper.
That said, there’s growing evidence that the human superforecaster’s edge is eroding. A contest called the Metaculus Cup recently showed that AI can sometimes beat even the best human predictors. And a benchmark called ForecastBench could provide further evidence that AI can match human skills when the results of an ongoing contest conclude soon. In fact, I think we can expect the big labs to turn their attention to future-gazing as a powerful new way to measure progress in machine intelligence.
All of these predictive experiments seem to suggest that, like math, coding, and law, industries predicated on trading may soon be upended by AI. Perhaps the best human traders will take on a new kind of role: supervising and orchestrating AI agents capable of divining the future far better than they can.
For the time being, my AI’s suggestions still don’t seem all that impressive. Its team hasn’t outscored the average over the past weeks. I certainly won’t be using any of its suggestions in my own selections.
But AI will no doubt get better, perhaps making it all too easy to dominate fantasy league pretty soon. I’m kind of hoping it won’t, though, as it would spoil the fun of trying to earn bragging rights by winning all by yourself.
This is an edition of Will Knight’s AI Lab newsletter. Read previous newsletters here.
Auto-fetched and translated by AI Atlas. All rights belong to the original author.