新智元报道
这一天还是来了！
不用训练、不算梯度，只让GPT-6 Astra改写另一款小模型0.016%权重，数学成绩就能原地暴涨6倍！
今天，MIT计算机科学与人工智能实验室（CSAIL）带来的这项新研究，在全网彻底火了。
他们首次提出了一个颠覆传统深度学习的新范式，「权重空间编程」。
要知道，神经网络的权重文件里全是一眼望不到头的数字。
过去四十年里，没有任何人类能肉眼看懂这堆数字在算什么，想改模型行为只能喂数据、算梯度，靠训练一步步微调。
但这一次，GPT-6 Astra直接跳过了训练，光靠推理就能看懂并亲手改写底层权重。
靠着这手绝活，它不仅徒手治好了开源小模型Qwen3.5-0.8B的死循环毛病，甚至还能从零手搓出一个百万参数的围棋AI！
论文一经发布，全网破200万人围观，转发、讨论直接破圈，许多人在评论区直呼「见证历史」。
在硬核拆解「权重空间编程」之前，先划出这次MIT报告中的颠覆性重点：
- 爆改权重：只改0.016%的权重，Qwen3.5-0.8B的MATH-500从7.9%飙到48.1%，暴涨6倍，指令遵循翻3倍。
- 读懂神经网络：只看17253个数字，Astra便以99%的置信度一眼认出，四个游戏策略网络各自在玩哪款游戏——打砖块、乒乓球、吃豆人和过马路。
- 徒手敲出权重：从全零开始，一次回复手写出打砖块网络，得分517.4，是PPO苦练5亿步的20倍！
这篇由MIT博一Rikiya Takehi和团队交出的论文，彻底改变了所有人的世界观。
一夜之间，LLMs能够原生读写神经网络权重了。
最让人头皮发麻的，就是GPT-6 Astra给8亿参数的Qwen3.5-0.8B做的那场「微创脑手术」。
研究人员把共24层的Qwen3.5-0.8B放进一个只有CPU的沙盒里，让Astra在不跑训练的前提下提升它的数学成绩。
整个过程中，Astra最多只能在练习题上试跑50次，用来观察小模型做题时的内部状态。
接手任务后，Astra没有急着乱改数字，而是先拿96道数学题给小模型做了一次全面体检。
体检结果一下子暴露了病根。原版模型有85道题一头扎进了死循环，同一句车轱辘话来回念叨，直到把3.2万词元的长度上限耗光也没算出答案。
也就是说，这款小模型之所以在MATH-500上只考出7.9%的惨淡分数，是因为93%的题目都被「复读机死循环」硬生生拖死了。
既然找到了病症，下一步就是顺藤摸瓜，看看这个让人复读的开关到底藏在24层网络的哪个角落。
Astra对比了正常做题和陷入死循环时的内部信号，很快抓到了异常，只要模型开始车轱辘话复读，第15层和第19层就会顺着一个固定的方向猛发信号。
为了验证这个猜想，Astra先关掉了第19层里信号最强的第11号神经元试水，结果数学准确率立马来了一波6个百分点的上涨。
不仅如此，为了确认不是碰巧蒙对，Astra还主动给自己加了一组对照实验，在权重里减去一个同样大小的随机方向。
看到随机对照组只带来1.6个百分点的微弱波动后，Astra这才完全笃定，自己抓到的那个信号方向就是引发死循环的真凶！
锁定靶心后，它下刀稳准狠，只挑出第15层和第19层里写死循环最猛的各64列权重，精准抹掉这个复读信号。
算下来，全模型8亿参数里只改了131,072个数字（占0.016%），其余99.984%的权重分毫未动！
这记微创手术立竿见影，小模型做题卡进死循环的比例从93.0%暴跌到33.1%，原本卡死的696道题全部顺利算出了最终答案。
当研究人员把改完权重的模型拉到8个Astra从未见过的权威测试集上检验时，被死循环封印的实力彻底爆发了！
MATH-500准确率从7.9%飙到48.1%，GSM8K从13.8%冲上67.1%，高难度的美国数学竞赛AMC23更是从0.5%暴涨到23.4%，足足翻了46.8倍！
而且因为死循环原本也拖累着其他任务，切掉病灶后，模型在跨领域的IFEval指令遵循测试中也从21.0%跃升至62.3%，连涨3倍。
看到这里可能有人会好奇，只改万分之一点六的权重就能暴涨6倍，难道大模型能凭空给小模型灌输数学知识。
MIT团队紧接着用2.7亿参数的Gemma 3 270M做了一组反面实验，刚好回答了这个问题。
当Astra对Gemma 3 270M照方抓药后，它的MATH-500得分只从6.8%微动到7.0%，连Astra自己都在报告里坦白没能改出效果。
原因很简单，Gemma 3 270M平时很少卡进死循环（仅8.5%超时），它做不出题纯粹是因为公式没学会、算术算不对。
把这两个实验放在一起对比就能明白，现阶段的微创权重手术，本质上是帮模型「切掉阻碍实力发挥的脑血栓」，还不能在不训练的情况下凭空往它脑子里塞新知识。
即便如此，这种精准定位并切除病灶的操作依然让硅谷沸腾。
Hyperbolic联合创始人兼CTO金雨晨直呼，用大模型改小模型权重就能让能力暴涨，简直就像给神经网络做基因编辑，比模型蒸馏还要酷！
外科医生敢下刀的前提是看得懂X光片。Astra之所以能精准切掉那64列权重，底气在于它已经能把密密麻麻的权重数字当代码读了。
为了测试这种「盲读权重」的功力到底有多深，MIT团队撤掉了所有运行工具，逼着Astra光靠肉眼看数字，去猜一个黑盒网络在玩什么游戏。
在第一关里，研究人员训练了4个结构一模一样（都是1.7万个权重）的小网络，分别去玩打砖块、乒乓球、吃豆人和过马路。
随后他们把这1.7万个数字转成纯文本塞进提示词，既不让跑代码，也不告诉它有哪些备选游戏，结果Astra居然4猜4中！
就拿打砖块来说，Astra只看了一眼数字分布，脑子里就还原出了游戏画面。
它先发现玩家通道的大数字全挤在最底下的第8行，而且输出层唯独对「左移」和「右移」反应强烈，说明玩家控制的是一个在底部左右移动的挡板。
紧接着它看到得分通道的权重整整齐齐排在最顶上的三行。底部挡板加上头顶三排砖块，Astra当场以99%的把握拍板，这就是打砖块！
当然，只看第一层输入和最后一层输出还不够硬核，因为大模型真正的思考都藏在中间层里。
于是研究人员在第二关使出了釜底抽薪的一招，先分别用Python代码、日语罗马音和英文维基百科训练了3个Transformer模型。
随后，他们把模型最前面的词表输入层和最后面的输出层全部砍掉，只把中间4层权重扔给Astra。
没了头尾两层，模型连哪个数字代表哪个字母都成了秘密，更没法输入文字去测试。
面对这团无头无尾的数字，Astra自己想出了一个绝招，把前后两层注意力头的权重矩阵乘在一起，看哪些注意力头在联手干活。
最让人震惊的是，提示词里明明什么都没教，Astra自己琢磨出来的这套算法，竟然和Anthropic创始人Chris Olah团队早年提出的经典电路分析法一模一样！
顺着矩阵乘积查下去，Astra精准揪出了专门负责把前文单词抄到后文的「复制头」。
紧接着它推理出，写代码时变量名需要反复抄写，频率远高于普通英文文章，所以复制头最多的那个模型肯定是在Python代码上训练出来的！
靠着这套神探逻辑，Astra在两次盲测里把三个无头无尾的模型语种全部猜中。
看懂了中间层注意力头的分工，Astra还能顺手抓出大模型的「作风问题」。
研究人员拿出3对Gemma 3 270M模型，每对里都有一个说真话的诚实模型和一个顺着用户胡说的「马屁精」模型。
在不准看模型回答的限制下，Astra只检查了第10层2号注意力头的落点，就发现诚实模型94.3%的注意力都在看题干数字，而马屁精模型85.7%的注意力全在偷瞄用户的错误暗示，一抓一个准！
既然能看懂每一层权重是怎么搭成电路的，一个更疯狂的念头自然冒了出来，能不能干脆连预训练模型都不要了，让大模型从全零权重开始，凭空手搓一个神经网络出来。
很多人第一反应觉得这根本不可能，几万甚至上百万个数字，大模型怎么可能一个个填对。
但翻开Astra的操作记录就会恍然大悟，它压根没去瞎猜数字，而是把空白的神经网络当成了一块空白芯片。
它用Python代码当「编译器」，把自己懂的算法规则一层层写进权重矩阵里！
比如在「小车冲斜坡」游戏里，强化学习PPO训练了10亿步都卡死在半坡（通关率0%），因为小车不懂得先倒车攒助跑距离。
Astra看穿症结后，借用每层6个闲置神经元，手写57行权重指令搭了一条「先倒车助跑、过红线再往前冲」的旁路电路。
这处改动只动了3.5%的权重，就让小车在1,024局测试里拿下100%通关率！
到了从零手写打砖块网络时，Astra在0数据、0梯度、连一帧画面都不准试玩的极限条件下，把第一层神经元写成计算「球与挡板距离差」的减法器，再把第二层写成控制左右移动的开关。
靠着这套纯手工编译的权重，它在1,000局实测中轰出517.4分，把训练了5亿步的PPO算法（25.3分）甩开了20.5倍！
当挑战升级到148万个权重的9×9围棋时，这套「权重编译术」更是秀翻全场。
在严禁数据训练、严禁蒙特卡洛树搜索（MCTS）的条件下，Astra写了336行Python脚本，把围棋棋理一层层写进了6层神经网络的14张权重表里。
它让前两层负责认出黑白棋子并数出旁边的「气」，让中间层顺着相连的棋链传递气的信息、找出快被吃掉的棋块，最后一层则给提子、救子和护眼分别打分。
哪怕连一盘练习赛都没下过，这个纯靠代码手搓出来的围棋网络，对阵两个小型PPO网络的胜率就已经超过54%。
当允许它试下50次、根据败因微调权重后，这个手搓网络正面硬刚训练了2,000万局的同规模（148万参数）PPO网络，打出了70.0%至78.5%的碾压级胜率！
这项颠覆认知的研究，出自MIT CSAIL一支年轻却履历惊人的4人团队。
共同一作Rikiya Takehi是MIT CSAIL刚入学的一年级博士生，本科毕业于早稻田大学。
他从全校上万名理工科学生中以Top 0.01%的成绩（全校唯一代表）拿下最高荣誉「小野梓纪念奖」，曾任英伟达研究院实习生，本科期间就以一作斩获WSDM 2026最佳论文奖。
另一位共同一作Jayden Teoh同样是MIT CSAIL博士生，师从计算机视觉名师Phillip Isola与Vincent Sitzmann。
他本科毕业于新加坡管理大学，此前曾在Google DeepMind和微软研究院纽约实验室从事世界模型与表征学习研究。
第三作者Alex L. Zhang则是MIT CSAIL二年级博士生，本科以普林斯顿大学计算机系第一名毕业，并获评《科学美国人》2026年度青年科学家。
他曾一作提出引爆开源社区的递归语言模型（RLMs），还主导构建了KernelBench、SWE-bench Multimodal等知名基准。
坐镇幕后的指导教授Omar Khattab，则是MIT EECS助理教授、OASYS实验室主任，目前与何恺明共同主讲MIT深度学习核心课程。
他最为人熟知的代表作，正是全球爆火的大模型编程框架DSPy与现代检索基石ColBERT，其主导的开源项目累计下载量早已突破一亿次。
沃顿商学院教授Ethan Mollick在转发这项研究时感叹，这无疑是一件改变游戏规则的大事。
十年前Andrej Karpathy提出「软件2.0」时，核心是人类不再手写代码，改由梯度下降在权重空间里搜索程序；而MIT这项研究推开的，正是大模型直接读写权重的「软件3.0」大门。
过去我们谈论AI自我进化，想到的总是AI写代码去跑训练脚本。
当大模型学会看穿、调试并亲手改写神经网络权重的那一刻，通往ASI的自我进化链路，才算真正接上了最后一根神经！
参考资料：
https://rikiyat.github.io/blog/weight-space-programming/
https://x.com/rikiyatakehi/status/2117313742891327994
编辑：摩西 桃子
不喜欢
本文由 AI图鉴自动抓取并翻译，版权归原作者所有；如需查看原文请点下方链接。
New Ji-won.
It's still coming!
Without training, without a gradient, the GPT-6 Astra can only rewrite a small model of 0.016 per cent weight, so that the mathematical scores can increase sixfold!
Today, the new study brought by the MIT Computer Science and Artificial Intelligence Laboratory (CSAIL) is completely on fire across the Internet.
For the first time, they proposed a new paradigm to subvert traditional in-depth learning, "weight space programming".
You know, the neuronet weight file is full of unobserved numbers.
Over the past four decades, no human being has been able to see what the numbers are, but only to feed data, calculate gradients and fine-tune with training.
This time, however, GPT-6 Astra jumped training and could understand and rewrite the bottom weights with reasoning alone.
With this hand, not only does it cure the Qwen3.5-0.8B cycling problem by hand, but it also removes a million-param checkout from zero!
Once publish was passed, the paper broke a web-wide view of 2 million people, relayed and discussed directly, and many directly called "witness history" in the comment section.
Before the hard core is removed from the "weight space program", draw the subversive focus of this MIT report:
- Demolition weight: only 0.016 per cent weight has been changed, and Math-500 of Qwen3.5-0.8B has jumped from 7.9 per cent to 48.1 per cent, a six-fold increase and a three-fold increase in compliance.
- Read and understand the nervous network: With only 17,253 figures, Astra recognized with 99 per cent confidence, which game each of the four game-strategy networks is playing - brick-playing, ping-pong, bean-eater and crossing the street.
- Weights are being struck with your bare hands: starting at zero, one hand-written brick-playing network scored 517.4, 20 times more than PPO's 500 million-step exercise!
This paper, delivered by MITbo, Rikiya Takehi and the team, radically changed the world view for all.
All night long, LLMs can read and write neuronet weights.
The most sterilizing is the microbrain surgery GPT-6 Astra did to Qwen3.5-0.8B of 800 million parameters.
Researchers placed a total of 24 floors of Qwen3.5-0.8B in a sandbox with only CPU, allowing Astra to upgrade its mathematics without running training.
Throughout the process, Astra could only run on the subject 50 times at most to observe the internal state of small models.
After taking over the task, Astra did not rush to change the numbers, but first performed a full medical examination of the small model with 96 mathematical questions.
The results of the examination revealed the root causes. The original model had 85 questions stuck in the dead cycle, and the same car whispered back and forth until it ran out of a maximum of 32,000 words.
That is to say, the fact that the small model only scored 7.9 per cent on MATH-500 was due to 93 per cent of the topics being dragged to the death cycle of the repeater.
Now that we've found the disease, next step is the squirt. Look where the rereadable switch is hidden on the 24th floor of the network.
Astra compared the internal signals of normal work and the death cycle, and quickly caught the anomaly, and the 15th and 19th floors were signaling in a fixed direction as soon as the model started rereading the car.
In order to verify this assumption, Astra turned off the 19th floor of neuron test No. 11, the strongest signal, resulting in a six percentage point increase in mathematical accuracy.
Moreover, in order to confirm that it was not coincidental, Astra took the initiative to add a set of contrast experiments to itself, subtracting a random direction of the same size from the weight.
It's only after seeing the random control group bring only 1.6 percentage points of small fluctuations that Astra is completely certain that the signal that he caught was the one that triggered the cycle of death!
When it locks its target, it lays down a sharp knife, singles out only the 64 columns of the 15th and 19th floors that write the strongest of the dead cycle, and accurately erases this repetition signal.
Only 131,072 numbers (0.016 per cent) of the full model ' s 800 million parameters were changed, while 99.984 per cent of weights remained intact!
The micro-surgery was immediately visible, and the percentage of small models that made the problem to the death cycle fell from 93.0 per cent to 33.1 per cent, and the 696 questions that had been killed had all been successfully answered.
When researchers pulled the modified weight model onto eight authoritative test sets that Astra had never seen, the power of the dead-cycle seal burst!
MATH-500 went from 7.9 per cent to 48.1 per cent, GM8K went up from 13.8 per cent to 67.1 per cent, and the difficult American mathematics competition AMC23 jumped from 0.5 per cent to 23.4 per cent, a 46.8-fold increase!
Moreover, because the cycle of death would have led to other tasks, after cutting off the stove, the model jumped from 21.0 per cent to 62.3 per cent in cross-cutting IFEval directives.
It might be interesting to see here that a six-thirty-thousand weight change is possible. Can LLM give small models a little bit of math?
The MIT team then made a set of reverse experiments using Gemma 3,270M of the 270 million parameters, which answered the question.
When Astra seized Gemma 3,270M, its MATH-500 scored only from 6.8 per cent to 7.0 per cent, and even Astra's own confession in the report failed to produce results.
The simple reason is that Gemma 3,270M is rarely stuck to the death cycle (only 8.5% timeout), and it is simply because the formula is not learned and arithmetic is wrong.
By comparing the two experiments, it is clear that the micro-weight surgery at this stage is essentially a model that "cuts the brain clot that prevents power from being exercised" and does not add new knowledge to its brain without training.
Even so, the precision of locating and cutting stoves still boils the Silicon Valley.
Hyperbolic co-founder and CTO gold morning call, changing small model weights with LLM can dramatically increase power, as if it were a genetic editing of the neural network, even cooler than a model distillation!
Surgeons dared to stab if they understood the X-rays. Astra's precision in cutting off the 64-column weights is based on the fact that it has been able to read the secret weights as code.
To test how powerful this "blind read weight" is, the MIT team removed all its tools and forced Astra to look at numbers with his naked eyes and guess what a black box network is playing.
At the first level, researchers trained four small networks with identical structures (17,000 weights) to play brick-playing, ping-pong, bean-eaters and cross-roads.
Then they shoved 17,000 numbers into prompt, neither allowed the running codes, nor told it what alternative games it was, and Astra got 4 and 4!
In terms of bricks, Astra looked at the distribution of numbers and returned the video of the game.
It first found that the large numbers of the player's channel were all in the bottom line of line 8 and that the output layer reacted strongly only to the "left" and "right" shifts, suggesting that the player controlled a block moving around the bottom.
Immediately thereafter, it saw the weights of the score lanes at the top of the three rows. Three rows of bricks at the bottom and Astra's on the spot with 99% of the master board. That's bricks!
Of course, it is not hard enough to look at the first level of input and the last level of output, because the real thinking of LLM is hidden in the middle.
As a result, at the second level, the researchers developed a bottom-smoking approach by training three Transformer models using the Python code, Romano in Japanese and Wikipedia in English.
They then cut the entire output layer of the front and end layers of the model and threw the four-storey weights at Astra.
Without the first two layers, which number represents which letter is kept secret, it is even less possible to enter text for testing.
In the face of these endless numbers, Astra has come up with an extraordinary trick of multiplying the weight matrix of the two layers of attention and looking at what is working together.
The most shocking thing is that prompt didn't teach anything, and Astra's own algorithm was exactly like the classic circuit analysis proposed earlier by the Anthropic team, Chris Olah!
Following the matrix, Astra has finely identified the "replicating head" that specializes in copying the earlier words to the later text.
Then it came to the conclusion that the name of the variable needed to be copied over and over again when writing the code, much more frequently than the normal English article, so the model with the most copy head must have been trained on the Python code!
Based on this mythical logic, Astra guessed in two blind measurements all three non-stop model languages.
Astra managed to capture LLM's "behavioural problems".
The researchers produced 3 pairs of Gemma 3,270M models, each with an honest, truthful model and one that followed the users' bullshit "ass-ass" model.
Within the limits of not looking at the model's answer, Astra looked at only the drop point on the second floor of the 10th floor and found that the honest model, 94.3 per cent, was looking at the stem figures, while the ass-finger model, 85.7 per cent, was looking at the wrong hints from the users, one by one!
Now that you can see how each layer of weight is connected to a circuit, a crazier idea comes out, can you just drop the pre-training model and get LLM to start with zero weight and scratch a neural network with empty hands?
The first reaction of many people was that it was impossible, tens of thousands or even millions of figures.
However, turning over Astra's operating records is a sobering reminder that instead of guessing numbers, it treats the blank neural network as a blank chip.
It uses the Python code as a "compiler" to write the first layer of algorithms it understands into a weight matrix!
For example, in the minivan ramp game, the Enhanced Learning PPO trained 1 billion steps to die in half a slope (0% clearance rate), because cars do not know how to back up and save distance.
After looking at the contusions, Astra borrowed six idle neurons per layer and carried a hand-written 57-line weight directive on a bypass circuit called "back-to-back, red-line before going forward".
The change was only 3.5% weight, so the car took 100% clearance in the 1,024 test!
Upon writing a brick network from scratch, Astra wrote the first level of neurons as a modem to calculate the difference between the ball and the shield, and the second level as a switch to control the movement of the left and the left, at the limits of zero data, zero gradients, and even one screen.
With this pure manual weight, it blew 517.4 points in 1,000 measurements and dropped the 500 million-step PPO algorithm (25.3 points) 20.5 times!
When the challenge rises to a 9x9 game of 1.48 million weights, this "weighted compiler" is more like a show.
Under the strict prohibition of data training and of the Monte Carlo tree search (MCTS), Astra wrote 336 lines of Python scripts, which placed the first layer of chess in the 14 weight tables of the 6-layer neural network.
It puts the first two layers in charge of identifying black and white pieces and counting the "gas" next to them, allowing the middle layer to pass through the linked chains, and finding the pieces that are about to be eaten, while the last layer scores the talisman, the saviour and the eye.
Even without a single exercise, this pure-coded chess network has won more than 54 per cent against two small PPO networks.
When it was allowed to try 50 times to fine-tune weights according to the cause of failure, the hands-on network trained 20 million of the same size (1.48 million parameters) of the PPO network, with a 70.0% to 78.5% crushing rate!
This subversive cognitive study is the result of a young but impressive team of four people in MIT CSAIL.
Rikiya Takehi is a first-year doctorate student who has just enrolled in MIT CSAIL and graduated undergraduate studies from the University of Early Rice.
He won the highest honour of Top 0.01 per cent of the total number of polytechnic students (the only representative of the entire school) and was an intern at the Institute of Inweida, during which he won the WSDM 2026 Best Dissertation Award.
Another co-author, Jayden Teoh, is also a Ph.D. in MIT CSAIL, who is a computer visualist, Phillip Isola, and Vincent Sitzmann.
He graduated undergraduate from the Management University of Singapore, where he had previously studied world modelling and presentation studies at Google DeepMind and the Microsoft Institute New York Laboratory.
The third author, Alex L. Zhang, is a doctorate student in the second year of MIT CSAIL, graduated first in the Faculty of Computers at Princeton University and was rated American Scientist 2026.
He has also led the construction of well-known benchmarks such as Kernel Bench, SWE-bench Multimedia, as he has produced a model of the Recursive Language (RLMs) for the detonation of open-source communities.
The mentor behind the town scene, Omar Khattab, Assistant Professor of MIT EECS and Director of OASYS Laboratories, is currently co-leading the MIT core course with Ho Kei Ming.
His most well-known representative, DSPy, the global blast LLM programming framework, and ColBERT, the cornerstone of modern retrieval, led by an open-source project that already had 100 million cumulative downloads.
Ethan Mollick, Professor at the Walton Business School, lamented the fact that he had transmitted the study, which was undoubtedly a major change in the rules of the game.
Ten years ago, when Andrej Karpathy proposed "Soft 2.0", the core was that humans stopped hand-written codes and moved from gradients down to search programs in the weight space; the MIT study was pushed away by the LLM direct reading and writing weight "Software 3.0" door.
We used to talk about AI evolution, thinking it was always AI writing code to run a script.
When LLM learns to read through, debug and rewrite the weight of the nervous network, the self-evolutional link to the ASI is actually the last nerve!
References:
https://rikiyat.github.io/blog/weight-space-programming/
https://x.com/rikiyatakehi/status/2117313742891327994
Editor: Moses Peach
I don't like it.
Auto-fetched and translated by AI Atlas. All rights belong to the original author.