MIT CSAIL 博士生 Rikiya Takehi 和团队提出了一种权重空间编程方法，用 GPT-6 Astra 直接读写神经网络的权重参数，无需传统的梯度下降训练流程。他们在 Qwen3.5-0.8B 模型上修改了约 0.016% 的权重，让指令遵循能力提升 3 倍。实验在小型模型上完成，Astra 如何实现权重读写的技术细节未公开。
传统训练神经网络需要准备数据集、定义损失函数、通过反向传播计算梯度、多轮迭代更新权重。这次实验跳过了这些步骤，让大语言模型直接操作权重矩阵里的数值，类似用代码编辑器修改程序变量，而不是通过编译器和优化器间接调整。
四组实验的具体结果
团队在 Hacker News 披露的实验细节显示，他们测试了从简单策略到语言模型的多个场景。实验 1 是在 10×10 网格上训练一个策略网络玩游戏，传统方法训练后模型在某项任务中成功率为 0%,Astra 修改权重后成功率提升到 100%。
实验 2 用 12×12 灰度图像训练一个视觉分类器，训练过程用 Adam 优化器跑 40000 步、批次大小 512。这是传统训练方法建立的对照基线，Astra 在训练完成后修改权重改变了模型行为。
实验 3 分析了一个宽度 384 的小型语言模型，Astra 识别出负责复制任务的注意力头得分在 0.22 到 0.55 之间，而其他头的得分最高只有 0.02。
实验 4 成功率同样从 0% 提升到 100%，但来源未区分实验 1 和 4 的任务差异。
方法的适用边界
目前披露的实验都在小型模型上进行，Qwen3.5-0.8B 有 8 亿参数，实验 3 中的语言模型宽度只有 384。
GPT-6 Astra 是 OpenAI 尚未公开发布的模型，其权重读写能力的实现机制、适用模型规模上限、是否需要模型架构先验知识，团队尚未说明。在已披露的 8 亿参数规模实验外，更大模型的适用性未知。
方法名为权重空间编程，但权重的语义可解释性一直是神经网络研究的难题。单个权重或一组权重对应什么功能，通常需要通过激活分析、因果干预等手段推断。Astra 如何定位需要修改的权重、如何判断修改的幅度和方向、修改是否会引发其他层的连锁反应，这些问题的答案决定了方法的泛化性。
与传统训练方法的对比
梯度下降训练的优势是有理论保证和大规模工程实践积累，但需要大量数据、算力和时间。权重空间编程如果成立，可以在特定任务上快速调整模型，但缺少对收敛性、稳定性和副作用的系统分析。
实验中 0.016% 的修改比例小于常见微调方法涉及的参数范围。直接修改原权重的风险是可能破坏预训练阶段学到的知识，除非修改的位置和幅度经过精确控制。
指令遵循能力提升 3 倍的衡量标准未公开，可能是成功率、准确率或人工评分。如果只在特定指令格式或任务类型上有效，泛化性就受限。
团队尚未发布代码、论文或更详细的技术报告，也没有第三方复现结果。方法的可重复性和在实际开发场景中的适用范围仍需验证。
原文地址：tools.wflynn.cn/aitools/tre…
关注公众号「YourTools」，每天推送 GitHub 趋势与开源工具速报：
本文由 AI图鉴自动抓取并翻译，版权归原作者所有；如需查看原文请点下方链接。
MIT CSAIL Ph.D. Rikiya Takehi and the team have proposed a weighting space programming method using GPT-6 Astra to read and write directly the weight parameters of the neural network without the need for a traditional gradient to drop the training process. They modified about 0.016% weight on the Qwen3.5-0.8B model to increase command compliance by three times. The experiment was completed on a small model, and the technical details of how Astra achieved the weight to read and write were not made public.
Traditional training nerve networks require the preparation of data sets, the definition of loss functions, the calculation of gradients by reverse transmission, and the updating of multi-rotational weights. This experiment skipped these steps by allowing large language models to directly operate the values in the matrix of weights, similar to changes in program variables by code editor rather than indirectly by compilers and optimizers.
Specific results of four sets of experiments.
The details of the experiments disclosed by the team in Hacker News show that they tested multiple scenarios ranging from simple strategies to language models. Experiment 1 is the training of a strategy network game on a 10x10 grid, with success rate being 0% in a traditional method training model and success rate being raised to 100% after the Astra modified weight.
Experiment 2 Trains a visual sorter with 12 x 12 greyscale images, and runs 40000 steps and batch sizes with Adam Optimizer. This is the contrast baseline established by traditional training methods, and Astra changed the model behaviour after the training was completed.
Experiment 3 analyzed a small-language model of 384 widths, and Astra identified a concentration score of between 0.22 and 0.55, while the other head scored only 0.02.
The experiment 4 success rate also rose from 0% to 100%, but the source did not distinguish between the tasks of the experiment 1 and 4.
Scope of application of the methodology
All currently disclosed experiments are carried out on small models, with Qwen3.5-0.8B having 800 million parameters and a language model width of 384 in experiment 3.
GPT-6 Astra is a model of publish that Open AI has not yet made public, a mechanism for the realization of its heavy reading and writing capability, application of the maximum size of the model, need for a priori knowledge of the model architecture, which the team has not yet explained. The applicability of LLM is unknown beyond the already disclosed 800 million-parameter scale experiment.
The method is called weight space programming, but semantic interpretation of weight has been a challenge for neuronetwork research. A single weight or a set of weights corresponding to what function is usually inferred from activation analysis, causal intervention, etc. The answer to Astra ' s question as to how to locate the weights that need to be modified, how to judge the extent and direction of the modification, whether the modification triggers a chain reaction from other layers determines the generality of the methodology.
Comparison with traditional training methods
Gradient drop training has the advantage of theoretical assurance and the accumulation of large-scale engineering practices, but requires considerable data, computing and time. If established, the weighting space programming can quickly adjust the model for a given mission, but there is a lack of systematic analysis of resistance, stability and side effects.
The percentage of 0.016% modification in the experiment is smaller than the range of parameters covered by the common fine-tuning method. The risk of direct modification of the original weight is that the knowledge acquired at the pre-training stage may be undermined unless the location and range of the modification are accurately controlled.
The three-fold increase in the ability to comply with the directive is not known and may be success rate, accuracy rate or manual rating. Generality is limited if it is valid only for a particular command format or type of task.
The team has not yet obtained publish codes, papers or more detailed technical reports, nor have the results been reproduced by third parties. The replicability of the methodology and its scope of application in the actual development landscape still need to be tested.
Original address:tools.wflynn.cn/aitools/tre...
Following the WeChat Official Account "YourTools", GitHub's trend and open source tool is sent daily:
Auto-fetched and translated by AI Atlas. All rights belong to the original author.