智东西（公众号：zhidxcom）
作者 | 王涵
编辑 | 漠影
近期，大模型行业掀起一股开源自研推理引擎的浪潮。
10月20日，DeepSeek正式开源生产环境实际使用的推理框架DeepInfer。几乎同时，10月22日，国内AI创业团队StartLux原点星辉也对外开源了其自研推理引擎Prism。
同样是自研推理系统，DeepSeek的DeepInfer服务于自家通用大模型的云端大规模业务；而Prism则聚焦“决策模型”，据公开资料，其也是首款专门面向决策模型打造并对外开源的推理引擎。
决策模型是近段时间快速走红的大模型新赛道。不同于主打对话、内容创作的生成式大模型，决策模型不需要输出大段文本，在读取材料之后，直接输出判断结果，给出是/否、选项选择、风险等级打分这类概率化结论。
自今年9月Jev模型爆火之后，决策模型迅速成为行业焦点，微软、AWS、Cloudflare等企业也相继入局布局。
在国内决策模型赛道，成立尚不足五个月的StartLux原点星辉已经跑出了不错的成绩。10月16日，该团队推出新一代决策模型StartLux‑Decision-v1.1，以69.1分登上Hugging Face的Decision Index 0.3.2榜单首位，在173个公开可复现模型中排名第一。
截至目前，StartLux决策模型累计下载量超18万，决策模型API已累计响应来自全球超300万次决策请求。
Github 推理引擎：
https://github.com/StartLuxLabs/StartLux-Prism
Github 决策模型：
https://github.com/StartLuxLabs/StartLux-Decision
Github 决策模型榜：
https://github.com/StartLuxLabs/DecisionPivot
Huggingface 决策模型：
https://huggingface.co/papers/2610.17309
Huggingface 决策模型榜：
https://huggingface.co/papers/2610.28522
不过，即便模型能力实现突破，落地环节仍存在现实瓶颈。
在此之前，StartLux-Decision-v1.1运行在基于vLLM 0.31的通用推理框架之上。然而，vLLM这类主流框架原生面向聊天、续写文本这类逐token生成场景设计，并不适配决策模型，模型的实际能力很难充分释放。
Prism就是StartLux给出的解法：按决策模型的需求重写推理引擎。
实测其在单张H200显卡上，Prism处理单个短问题比原方案快2.8倍，同一份材料连问三个问题，最快可提速5.3倍。
这套专为决策模型重构的推理引擎，到底做了哪些关键改动？各项性能数据的实测结果如何？为何越来越多AI企业选择自研推理框架，而非直接使用社区通用方案？本文将围绕以上问题展开解读。
一、StartLux用四大创新，破解推理效率难题
想要看懂Prism的优化思路，首先要厘清决策模型的运行逻辑。
例如有一位客户投诉：“订单被扣了两次钱，App里却还显示未付款。”接到这个投诉，系统需要同时判断：工单交给哪个部门、今天是否必须回复、问题有多严重。决策模型需要读完材料，分别给每道题的备选答案打分，返回各选项的概率。
反观vLLM、SGLang等通用推理框架，其设计初衷服务于文本生成任务。如果用它们跑决策模型，每一道问题都要被拆解成独立请求，分别完成网络请求、分词、调度，并且重复执行庞大输出层的计算。
Prism的全部优化，都围绕一个核心目标：完成一次决策，只执行必不可少的计算。
优化一：收敛输出计算范围
以27B参数量的 StartLux‑Decision 模型为例，它的输出层包含12.7亿权重，通用引擎需要对24.8万个候选词做完整打分，但决策任务只用到A‑Z共26个字母作为输出选项。
Prism将输出层计算范围压缩至原先的0.01%，模型依旧完整完成材料理解，只是在输出判断结果阶段，跳过大量无关候选词的计算。
优化二：“森林式”计算
业务场景中经常出现同一份材料对应多项判断需求。比如一份客户工单，需要同时判定归属处理团队、是否需要当日回复、故障严重等级。传统框架每解答一道题目，就要重新完整处理一遍工单材料。
Prism创新性地将批量任务构建成“森林式”计算结构：通用指令作为树根，共用的参考材料作为树干，各个独立问题作为分支树枝。
树根、树干对应的计算只执行一次，不同问题仅处理自身新增部分。针对同一份材料的多轮追问同样可以复用已有计算结果。
优化三：采用整数运算
在和大语言模型互动的时候，我们常常会遇到一个问题：同一个问题，大模型每次给出的答案都不尽相同，有时候甚至完全相反。
这一点在决策模型上是十分致命的，因为决策模型输出的概率值会直接指导业务系统执行后续动作。
Prism通过整数运算，解决结果漂移问题：相关矩阵运算全部使用int8精度，在int32精度下完成累加。因为整数加法不受计算顺序影响，计算结果不会受批大小、调度顺序干扰。
优化四：混排调度
StartLux‑Decision-v1.1支持AutoThink自动思考模式，模型把握充足时直接输出答案，把握不足则先生成最长4096 token的推理思考过程。
但这也带来了调度层面的难题：如果简短请求和长思考请求进入同一个任务队列，简单任务就会被长耗时请求阻塞。
Prism为此设计了一套混合调度机制：思考类任务的提示词并入直接输出批次完成预处理，后续解码步骤预编译为CUDA图，最多支持64条思考任务与直接输出任务分时占用显卡资源。新接入的简单请求最多等待1‑2个解码步骤就可以插队执行。
二、推理最高快5.3倍，231 题均速仅41毫秒
这样优化后，性能有着怎样的提升呢？StartLux对此进行了一系列对比测试。
（注：以下数据来自StartLux在不同模型版本及测试条件下的实测结果。其中，5.3倍加速测试使用StartLux-Decision-v1.1-27B AutoThink模型；231题跨引擎性能对比及吞吐量测试使用StartLux-Decision-v1-27B模型。具体测试配置与结果均已在GitHub公开。）
第一题：单请求延迟
测试结果显示，单道短问题场景下，Prism耗时19毫秒，原方案为53.1毫秒，提速2.8倍。针对同一份材料连续处理三道问题，Prism仅需17.4毫秒，原方案则要92.9毫秒，提速幅度达到5.3倍。
如果在三道问题基础上开启思考模式，Prism耗时785毫秒，原方案为1296毫秒，提速1.7倍。面对单条约8192 token的长问题，二者的性能差距收窄至1.2倍。
在更加直观的视频理解测试中，StartLux-Prime的速度比vllm快3.38倍，且输出的概率基本一致。
第二题：跨引擎延迟
研究团队选取JevBench测试集共231道题目，将Prism与vLLM、SGLang、llama.cpp等主流开源推理框架做横向对标。
结果显示，Prism单题平均延迟41毫秒，在所有参测框架中排名第一。紧随其后的decider编译版为66.8毫秒，SGLang为75.6毫秒，vLLM达到112.7毫秒，llama.cpp耗时最长，单题平均589毫秒。在同为8位精度的同等条件下，Prism的延迟比次快的框架低约40%。
这里有两点测试前提需要说明：decider和JevK5两款引擎无法直接搭载StartLux原生模型，测试使用了结构一致的替代模型，仅对比速度不考核准确率；另有部分框架不支持8位量化，只能以更高精度运行。
第三题：吞吐量
高并发吞吐测试中，64路并发跑满显卡时，Prism相对竞品的优势收窄到17%到22%。这一阶段瓶颈出在硬件算力层面，双方都已经将H200的性能拉满，软件优化很难再带来明显提升。
但决策类业务最常见的并非极限满载场景，而是高频出现的小型请求。在这种日常典型负载下，Prism每秒可以处理100道请求，表现最优的竞品为每秒38道，Prism领先2.6倍。
换言之，Prism的性能优势恰好落在企业真实高频使用的业务区间，这也是专用推理引擎最核心的价值。
第四题：多卡部署灵活适配
多卡部署层面，Prism支持数据并行与张量并行两种模式，也可以组合使用，适配不同的业务需求。
以27B版本模型为例，四张H200采用数据并行模式，各自独立接收请求，合计每秒可处理121个请求，约为单卡吞吐的3.92倍。如果四张卡采用张量并行模式合力计算同一道题，单题延迟可以从41毫秒降到29毫秒。
两组双卡混合部署则可以兼顾吞吐与延迟，实现每秒86道请求、单题36毫秒的表现。无论采用哪种部署方式，输出结果都与单卡运行时逐位一致。
第五题：精度与部署门槛
提速之外，模型的推理精度并未出现折损。
在覆盖七个评测集、共12351个决策样本的带思考测试中，Prism int8版本的平均准确率为92.94%，原引擎FP8版本为92.83%，未做量化的BF16原版为92.97%。Prism量化版本与满血原版的准确率差距仅0.03个百分点，甚至略优于原有引擎的量化版本。
部署门槛也比预想中更低。该模型有16.55GB的压缩版本，单块32GB显存的RTX 5090消费级显卡就可以运行，输出效果对标专业级显卡H200。在这个压缩版本上，Prism比主流开源框架llama.cpp快3.9 倍。
三、推理系统拉开身位，模型企业开始自研推理引擎
短短两天之内，DeepSeek与StartLux先后开源自研推理引擎，折射出大模型行业的共同转向：模型能力的比拼之外，推理工程的重要性正在快速抬升。
无论是通用大模型还是垂直领域的决策模型，一旦嵌入业务核心流程，每一次调用的延迟、每万次请求的算力开销都会直接体现在业务成本中。同款模型的落地成本、响应速度、运行稳定性，开始成为决定其商业化空间的关键因素。
而通用推理框架需要适配不同结构的模型、覆盖千差万别的应用场景，必然要在专用性上做出妥协，只有模型厂商自身最清楚自家模型的接口特性与优化空间。
StartLux走的正是这条自研路径，同时又在决策模型这个细分赛道走出了自己的开放风格。
和很多厂商仅开源模型权重或推理代码不同，StartLux的每一次产品发布，都同步输出底层方法论与配套工具链，把自身的实践经验转化为行业可以复用的公共资产。
此前发布 StartLux-Decision 决策模型时，团队就同步公开了RLCD²强化学习训练算法，系统性解决了决策模型“何时启动思考、何时终止思考”的训练难题，填补了决策模型训练方法论的空白。同时推出的DecisionPivot评测基准，也补上了行业对决策模型细节识别能力的评测标准缺口，为全行业提供了统一的对比参照。
本次Prism推理引擎的开源，同样延续了这一思路。团队不仅放出了完整的引擎代码，更公开了针对决策模型推理的全套优化逻辑，相当于把决策模型推理工程的核心设计思路完整开放给行业。
DeepSeek与StartLux这类开源AI厂商，把推理工程从单家厂商的内部能力，变成行业可复用、可验证的公共基础设施。
对于尚处发展初期的决策模型赛道而言，这种“模型+方法+工具”的开放模式，直接降低了全行业的试错成本，让更多团队不用从零开始搭建训练、评测、部署的完整链路，可以聚焦于上层的场景创新与效果优化。
结语：决策模型的竞争，推理层将是胜负手
想要把决策模型真正用在风控、内容审核、业务工单路由等真实业务当中，仅有高性能模型远远不够。
Prism的本质，是针对决策模型重新核算整套推理计算逻辑。当大模型的输出从长篇文本，转变为一组概率判断，过去文本生成时代推理引擎的各类默认假设，都有重新审视、重新优化的空间。
未来决策模型赛道的竞争，也会朝着这个方向演进：模型架构的差距会不断缩小，但推理工程层面的壁垒会变得更显著，而后者无法依靠堆更大参数量的模型简单补齐。
这其实让决策模型和整个模型领域的竞争核心变得更突出，也让外界更加期待，谁能跑得更远，做的最深。
本文由 AI图鉴自动抓取并翻译，版权归原作者所有；如需查看原文请点下方链接。
Intellectual (WeChat Official Account:zhidxcom)
The author, Wang Xin
Edit |
In the recent past, the LLM industry has unleashed a wave of reasoning engines.
On 20 October, DeepInfer, a framework of reasoning for the practical use of the formal open source production environment in DeepSeek. At almost the same time, on 22 October, the national AI entrepreneurial team, StarLux Probe, also opened its own self-study reasoning engine, Prism.
Also a self-study reasoning system, DeepInfer in DeepSeek serves its own General LLM cloud operations on a large scale, while Prism focuses on the “decision-making model”, which, according to publicly available information, is also the first paragraph of the reasoning engine designed specifically for decision-making models and open to external sources.
The decision model is the new track of LLM, which has been in red for some time. Unlike LLM, which is a generating mode of dialogue, content creation, the decision model does not need to output large sections of the text and, after reading the material, outputs the findings directly, giving such probabilistic conclusions as yes/no, option selection, risk rating.
Since the Jev model exploded in September this year, decision-making models have rapidly become the focus of the industry, with Microsoft, AWS, Cloudflare and others joining in.
In the domestic decision-making model track, the StarLux Prototype Faipule, which was established less than five months ago, has made a good run. On 16 October, the team launched a new generation of decision-making models, StartLux-Decisión-v1.1, with 69.1 points to the top of the Hungging Face list of Decision Index 0.3.2 and ranked first among the 173 publicly revolving models.
To date, the StartLux decision model has accumulated over 180,000 downloads and the decision model API has responded to over 3 million requests for decision-making globally.
Github reasoning engine:
https://github.com/StartLuxLabs/StartLux-Prism
Github Decision Model:
https://github.com/StartLuxLabs/StartLux-Decision
Github Decision Model List:
https://github.com/StartLuxLabs/DecisionPivot
Huggingface Decision Model:
https://huggingface.co/papers/2610.17309
Huggingface Decision Model:
https://huggingface.co/papers/2610.28522
However, even if there is a breakthrough in modelling capacity, there are still real bottlenecks in the landing chain.
Prior to this, the StartLux-Decisión-v1.1 operated on a generic reasoning framework based on v LLM 0.31. However, mainstream frameworks such as v LLM are originally designed to chat, to renew text, to generate scene-by-scenes designs that are not suitable for decision-making models, and the actual capabilities of models are difficult to fully release.
Prism is the solution given by StartLux: rewrite the reasoning engine according to the needs of the decision model.
On a single H200GPU, Prism was measured to be 2.8 times faster than the original programme to deal with a single short issue, and the same material asked three questions in a row, with a faster rate of 5.3 times faster.
What are the key changes in the reasoning engine, which was re-engineered for the decision model? What were the results of the performance data? Why are more and more AI enterprises choosing to study their own reasoning frameworks rather than using community-wide programmes directly? This paper will be read in the light of the above.
1. StartLux uses four innovations to solve the problem of the efficiency of reasoning
In order to understand Prism ' s idea of optimization, it is first necessary to clarify the operational logic of the decision model.
One customer, for example, complained: “The order was withheld twice and the App showed that it was not paid”. The system needs to determine the department to which the work orders are given, whether it is necessary to respond today and how serious the problem is. The decision model needs to finish reading the material, score the alternative answers to each question and return the probability of each option.
A generic reasoning framework such as vLM, SSGLang, which was designed to serve text generation tasks. If they run the decision-making models, each issue is to be broken down into independent requests, the network requests, the narratives, the movements, and the large output layer calculations are repeated.
All of Prism ' s optimization revolves around a core objective: to complete a single decision and to implement only essential calculations.
Optimization I: Computation range
Using the StarLux-Decisión model of 27Bparameter count, for example, the output layer contains 1.27 billion weights, and the universal engine requires a full score of 248,000 candidates, but only 26 letters of A-Z are used as an output option for decision-making tasks.
Prism compresses the range of the output layer to 0.01 per cent, and the model is still complete with material understanding, and only skips a large number of unrelated calculations at the output judgement stage.
Optimization II: “Forest” calculations
The same material is often used in the business landscape to respond to multiple judgement demands. For example, a client's work list would need to be determined simultaneously as belonging to the processing team, whether a response was required on that date, and the level of failure. For each answer to a title, the traditional framework has to reprocess the worksheet.
Prism innovatively constructs bulk tasks into “forest-like” computing structures: generic directives as roots, shared reference materials as trunks, and individual issues as branch branches.
The root and trunk calculations are performed only once, and the different issues deal only with their own new components. Multiple-wheel queries for the same material can also be reused.
Optimization III: use of integer numbers
When we interact with big language models, we often encounter a problem: the same question, the answers given by LLM vary from one question to another, and sometimes even the opposite.
This is fatal in the decision-making model, as the probability values of the decision-making model output directly guide the operational system in implementing follow-up actions.
Prism solves the problem of drifting results through integer operation: the matrix operation is fully using int8 precision and is cumulative at int32 accuracy. Since integer additions are not affected by the order of calculation, the result of the calculation will not be interfered with by batch size, movement order.
Optimization IV: Mixed schedule
StartLux-Decisión-v1.1 supports AutoThink's self-thinking model, with the model exporting the answer directly when it is adequate, and Mr. under-accepted as the reasoning process of up to 4096 token.
But it also poses problems at the control level: If brief requests and long reflection requests enter the same task line, simple tasks are blocked by lengthy requests.
Prism designed a hybrid movement control mechanism for this purpose: the pre-processing of prompt for thought-type tasks into direct export batches, the pre-compilation of subsequent decodering steps into CUDA maps, and the use of GPU resources for up to 64 reflection and direct output missions. Simple requests for new access can be executed in teams with a maximum of 1-2 decoded steps.
ii. A maximum of 5.3 times the reasoning, with an average speed of only 41 ms.
How did performance improve when it was optimized? StartLux performed a series of comparative tests.
(Note: The following data are derived from the results of the tests carried out by StartLux in different model versions and under test conditions. Of these, 5.3 times the acceleration test uses the StarLux-Decision-v1.1-27B AutoThink model; 231 cross-engine comparison and throughput tests uses the StarLux-Decision-v1-27B model. Specific test profiles and results are available in GitHub.
Question one: One request for delay
The test results show that in a single short problem scenario, Prism took 19 milliseconds and the original scheme was 53.1 milliseconds, at 2.8 times faster. Prism only takes 17.4 milliseconds for three consecutive treatments of the same material, while the original programme takes 92.9 milliseconds at a rate of 5.3 times.
If the mode of thinking is initiated on the basis of three sets of questions, Prism takes 785 ms and the original scheme is 1296 ms, at 1.7 times faster. Faced with the long question of a single treaty, 8192 token, the performance gap was reduced to 1.2 times.
In a more intuitive video understanding test, StartLux-Prime is 3.38 times faster than vllm, and the probability of output is largely consistent.
Question two: Delays across engines
The research team selected a total of 231 topics from the JevBench test set, cross-referenced the main open-source reasoning frameworks of Prism and v LLM, SGLang, llama.cpp.
The results showed an average delay of 41 ms for Prism and ranked first in all reference frames. The subsequent decider compilation was 66.8 milliseconds, SGLang 75.6 milliseconds, v LLM 112.7 milliseconds, llama.cpp the longest, with an average of 589 milliseconds. Under the same eight-bit accuracy, Prism ' s delay is approximately 40 per cent lower than the secondary framework.
There are two test premises that need to be explained: the Decider and JevK5 engines could not carry the StartLux prototype directly, and the tests used a structured alternative model that did not check the accuracy rate against speed; and the other part of the framework did not support 8 bits of quantization, which could only operate at higher accuracy.
Question number three: Bumping volumes
The Prism ' s comparative advantage in the high-spam test narrows to between 17 and 22 per cent when the 64th road runs with GPU. The bottlenecks at this stage are at the level of hardware computing, with both parties already filling up the performance of H200, and it is difficult for software optimization to bring about significant further improvements.
However, the most common type of decision-making business is not a maximum full-loading scenario, but rather a small request for high-frequency presence. Under this typical daily load, Prism can handle 100 requests per second, with 38 best-performing bids and Prism leading 2.6 times.
In other words, Prism ' s performance advantages happen to fall within the operational zone of the real high frequency used by the enterprise, which is also the core value of the dedicated reasoning engine.
Question No. 4: Flexible deployment of Doka
At the deployment level, Prism supports two models of data running in parallel and in parallel, which can also be used in combination to suit different operational requirements.
In the case of the 27B version model, four H200s, using a data parallel model, receive requests independently, each with a combined capacity of 121 requests per second, approximately 3.92 times the single-calorie amount. If the four cards calculate the same issue using a parallel block mode, the single delay can be reduced from 41 ms to 29 ms.
The combined deployment of the two sets of double-calories allows for a balance between throughput and delay, resulting in 86 requests per second and 36 milliseconds per second. Regardless of the mode of deployment, the output results are consistent with the single card running.
Question 5: Precision and deployment threshold
The accuracy of the model ' s reasoning has not been compromised beyond the acceleration.
The average accuracy rate of Prism int8 was 92.94 per cent, of the original FP8 version 92.83 per cent and of the original BF16 without quantization 92.97 per cent, out of a total of 12,351 decision-making sample. The difference between Prismquantization version and the original full blood version is only 0.03 percentage points, and even slightly better than the original engine version of quantization.
The deployment threshold is also lower than expected. The model has a compressed version of 16.55GB, and a single 32GBVRAM RTX 5090 consumption class GPU can be run, with output effects on the standard professional level GPUH200. In this compressed version, Prism is 3.9 times faster than the mainstream open source framework,llama.cpp.
III. ELECTRONIC SYSTEMS PULLING UP AND THE MODEL COMPETITION STARTED LEARNING THE EDITION ENGINE
In just two days, DeepSeek and StartLux started from the research and reasoning engine, reflecting a common shift in the LLM industry: beyond model capabilities, the importance of reasoning works is rising rapidly.
Both generic LLM and vertical area decision-making models, once embedded in core business processes, each call is delayed and every 10,000 requests are accounted for directly in operational costs. The cost of landing, response speed and operational stability of the same model are beginning to be key determinants of its commercial space.
The generic reasoning framework, which requires models that fit different structures and cover a wide variety of applications, must necessarily compromise in terms of specificity, with only model manufacturers themselves being the ones who know best how to interface with their own models and optimize space.
It's this self-study path that StartLux is following, and at the same time he's out of his own open style in the decision-making model, the segmented track.
Unlike many manufacturers that only open-source model weights or codes of reasoning, each of the products publish of StartLux synchronizes the bottom methodological and supporting tool chains, translating its practical experience into public assets that industry can reuse.
Prior to the publish StartLux-Decisión decision-making model, the team synchronized the publication of the RLDCD2 intensive learning training algorithm, systematically addressing the training challenge of the decision-making model “when to start thinking and when to stop thinking”, filling a gap in the methodology of the decision-making model. The simultaneous launch of the DecisiónPivot assessment benchmark has also filled the gap in industry standards for identifying the details of decision-making models, providing a common comparison across industry.
The opening source of this Prism reasoning engine also continues this line of thinking. The team not only released the full engine code, but also made public the entire optimised logic of the decision model reasoning, which is tantamount to opening up the core design thinking of the decision model reasoning project to the industry.
Open-source AI manufacturers such as DeepSeek and StartLux have transformed reasoning from the internal capacity of a single manufacturer into a reusable and verifiable public infrastructure for the industry.
This open model of “model + methodology + tool” has directly reduced industry-wide trial error costs for the initial development of decision-making models, allowing more teams to focus on top-level scenario innovation and optimization without starting with a complete chain of training, evaluation and deployment.
CONCLUSION: Competition in decision-making models, the reasoning level will be the winner. Hands.
High-performance models are far from sufficient to actually use them in real operations such as wind control, content auditing, and individual routing.
The essence of Prism is to recalculate the reasoning logic of the entire system for the decision model. When the output of LLM moves from a long text to a set of probabilistic judgements, there is scope for revisiting and optimising the various default assumptions of the engine of reasoning of the old generation.
Competition for future decision-making models will also evolve in this direction: while gaps in model structures will continue to narrow, barriers at the engineering level of reasoning will become more significant, and the latter cannot simply be supplemented by models with larger parameter count piles.
This in fact makes the core of competition in decision-making models and in the model area as a whole more prominent, as well as greater expectations from the outside world as to who can go further and do the most.
Auto-fetched and translated by AI Atlas. All rights belong to the original author.