AI 模型和平台
Together AI 带来生产推论连续汇编
在 Google 上的首选来源中添加 Unite.AI
2026年11月4日，Together AI 宣布多态推论，引入了"共同编译器"作为首次实施。 agent驱动的系统不断对改变模型，流量和硬件的推论进行调整，并且已经贯穿了公司的许多顶级生产推论工作量和一些流行的无服务器端点。
Together AI报告，编译器在24小时内通过涉及Kimi K3,GLM 5.3,MiniMax M3.1-Flash和DeepSeek v4.1-Flash的测试，提供了每秒高度优化的开源基线的符号1.3x至3x. 这些结果为公告的核心提议提供了框架：优化可以是一个与部署挂钩的持续搜索过程，而不是在发布冷冻的一轮人工调频。
改变模式、交通和硬件
多态推论(polymorphic Information)是Together AI的术语，用于服务适应更快的模型发布的基础设施，多样化的建筑，不断发展的交通形状和服务级别要求，以及更频繁的硬件变化。 对比的静态方法从固定引擎配置开始。 调制一个设置的配置不一定适合另一个设置，特别是在模型行为和请求分布改变时。
该公司的生产观察提供了具体的例子。 Together AI报告，在一个国家的工作时间里，促导值达到约97,000令牌，而其他地方为70,000令牌，在一次模式发布后的两周里，由于使用转向代理工作量，促导长度上升了80%。 客户突袭还可以将快速长度增加一倍，降低缓存命中率，从而改变调整现有配置的条件。
公告还修改了Together AI早先对巨型内核的强调，巨型内核将操作导导为单一内核。 在将早期的版本投入生产后，公司得出结论，单是聚变不是一个通用的设计目标：混合专家架构和动态工作量可能需要不同的技术。 围绕固定或随机流量开发的内核同样可能因数据依赖的路由影响从而取得最佳效果的生产流量而被误判。
如何持续编纂工作
共同编译器将流量形状，模型和硬件作为输入，然后使用agents生成并测量推论引擎的候选优化。 恒定引擎一次加载该型号，热重加载候选人前行通行证而不重启全引擎。 平行的agents通过消息板生成候选人并进行交流，在其它agents进行推理或使用工具时保持实验排队。
两个测量级平衡了实验量与现实性。 快引擎加载一个子集层，用于快速迭代和低变速计时；慢引擎加载更多层，以测试候选人是否坚持现实的数据分布。 一个单独的成本模型结合硬件规格，微观基准和测量前传，在agents花时间优化候选人之前对平行-配置搜索进行推敲。
技术论文称获奖候选人在获得服务之前接受KV-cache污染和输出质量的核查测试。 在优化预算用尽后，当最终源代码准备融入服务引擎时，检查会运行。
发布的启动比较报告467秒的状态序列，这些数字包括优化期间的发动机启动情况，而不是个别请求的延迟情况，并有助于解释为什么热重装是系统试验吞吐量的核心。
评价范围和客户准入
对于引擎比较，论文从2026年10月30日起使用领先的开放配置，并在隔离代理优化效果时保持基线平行战略固定。 Trainium2是标准评价工作量中披露的例外：它使用合成多回合数据集，因为实施尚不能支持在现有氢氟烷烃能力范围内获得更高一致。
Together AI将优化结果部分归结为252倍的实验吞吐量高于基线代理递归自改进优化系统。
除了语言模型推论之外，论文还报告了1.5x至6.4x的速率，涉及经测试的天气、生物学、时间序列和部分差异方程模型，同时保持了上述评价的比特精确产出。
End 客户可以通过有限的β请求早期访问。 Together AI计划一击优化专用端点，而其研究路线图则包括按域，请求，信使和流量负载调整的投机解码，外加跨工人的动态路由和异端服务器流量的部署配置。
发布使部署特定的测量，而不是普遍优化的内核，Together AI推论堆栈的组织原理。 客户的直接问题是，有限的β能否在通过同样的质量检查的同时，在自己的模型、流量和硬件上复制所报告的增益——连续汇编所要处理的变异。
本文由 AI图鉴自动抓取并翻译，版权归原作者所有；如需查看原文请点下方链接。
AI Models & Platforms
Together AI Brings Continuous Compilation to Production Inference
Add Unite.AI to your preferred sources on Google
Together AI announced Polymorphic Inference on November 4, 2026, introducing Together Compiler as its first implementation. The agent-driven system continuously retunes inference for changing models, traffic and hardware, and already runs across many of the company’s top production inference workloads and some popular serverless endpoints.
Together AI reports that the compiler delivered 1.3x to 3x the tokens per second of highly optimized open-source baselines in under 24 hours across tests involving Kimi K3, GLM 5.3, MiniMax M3.1-Flash and DeepSeek v4.1-Flash. Those results frame the announcement’s central proposition: optimization can be an ongoing search process tied to a deployment, rather than a round of manual tuning frozen at launch.
Changing Models, Traffic and Hardware
Polymorphic Inference is Together AI’s term for serving infrastructure that adapts to faster model releases, diverse architectures, evolving traffic shapes and service-level requirements, and more frequent hardware changes. The contrasting static approach starts from a fixed engine configuration. A configuration tuned for one setting is not necessarily suitable for another, particularly when model behavior and request distributions change.
The company’s production observations supply concrete examples. Together AI reports that prompts during one country’s working hours reached about 97,000 tokens, compared with 70,000 elsewhere, and that prompt lengths rose 80% in the two weeks after one model release as usage shifted toward agentic workloads. Customer bursts can also double prompt lengths and reduce cache hit rates, changing the conditions under which an existing configuration was tuned.
The announcement also revises Together AI’s earlier emphasis on megakernels, which fuse operations into a single kernel. After putting early versions into production, the company concluded that fusion alone is not a universal design goal: mixture-of-experts architectures and dynamic workloads can require different techniques. Kernels developed around fixed or random traffic can likewise be miscalibrated for production traffic whose data-dependent routing affects which optimizations win.
How Continuous Compilation Works
Together Compiler takes the traffic shape, model and hardware as inputs, then uses agents to generate and measure candidate optimizations for an inference engine. A persistent engine loads the model once and hot-reloads candidate forward passes without restarting the full engine. Parallel agents generate candidates and communicate through a message board, keeping experiments queued while other agents are reasoning or using tools.
Two measurement tiers balance experiment volume against realism. Fast engines load a subset of layers for rapid iteration and low-variance timings; slower engines load more layers to test whether candidates hold up on realistic data distributions. A separate cost model combines hardware specifications, microbenchmarks and measured forward passes to prune the parallelism-configuration search before agents spend time optimizing candidates.
The technical paper says winning candidates undergo verification tests for KV-cache contamination and output quality before they can be served. The checks run after the optimization budget is exhausted, when the final source code is prepared for integration into the serving engine.
The launch post’s startup comparison reports 467 seconds to serving for its status-quo sequence, versus 9.0 seconds with Together Compiler’s setup. Those figures cover engine startup during optimization rather than individual request latency, and help explain why hot-reloading is central to the system’s experiment throughput.
Evaluation Scope and Customer Access
For the engine comparisons, the paper used leading open configurations as of October 30, 2026, and kept the baseline parallelism strategy fixed when isolating the effect of agentic optimization. Trainium2 was the disclosed exception to the standard evaluation workload: it used a synthetic multi-turn dataset because the implementation did not yet support higher concurrencies within the available HBM capacity.
Together AI attributes the optimization results partly to 252-fold higher experiment throughput than baseline agentic recursive-self-improvement optimization systems.
Beyond language-model inference, the paper reports 1.5x to 6.4x speedups on tested weather, biology, time-series and partial-differential-equation models while maintaining bit-exact outputs on the stated evaluations.
End customers can request early access through a limited beta. Together AI plans one-click optimization of dedicated endpoints, while its research roadmap includes speculative decoding that adapts by domain, request, token and traffic load, plus dynamic routing across workers and deployment configurations for heterogeneous serverless traffic.
The launch makes deployment-specific measurement, rather than a universally optimal kernel, the organizing principle of Together AI’s inference stack. The immediate question for customers is whether the limited beta can reproduce the reported gains on their own models, traffic and hardware while passing the same quality checks—the variation that Continuous Compilation is designed to address.
Auto-fetched and translated by AI Atlas. All rights belong to the original author.