最常被问到的问题之一是我们是如何在Cerebras wafer比例芯片上部署大型模型的一个单一的Cerebras WSE-3T wafer拥有44 GB的芯片存储器，万亿参数的边框模型需要三字节来单独表示其重量。
以每秒1000-2000+令牌的方式运行。
我们将使用开源模型Kimi K3
到最后，你会明白：
- 我们如何把完整的模型融入芯片记忆中
- 我们如何在多个芯片上映射模型
- 我们如何处理大量的并行请求
用Kimi K3拆开
Kimi K3是一个开源，3T参数模型，具有69个线性关注层和24个密集层。
它在终端-Bench 2.1(vs.GPT-5.6 Sol的88.8%)和浏览孔普(vs.GPT-5.6 Sol的90.4%)上得分惊人的88.3%. 这些结果表明前沿编码和代理能力，甚至在一些基准测试上表现优于领先的专利模型。 详见此处Kimi K3.
举例来说，我们使用：
- 假定平均请求大小256k
- 4位重量量化(如Hugging Face执行)
- 8位 KV 缓存
# 1：每层的大小
为了计算Cerebras WSE-3T wafers的总数，我们需要支持Kimi K3，让我们首先计算每个模型层的大小。
Kimi K3有3兆参数。 如果所有3T参数每个位存储4位，原始参数有效载荷是：
将有效载荷分为93层，然后给出：
#2：瓦费尔数
现在，我们知道每层需要大约16GB，我们可以找出每44GB饼上配有多少层。 答案取决于层类型：
- Dense层需要一个额外的 KV 缓存空间，因为每个密集( 完全注意) 层会存储每个符号的密钥和值。 因此，我们每卷饼放置一个密集层，留下剩下的SRAM作为它的KV缓存。
- 线性观察层不需要为KV缓存增加空间。 线性注意层不是为每个令牌存储密钥和值，而是为每个请求保持一个小的，固定大小的状态，无论上下文长度如何。 这为每卷饼(2×16 GB=32 GB)两个线性层留下了空间。
因此，有了Kimi K3的69个线性层和24个密集层，我们得到：
这让我们能计算出35+24=59个饼
#3：连接饼饼
Cerebras在管道中执行模型。 在Cerebras编译器绘制了跨区域wafers的模型层之后，运行时间绘制了符号在wafers之间流动的路径。 对于超快的潜伏性，只需要在饼饼之间传递激活，这可以通过RDMA通过以太网完成。 这大约是每跳2微秒。
需要强调的是，由于每个Cerebras饼饼有44GB的芯片SRAM，每层只有16GB，所以我们通常能够直接将模型的整个层装在芯片上。
这意味着在多数情况下，模型重量从来不需要移动，层层也不需要被分割成瓦片。 只有从一个饼饼到另一个饼饼的活化物
#4：我们如何满足一个请求
现在，我们把蜡烛连接在一起，并铺设了模型，我们可以把每块蜡烛看作是一个装配线上的一阶段：模型对每个阶段的重量保持不变，请求的中间结果贯穿了各层阶段。
例如，Wafer 1运行了1–2的模型层，并发送其激活到wafer 2. Wafer 2运行了3–4层，并将结果传递给Wafer 3，等等，直到最后一个wafer产生下一个标志。
#5：我们如何满足许多要求
这个融资是一个常见的误解。 每个饼饼在等待轮到的时候，实际上，系统不是以一个批量大小运行吗？
没有！如果我们一次向下发出一个请求，等待它完成然后开始下一个请求，这将是对管道的低效利用。 相反，我们保留整个管道充满不同的要求。 由于模型已经分为管道阶段，每个阶段可以同时处理不同的请求。
第3阶段处理请求A的代号，第2阶段处理请求B的代号，第1阶段处理请求C的代号。 在我们的例子中，每个阶段都是一层。 在生产中，Cerebras使用许多精细的加磨阶段，还有更多的。
每个阶段还可以处理一个微批，一个小组的请求，而不是一个单一的请求。 因此飞行中的请求数量大致是级数乘以微批大小。
为了理解取舍，较小的微管使每个请求更快，而较大的微管则增加总吞吐量。
#6：我们能满足多少请求？
分配给 KV 缓存的芯片内存量决定了同时请求的最大数量。 飞行中的每一个请求都需要自己在每个密集层的瓦片上的KV缓存，因此芯片上存储器的大小为同时提出的请求设定了上限。 如上所述，我们只需要计算密层的KV缓存要求。
对于已经有一个密集层的44GB卷轴，我们可以用：
总圆饼存储器 - 一个密集层的大小。 在这种情况下：
计算同时请求的 #
为了找到任何特定模型映射支持的并行请求数量，我们首先计算一个256k请求要处理多少缓存：
然后计算此配置将支持的并行请求总数，我们除以：
同时提出的申请总数是任何时间通过管道提出的同时提出的申请数目。 这些请求与GPU批次不同
当然，无论是货币还是批量大小，都无法确切地告诉你一个推论系统是如何高效运行的。 在评价一个推论系统的吞吐量时，更有意义的衡量标准是每单位功率的总信使吞吐量(如每兆瓦每秒的信使)，当测量输出速度相同时。
如何超越3T参数
同样的方法也扩展到更大的模型：计算重量和请求状态所需的内存，将模型分布在饼饼之间，并将其连接到一条管道中。 每个额外的饼饼都贡献了计算和芯片记忆，使我们能够容纳超过任何单一芯片容量的模型。
谢谢你跟我们学的有兴趣在Cerebras上部署模型还是用我们的推论云建造？
- 开始我们的免费API：云.cerebras.
- 联系我们的销售团队：销售@cerebras.ai
由哈雷·张设计。 感谢阿塔尔瓦·塔尔帕德，高振伟，安吉拉·杨，杰西卡·刘，乔伊斯·埃尔，艾玛·卡爾，以及罗汉·古普塔的审查和反馈。
本文由 AI图鉴自动抓取并翻译，版权归原作者所有；如需查看原文请点下方链接。
One of the most frequently asked questions we get is how we deploy large models on Cerebras wafer-scale chips. A single Cerebras WSE-3T wafer has 44 GB of on-chip memory, and trillion-parameter frontier models need terabytes for their weights alone.
In this post, we’ll explain how we split a large frontier model across multiple WSE-3T wafers, to run multi-trillion parameter models at 1000-2000+ tokens per second.
We’ll use the open source model Kimi K3, with ~3 trillion parameters, as an illustrative example.
By the end, you'll understand:
- How we fit the full model into on-chip memory
- How we map the model across multiple chips
- How we handle a high # of concurrent requests
Breaking it down with Kimi K3
Kimi K3 is an open source, 3T parameter model with 69 linear-attention layers and 24 dense layers.
It scores an impressive 88.3% on Terminal-Bench 2.1 (vs. GPT-5.6 Sol's 88.8%) and 91.2% on BrowseComp (vs. GPT-5.6 Sol's 90.4%). These results demonstrate frontier-level coding and agentic capabilities, even outperforming leading proprietary models on some benchmarks. Read more about Kimi K3 here.
For this illustrative example, we use:
- A 256k assumed average request size
- 4-bit weight quantization (like the Hugging Face implementation)
- An 8-bit KV cache
#1: Size of Each layer
To calculate the total number of Cerebras WSE-3T wafers we need to support Kimi K3, let’s first calculate the size of each model layer.
Kimi K3 has 3 trillion parameters. If all 3T parameters are stored at 4 bits each, the raw parameter payload is:
Dividing that payload by 93 layers then gives:
#2: Number of Wafers
Now that we know each layer needs about 16 GB, we can figure out how many layers fit on each 44 GB wafer. The answer depends on the layer type:
- Dense layers need additional room for a KV cache, because each dense (full-attention) layer stores the keys and values for every token in every active request. So, we place one dense layer per wafer and leave the remaining SRAM for its KV cache.
- Linear-attention layers do not require additional room for a KV cache. Instead of storing keys and values for every token, a linear-attention layer keeps a small, fixed-size state per request, regardless of context length. That leaves room for two linear layers per wafer (2 × 16 GB = 32 GB).
Therefore, with Kimi K3’s 69 linear layers and 24 dense layers, we get:
This gives us a total wafer count of 35 + 24 = 59 wafers.
#3: Connecting the wafers together
Cerebras executes models in a pipeline. After the Cerebras compiler maps the model layers across regions of wafers, the runtime maps the path for tokens to flow between wafers. For ultrafast latency, only activations need to be passed between the wafers, and this can be done via RDMA over Ethernet. This is roughly 2 microseconds per hop.
It’s important to emphasize that because each Cerebras wafer has 44 GB of on-chip SRAM and each layer is only 16 GB, we are typically able to fit entire layers of the model directly on the chip.
This means that in most cases, the model weights never have to move, and layers don’t need to be split across wafers. Only activations cross from one wafer to the next.
#4: How we serve 1 request
Now that we have our wafers connected together and the model laid out, we can think of each wafer as one stage in an assembly line: the model’s weights for each stage stay in place, and the request’s intermediate results move through the layer stages.
For example, Wafer 1 runs model layers 1–2 and sends its activations to wafer 2. Wafer 2 runs layers 3–4 and passes its results to Wafer 3, and so on until the last wafer produces the next token.
#5: How we serve many requests
This raises a common misconception. If a token has to travel through each wafer in order, doesn’t each wafer sit idle while it waits for its turn? In effect, wouldn't the system be running at a batch size of one?
No! If we only sent one request down the line at a time and waited for it to complete before starting the next request, this would be an inefficient use of the pipeline. Instead, we keep the entire pipeline full with different requests. Because the model is split into pipeline stages already, each stage can work on a different request at the same time.
While stage 3 processes a token for request A, stage 2 processes one for request B, and stage 1 one for request C. In our example, each stage is a layer. In production, Cerebras uses much finer-grained stages, and many more of them.
Each stage can also process a micro-batch, a small group of requests, rather than a single request. So the number of requests in flight is roughly the number of stages times the micro-batch size.
To understand the tradeoff, smaller micro-batches make each request even faster, while larger micro-batches increase total throughput.
#6: How many requests can we serve?
The amount of on-chip memory allocated to KV cache determines the maximum number of concurrent requests. Every request in flight needs its own KV cache on each dense-layer wafer, so the size of on-chip memory sets the ceiling on concurrent requests. As explained above, we only need to calculate KV cache requirements for dense layers.
For each 44 GB wafer that already has one dense layer, we can calculate the remaining space for the KV cache with:
Total wafer storage - size of one dense layer. In this case:
Calculating the # of Concurrent Requests
To find the number of concurrent requests supported for any particular model mapping, we first calculate how much cache a single 256k request takes up:
To then calculate the total number of concurrent requests this configuration would support, we divide:
The total number of concurrent requests represents the number of simultaneous requests moving through the pipeline at any point in time. Unlike a GPU batch, those requests don’t wait for one another to finish, they stream through the pipeline continuously.
Of course, neither concurrency nor batch size tells you exactly how efficiently an inference system performs. When evaluating the throughput of an inference system, the more meaningful metric is total token throughput per unit of power (e.g. tokens per second per megawatt), when measured at the same output generation speed.
How this scales beyond 3T parameters
The same approach extends to larger models: calculate the memory needed for weights and request state, distribute the model across wafers, and connect them into a pipeline. Each additional wafer contributes compute and on-chip memory, allowing us to accommodate models that exceed the capacity of any single chip.
Thanks for learning with us! Interested in deploying a model on Cerebras or building with our inference cloud?
- Get started with our free API: cloud.cerebras.ai
- Reach out to our sales team: sales@cerebras.ai
Designs by Halley Chang. Thanks to Atharva Talpade, Zhenwei Gao, Angela Yeung, Jessica Liu, Joyce Er, Emma Call, and Rohan Gupta for their review and feedback.
Auto-fetched and translated by AI Atlas. All rights belong to the original author.