过去二十年，服务器峰值浮点算力每两年增长约3倍，而DRAM带宽每两年只增长1.6倍，互连带宽仅1.4倍，这就形成了行业内称之为存储墙的痛点，而这个问题其实已经存在很久，而如今的AI时代，随着计算能力的增强及其对数据需求量的扩大，据中国信通院统计，中国每天平均的Token消耗量，从2024年的1000亿，跃升到2025年8月的500万亿，一年多时间增长了约5000倍。在此背景下，存储墙问题也被放大了。
推理时代，算力与存储必须加强融合
大模型推理的核心流程之一，是自回归解码，模型每生成一个Token，都要回头看一遍之前所有的上下文。如果每次都重新计算全部历史数据，算力会被海量的重复计算消耗殆尽。于是KV Cache诞生了，它把中间计算结果缓存起来，用“以存换算”的方式跳过重复计算。这个机制极大地减少了计算开销，但它同时埋下了一颗雷：数据越攒越多，而缓存总得有地方放。
这个问题随着Token消耗量的提升在不断被放大，对此，联想集团首席科学家肖利民告诉我们，过去典型的AI应用是一问一答的对话机器人，如今变成为能够感知、会规划、能执行任务的数字员工。实际上，每一次Agent执行任务，背后都是动辄数十轮、上下文动辄数万Token的推理调用。
而推理，恰恰是对存储要求最高的环节。KV Cache的存放问题，本质上是一场空间挤压战，数据如放全部在GPU显存，显存通常不够用，进而人们想到将数据卸载到CPU直连内存，但CPU直连内存容量同样扛不住。因此，数据很自然地再往外卸到外部存储，但是GPU无论走PCIe或网络去访问外部存储，带宽和延迟与访问本地显存相比，存在数量级的差距，“数据在外部存储和GPU之间搬运的开销，往往会拖累推理性能。”肖利民指出。
在此背景下，推理基础设施就出现了两个痛点。一个是KV可用的GPU显存和CPU直连内存容量不足，一个是GPU访问外部存储的开销太大。业界现有的主流方案是KV Cache 卸载到外部共享CXL内存池，但该方案主要解决前一个问题，通过CXL交换机，把内存资源与CPU解耦，实现独立弹性扩容和多主机共享。但对后一个痛点，也就是GPU访问CXL内存池开销大问题，始终力不从心。
实际上，更底层的核心问题是存储硬件与计算硬件的发展速度不匹配。计算性能的提升速度远快于存储，而且这个差距还在逐年拉大。这就是计算机体系结构领域经典的存储墙问题，而这个问题从来没有被完全真正解决过，人们用多级缓存等各种手段不断缓解它，但缓解不等于消除。大模型推理的爆发，不过是把这道旧墙，重新又推到了所有人的面前。
于是，存算融合的思路呼之欲出：既然把数据搬到算力那边成本太高，那就反其道而行之，把计算单元搬到数据旁边去。与其让海量的KV数据经过PCIe链路在GPU和外部存储之间奔波，不如让计算单元驻扎在内存旁边，就地处理数据，只把计算结果传回来。
对此，肖利民给出的路径是，把内存池化技术和近存计算技术结合在一起，构建一套面向大模型推理的存算融合池化系统，用内存池化解决KV Cache的大容量扩容和跨实例共享，用近存计算在内存池内嵌计算单元，把 attention这类访存密集型算子下沉到内存池一侧，从整体上降低端到端的推理时延、提升系统吞吐。
在肖利民看来，这条路线并不算激进，“技术的发展通常是逐步演进的，”他进一步指出，“现阶段算力基础设施出现断崖式重写的可能性不大，但推理负载带来的这些新变化，确实需要更多创新技术被引入到推理算力基础设施里面去，存算融合池化技术就是这个演进过程中一个非常重要的技术点。”
平衡点在哪？
从当前存算融合技术路径上看，主要分为CIM（存算一体）、PIM（存内计算）、以及PNM（近存计算）三条路径。前两者通常把计算单元直接做到了存储芯片内部，带宽极高、延迟极低，这种做法在某种意义上已经算是存算一体了，但这通常需要改变现有存储器本身的结构，属于高度定制化的硬件，与现有的软件生态难以兼容。对任何一个要考虑产品化落地的团队来说，这都是一道需要攻克的坎。
有鉴于此，在肖利民看来，不改动存储器本身结构的PNM是当下较为现实的路径。对此，肖利民表示，把计算单元放置在存储器的边上、内存控制器的位置，既拉近了计算与存储的距离，又不改动存储器原有的结构，从源头上维持了与现有软硬件生态的兼容性。这个选择背后是清醒的商业判断：“我们未来要真正落地做产品化实现，兼容现有生态的路径会有很大的优势。”肖利民解释道。
但在实现近存计算的产品化路上，还有很多挑战。其中肖利民认为最大的挑战有两个。第一个是近存计算单元放置策略选择上的两难。计算单元要既离存储足够近，又不对现有生态产生大的改变，放在哪里本身就是一道难题。选定了内存控制器附近之后，新的问题接踵而至，内存控制器原本就承担着来自主机的访存请求，如今旁边又多了一个近存计算单元的访存请求，两路请求同时涌向存储器，冲突不可避免。团队的解决办法是在内存总线上内置一套 QoS 仲裁机制，对两路不同的访问进行仲裁调度，既避免冲突，又保证两边的服务质量不会互相拖累，还能维持一定的公平性。“存储总线的速率高，如何做冲突检测，如何保证两边的QoS不往下掉，同时还要保证公平性，确实是个较为复杂的问题。”肖利民强调道。
第二个是算子相对固化与模型快速迭代之间的矛盾。大模型几乎每隔几个月就要更新换代，而硬件计算单元相对固化。联想采用的是指令驱动的可重构计算编排思路，让同一套硬件通过指令驱动算子编排适配不同模型的计算需求，缓解了这个矛盾。
除此之外，在软件实现和负载运行层面，还要面临如何用统一编址实现算力统一纳管和高效调用？多机共享内存池时如何保证数据一致性？以及一次推理请求到来时，哪些计算该下沉到内存池侧、哪些留在GPU本地算、KV该不该卸载等问题，这些问题都需要寻找到一个最佳的平衡点。据肖利民介绍，联想研究院智能计算基础设施团队为此建立了一个成本分析模型，根据链路拓扑、上下文长度等输入动态决策，在编排策略上按头切分KV并与GPU张量并行策略对齐，尽量避免设备之间的通信，提高设备并行计算效率。
当所有问题都找到解决的平衡点之后，联想团队进行了测试，肖利民介绍，他们在三个核心模块上进行了分项性能测试。在内存池卸载方面，与NVMe层池化方案对比，在固定输入长度场景下，首Token延迟（TTFT）可降低3.5倍；多轮对话场景下可降低2.4倍，且随着对话轮次增多，优势愈发明显；在KV跨实例共享方面，借助内存池实现KV复用，相比完全重算，首Token延迟最高加速12~28倍；相比RDMA分布式内存架构，也有2~3倍的加速；在存算融合方面，把注意力计算下沉到内存池侧后，解码延迟可降低2.5倍，同时腾出大量显存，用以支持更长上下文、更大规模的模型。为验证对大模型推理的整体加速效果，联想团队还进行了端到端推理性能测试。在推理延迟方面，与RDMA分布式内存池相比，取得了2~3倍的加速；在系统吞吐方面，性能提升更为可观，随着上下文变长，系统吞吐提升6~20倍。
从G2.5到G1.5是接下来要做的事
肖利民团队目前实现的是G2.5层次的基于内存池化和近存计算技术的存算融合池化系统。但肖利民告诉我们，接下来从G2.5向G1.5演进将会是重点研发的方向之一。
具体来说，其思路是借助超级节点里scale-up高速互连技术，把共享存算融合池进一步向GPU靠拢，从G2.5推进到G1.5层次，进一步缩短访问路径。“一旦超节点成为AI系统主流，利用它里面scale-up高速互连，不仅享有G2.5已获得的收益，而且在计算与存储的互连带宽、延迟上，都会比现有互连网络有一个数量级的提升。”肖利民说。在硬件上，3D堆叠封装、TSV垂直互连等技术逐渐成熟后，也有望进一步提升存算之间的带宽密度；软件上则会持续扩展算子库，适配更多主流模型。肖利民同时强调，随着芯片新技术的日益成熟，团队会在“存算融合”理念下会吸纳更多新的芯片技术，使得推理性能在现有水平上再提升一个层次。
在肖利民看来，联想存算融合池化技术的差异化特色，主要在于将内存池化和近存计算相结合，同时解决容量不足和访存开销大这两大难题。“目前国内外相关工作大多聚焦于怎么做出一个大容量内存池，解决KV Cache容量不足的问题；或者，通过存算融合技术，缓解访存开销大的问题”肖利民进一步指出，“目前暂未见到把近存计算引入到大容量内存池里，同时KVcache解决容量不足和访存开销大两个难题的产品。”。
当然，算力基础设施技术的演进并不会一蹴而就，目前阶段出现类似AI技术的“GPT时刻”的可能性不太大。在肖利民看来，在算力基础设施中，计算、存储、通信、调度多种要素交织在一起，相互融合、协同优化，“某一块跑快了，另一块拖后腿，然后被加快，再又换一块成了短板，有点像是按下葫芦浮起瓢。” 肖利民进一步表示，“我们的存算融合池化技术不是要终结这样一种局面，而是在大模型推理负载把存储墙问题推到舞台中央之后，通过近存计算靠近大内存池的方式，来加速大模型推理性能。实际上，更为理想的方式，不仅局限于存储和计算的融合，而是围绕计算、存储、通信、调度多种要素做系统性的协同优化，以实现算力基础设施及其上运行大模型推理应用的更大性能提升。”
本文由 AI图鉴自动抓取并翻译，版权归原作者所有；如需查看原文请点下方链接。
Over the past two decades, server peak floats have increased roughly threefold every two years, while the DRAM bandwidth has increased only 1.6 times every two years, and only 1.4 times the bandwidth, creating what is known as the pain of the storage wall in the industry, a problem that has been in existence for a long time, and the current AI era, with increased computing capacity and its increased demand for data, has shown that China's average daily consumption of Token rose from 10 billion in 2024 to 50 billion in August 2025, an increase of about 5,000 times over the year. In this context, the problem of storage walls has also been magnified.
In the age of reasoning, it is important to combine computation and storage.
One of the core processes of LLM ' s reasoning is the self-regression decodement, and each of the models produced by Token is to look back at all the contexts before. If all historical data are recalculated each time, the ability to calculate is depleted by the double counting of the volume. So KV Cache was born, and it brought up the intermediate calculation cache, skipping the double calculation by “Stocking convert”. This mechanism has greatly reduced the cost of computing, but at the same time it has laid a mine: the more the data, the more cache has to be placed.
In response to this problem, which has been amplified with the rising consumption of Token, the Chief Scientist of the United Nations Think Group, Shalimin, told us that the typical AI application in the past was a question-and-answer dialogue robot, which has now become a digital workforce capable of understanding, planning and performing tasks. In fact, every time Agent carried out a mission, it was followed by dozens of rounds and tens of thousands of Tokens.
The reasoning is precisely the highest level of storage requirements. The storage of KV Cache is essentially a space-crowding battle, and data such as all in GPUVRAM, VRAM, is usually not sufficient, leading to the idea of offloading the data to the CPU direct memory, which is equally unaffordable. As a result, data are naturally removed from external storage, but whether the GPU goes to PCI or the network for access to external storage, there is a quantitative gap between bandwidth and delay compared to access to local VRAM. “The cost of moving data between external storage and GPU tends to slow down reasoning performance.” As pointed out by Schorimin, the data are not available to the public.
Against this background, there are two painful points in the reasoning infrastructure. One is the lack of direct memory capacity of GPUZ82Z and CPU available to KV, and one is the excessive cost of GPU access to external storage. The current mainstream industry programme is KV Cache unloaded to an external shared CXL memory pool, but the programme primarily addresses the former problem by decoupling memory resources with CPUs through CXL switches, and achieving independent flexible expansion and multi-host sharing. But the latter point, the GPU ' s access to the CXL memory pool, has been a major problem.
In fact, the central problem at the lower end is the mismatch between storage and computing hardware. Computation performance increases much faster than storage, and the gap widens year by year. This is the classic storage wall problem in the architecture of computer systems, which has never been completely and truly resolved, and which is constantly being mitigated by a variety of means, such as multi-level cache, but which is not equivalent to elimination. The outbreak of LLM ' s reasoning was simply to push the old wall back to everyone.
So, the idea of integration is to move the data to the other side of the calculation, which is too expensive to do, to move the computing unit to the next side of the data. Instead of allowing large KV data to travel through the PCIe link between GPU and external storage, the computing unit should be located next to the memory, processing the data in situ, and only passing the results back.
In response, Shaolin gave the path of combining memory poolization techniques and close-in computing techniques, building an integrated storage system for LLM reasoning, using memory poolization to resolve KV Cache ' s large-capacity build-up and cross-example sharing, close-up calculation of memory embedded computing units, sinking access-intensive calculations such as attention to one side of the memory pool, lowering the whole end-to-end reasoning latency, and insulating the system.
In the view of Shalimin, this route is not radical, “technology development is usually gradual,” he further states, “it is unlikely that there will be a cliff-breaking rewriting of the computing infrastructure at this stage, but these new changes brought about by the reasoning load do require that more innovative technologies be introduced into the reasoning algorithm, and that the storage and integration poolization technology is a very important technical point in this evolution”.
What's the balance point?
In terms of current storage integration technology pathways, the main lines are CIM, PIM and PNM. While the first two typically directly capture the computing unit within the chip, with very high bandwidth and very low delay, which is already in a sense integrated, this usually requires a change in the structure of the existing repository itself, which is highly customized hardware and is not compatible with the existing software ecology. For any team that considers productization to land, this is an attack.
In the light of this, the PNM, which does not change the structure of the memory itself, is, in the view of Shalimin, a more realistic path in the present. In response, Shalimin stated that the location of the computing unit on the side of the memory controller had brought the calculation closer to the storage, without altering the original structure of the memory and maintaining compatibility with the existing soft and hardware ecology from the source. Behind this choice is a sober business judgment: “We have to be truly product-oriented in the future, and there is a great advantage in a path that is compatible with existing ecology.” Shaley explains.
However, there are many challenges in achieving the productization of near-stock calculations. Of these, Shaolimin saw two as the greatest challenges. The first is the dilemma in the policy selection of close-to-exposure computing units. Where they are placed is a challenge, both in close proximity to storage and without significant changes to existing ecosystems. Once the memory controller has been selected, new problems follow, and the memory controller, which was already carrying a request for access from the host, is now accompanied by an additional request for access to the close-deposit computing unit, with two-way requests pouring into the memory. The team's solution is to place a QoS arbitration mechanism on the memory bus to arbitrate different visits, both to avoid conflict and to ensure that the quality of services on both sides does not delay each other and to maintain some fairness. “The high rate of storage of the bus, how to detect the conflict, how to ensure that the QoS on both sides does not fall, while also ensuring fairness, is a more complex issue.
The second is the contradiction between the relative solidification of algorithms and the rapid iterativeity of models. LLM is updated almost every few months, while the hardware computing unit is relatively solid. This contradiction is mitigated by the proposed command-driven, re-engineered approach to computing, which allows the same set of hardware to fit the computing needs of different models through command-driven algorithms.
In addition, at the level of software realization and load operation, how can one achieve uniform algorithms and efficient calls with a unified site? How can data consistency be ensured when multiple memory pools are shared? And when a reasoning request comes, the questions of which calculations should sink to the side of the memory pool, which should remain at the GPU local count, and whether KV should not be unloaded need to find an optimal balance. According to Shaolimin, the ICMI team has developed a cost analysis model for this purpose, based on dynamic input decisions such as chain tripping and context length, aligning the organization strategy with a head-to-head KV and GPU-size parallel strategy, avoiding as much as possible communication between equipment and increasing the efficiency of parallel computing.
After all the problems had been resolved, the Consortium team had conducted a test, a presentation by Shalimin, and they had performed sub-performance tests on three core modules. For the unloading of the memory pool, compared to the NVMe layer coilization programme, the first token delay (TTFT) could be reduced by 3.5 times in a fixed input length scenario; 2.4 times in a multi-wheel dialogue scenario, and with the increase in the number of dialogue rounds, the advantage becomes more evident; for the sharing of the KV cross-cases, KV can be reused using the memory pool, with a maximum acceleration of 12-28 times for the first token delay compared to a full recalculation; there is also an acceleration of 2-3 times in comparison with the RDMA distributed memory structure; and, in terms of storage integration, decodering can be reduced by 2.5 times when attention is sunk to the inside side of the pool, while large amounts of VRAM have been released to support longer context, larger models. To verify the overall acceleration effect of the LLM reasoning, the Consortium team also performed end-to-end reasoning performance tests. In terms of the delay in reasoning, the acceleration was 2-3 times greater than in the RDMA distributed memory pool; in the case of system throughput, the performance was much more significant, increasing 6-20 times as context became longer.
From G2.5 to G1.5 is what's next.
The team is currently implementing a G2.5-level memory pool-based and near-deficit computing system. But Shawliten told us that the next move from G2.5 to G1.5 would be one of the main directions of research and development.
Specifically, the idea is to further align the shared memory integration pool to the GPU, moving from the G2.5 to the G1.5 level, with the use of scale-up high-speed interconnection technology at the super node. “When the supernodes are mainstreamed into the AI system, using the scale-up high-speed interconnectivity inside it, not only enjoy the G2.5 gains already made, but also add an order of magnitude to the existing interconnectivity network in calculating the bandwidth and delaying storage. On hardware, the technological maturity of 3D stacks of stacks, vertical TSV interconnections, etc., is also expected to further increase the bandwidth density of the savings; on software, it will continue to expand the algorithm and adapt to more mainstream models. At the same time, it was stressed that, as new chip technologies matured, the team would incorporate more new chip technologies under the concept of “compatibility of memory”, enabling reasoning to rise to another level at the current level.
“Most of the current work at home and abroad has focused on how to create a large-capacity memory pool to address KV Cache’s lack of capacity; or, by means of CIF, to mitigate the high cost of visiting storage.” Shalimin further states that “no nearness calculations have been introduced into a large-capacity memory pool at this time, while KVcache has been able to address two difficult problems.
Of course, the evolution of computing infrastructure technology will not take place overnight, and the possibility of having an AI-like “GPT moment” at this stage is less likely. In the view of Shaliman, in the computing infrastructure, the various elements of computing, storage, communication and movement are interwoven, integrating and synergizing. “One block runs fast, the other drags behind, and is then accelerated and replaced by a short sheet, which is sort of like a tumble-up. “Our storage and integration pool technology is not intended to put an end to such a situation, but rather to accelerate the LLM reasoning function by close-in calculations of the size of the large memory pool, after the LLM reasoning load pushed the storage wall to the center of the stage. In fact, a more desirable approach would be not just to integrate storage and computing, but to achieve a systematic synergetic optimization around multiple factors of computing, storage, communication, and movement in order to achieve greater performance in the computing infrastructure and in the application of the LLM reasoning on which it operates.”
Auto-fetched and translated by AI Atlas. All rights belong to the original author.