如果把科研计算想象成一条流水线,那么过去几十年,这条流水线的节奏是由人来把控的,科研人员找到工具、写脚本、提交作业、盯着排队、排错重跑,最后再把结果一格格归纳出来。算力本身很强大,但喂给算力任务的方式却始终停留在手搓时代。
阿里云高性能计算&性能加速负责人游亮告诉我们,在当前的科研流程中,数据准备、环境配置、任务编排、结果分析等环节,80%以上的时间仍然依赖人工流转,真正用于计算执行的时间不到20%。
超智融合已是必然趋势
随着AI大规模进入科研流程,过去以HPC高精度计算为主的工作负载,如今叠加了AI训练、AI推理、智能体的通用计算,甚至还有智能体沙箱计算。
近年的诺贝尔奖已经出现了多个与AI直接相关的获奖成果,AI不再是科研的辅助工具,而是逐步进入观测、模拟、猜想与实验的各个环节,成为科研流程的关键组成部分。与此同时,传统HPC的高精度科学计算和AI模型的训练推理,正在同一个研究项目里并行出现,同一份研究,既离不开高精度的物理仿真,也离不开大规模的模型训练。IDC与浪潮信息联合发布的《中国人工智能计算力发展评估报告》显示,2024年中国智能算力规模达到725.3 EFLOPS,同比增长74.1%,远高于通用算力20.6%的增幅;该报告预计到2028年中国智能算力规模将达2781.9 EFLOPS,长期保持高速增长。IDC发布的另一份智算服务市场跟踪报告也显示,2024年上半年中国智算服务市场同比增长近八成,其中生成式AI基础设施服务增速超过200%。
在游亮看来,多种计算类型同时涌向同一个算力平台,传统HPC架构很难让它们同时做到性能最优、成本最低。更麻烦的是调度,传统HPC的调度逻辑是以作业为单位的,Slurm或PBS知道一个任务要多少核、多少内存、预计跑多久,但它不知道这个任务在科学上要解决什么问题。一个生物信息学研究者提交一个变异检测作业,调度器看到的是一个计算作业,看不到它处在一个从FASTQ到候选突变的完整链条里的哪一环,失败了应该换工具还是调参数,结果出来之后下一步该跑什么,这些判断全部留给人。
而现在,系统要同时调度HPC作业、AI训练推理、智能体任务和沙箱任务,这些负载的特征差异极大,高精度仿真吃双精度浮点性能和内存带宽,AI训练吃互联带宽和显存容量,Agent的工具调用和沙箱任务吃的是轻量、快速启停和弹性。传统HPC集群按照单一负载设计的架构,做不到让每一类计算都跑到最优,此时,传统的单一种类调度方式显然撑不住了。
如今,科研的流程正在从人驱动工具,走向人定目标、智能体自主执行,而这个新流程对算力的要求是:既要能跑HPC,又要能跑AI,还要能跑智能体,且必须统一调度、统一计量。谁能把这三种胃口不同的负载喂到一张桌子上,谁就拿到了下一代科研基础设施的入场券,而这也是超智融合得以被推到台前的原因。
超智融合难在哪?
超智融合看上去是将超算的服务器和智算的服务器连接起来,协同工作,但真做起来并不是那么简单。超智融合最直观的难点在精度和数据流动。气象预报、流体仿真、分子动力学通常需要FP64双精度来抑制累积误差,大模型训练和推理大量使用FP16、BF16甚至FP8,而Agent的沙箱和工具调用又涉及大量碎片化、短生命周期的轻量计算。传统算力体系将高精度计算与低精度计算割裂分离,难以满足混合精度计算、AI4S等复合型业务需求。精度不是“选一个档位”的问题,而是同一条科研链条上要在不同档位之间无缝切换。
游亮与我们分享了阿里云的解法,他表示,具体可以拆成三层来看。
第一层,让不同的算力各得其所,又连成一张算力网络。游亮介绍,阿里云的思路是面向不同的工作负载提供最匹配的实例类型。比如,面向传统HPC,提供HPC优化实例,面向AI训练和推理分别提供对应的AI加速实例,面向智能体提供通用计算和容器实例等。关键在于,这些异构实例通过云上统一的虚拟网络串联成一张“算力网络”,并借助CIPU 2.0架构实现无虚拟化损失的资源调度,再通过统一的开放OpenAPI,让所有算力可以被统一创建、调度和编排。
在CPU层面,阿里云打造了hpc9a,其搭载第五代AMD EPYC(Turin-C)处理器,最高睿频5GHz,128个物理核心,内存可达3TB,依托CIPU2.0架构消除虚拟化损耗,配合200G的低时延eRDMA网络,相比上一代性能提升约30%。
游亮介绍,hpc9a专门面向CAE仿真、电子设计自动化这类对计算性能、内存带宽和网络时延都极度敏感的场景。AI侧则交出了平头哥自研芯片的成绩单:M890P相比上一代性能提升至3倍,新一代V900又在此基础上再提升至3倍性能。配合超节点架构和弹性算力,E-HPC可以为科研场景提供从单卡到大规模集群的AI加速能力。
在智算方面,阿里云则在自研芯片的道路上有所收获,其推出的M890P相比上一代性能提升至3倍,新一代V900又在此基础上再提升3倍。配合超节点架构和弹性算力,E-HPC可以为科研场景提供从单卡到大规模集群的AI加速能力。
性能与成本的平衡靠两条腿走路。一是多代实例并存,客户按工作负载选择性价比最高的实例规格;二是云的弹性,客户在业务波峰需要上千台实例,波谷可能只需要几十台,传统模式必须按峰值预留资源,云上则可以按需创建和销毁,让性能与成本同时逼近最优。游亮与我们分享了一个案例:阿里云与国内某头部乘用车公司深入合作,通过HPC优化实例、弹性eRDMA低延迟网络,叠加CPFS并行存储优化,端到端帮助客户节省了约25%的计算周期,大幅提升了工作效率。
第二层是让数据跑得比算力快。算力再强,数据喂不上就是空转。游亮介绍,eRDMA负责让节点之间沟通速度越来越快,CPFS负责让HPC计算、分布式AI训练和推理等负载,用最快的时间拿到数据,让计算饱和运转,把昂贵算力的浪费降到最低。存储侧的优化包括冷热分层,把常用数据放进缓存层,使用时不必再从远端存储拉取,以及让网络邻近、存储邻近的计算节点做亲和性调度。
第三层是让智能体替人跑完执行流程。围绕超算与智算持续融合、AI在科研中作用攀升、人工流程成为瓶颈等趋势,阿里云推出了E-HPC智能体平台,其架构从上到下分为四层:最上层是面向具身智能、生命科学、CAE、EDA、量化、渲染等行业的智能体;行业接入层把行业知识和专家方法沉淀为Skills,供智能体调用,同时提供命令行、开放API和SDK;中间是统一的Agent Runtime,负责理解科研目标、规划任务、用skills实际执行、依据日志和结果反馈再规划;往下是E-HPC执行层,统一承担作业编排、异构资源调度、环境与数据准备以及状态日志管理,并调度CPU、GPU、eRDMA网络、镜像软件与安全治理资源。
通过搭建这三层架构,目前阿里云已经在超智融合方面有所收获。在具身智能场景,RoboAgent打通了端侧数据采集、数据处理标注、模型训练推理、远程仿真到部署回传的完整链路,一站式降低数据准备成本;某具身领域头部企业借助这套方案,实现了从真机采集上传、多模态标注处理到QwenVL等多类算法训练的快速闭环。
在生命科学场景,BioAgent根据研究目标自动编排WDL工作流,自主识别FASTQ、PDB等数据格式并推荐分子模拟参数,调用GATK、GROMACS等工具并智能匹配CPU/GPU资源,让生信与药研流程可复用、可解释、可持续迭代;最具代表性的是上海科学智能研究院旗下格物智研的“浑天绫”,这个编排出140多个科研skills的物质科学智能研发平台,与阿里云E-HPC的弹性算力底座配合,把分子筛选任务从数据检索、算力调用、分子对接、FEP计算到自动化合成,这些以前由人工串联的环节,现在全部由智能体自主完成。
在CAE场景,工程师只需聚焦设计判断,前处理、求解、诊断、后处理与参数探索由Agent自主贯通。
要实现这些也并不是一蹴而就的,游亮也坦言,最大的挑战在于不同行业对算力、精度、实现方式的需求各不相同,需要把各行业的专家经验和行业知识沉淀为skills,同时把E-HPC在调度、编排、任务识别、资源匹配上的专家能力也沉淀为平台skills,再通过统一的Agent Runtime承接两者,并解决行业skills与E-HPC skills的协同、海量算力拉起、行业负载与算力负载最佳匹配等难题。他的总结是:行业Agent可以非常稳定、弹性、可观测、可治理地调用阿里云的HPC服务,而这背后需要反复打磨每一层。
更值得关注的是,E-HPC已经不只是通用算力平台,而是针对芯片设计、AI4S与生命科学、具身智能三个重点行业形成了专属解决方案。面向芯片设计,E-HPC混合云可以无缝对接企业现有的LSF、Slurm、PBS调度环境,云上云下统一调度、按队列自动扩缩容,工程师无需改变使用习惯,就能在仿真验证高峰随时补齐算力;针对STA、PR等大内存作业,大内存HPC实例可以直接承接,帮助芯片团队守住流片周期、优化整体拥有成本。面向AI4S与生命科学,E-HPC预集成AlphaFold3、OpenFold3、RFDiffusion、GROMACS、GATK等主流科研软件,开箱即用,客户自有的科研Agent还能通过CLI、API或Skill直接调用弹性算力,把药物发现、基因分析从繁琐的环境搭建中解放出来。面向具身智能,HPDA Robotics将采集、管理、标注、训练、仿真、推理串成一条完整数据闭环,内置APAK加速引擎,多模态数据处理的性价比可提升2.1至5倍,显著缩短数据准备与模型迭代周期。三套专属方案共享同一张弹性算力网络和同一套调度、存储与安全能力,既能开箱即用,也能平滑接入客户现有环境。
智能体将成为科研的操作系统
通过阿里云所搭建的三层架构,可以看出当前超智融合的发展路径。技术侧,算力会继续变大,传输通道会继续变长。 游亮透露,在Scale Up方向上,阿里云上超节点由几十到上百颗AI芯片组成,未来超节点支持更大规模;在Scale Out方向上,通过弹性RDMA网络实现跨节点、跨可用区的互联。
超节点本质上可以被看作一个巨大的AI算力节点,其规模上限的持续抬升,将直接决定大规模训练与仿真的效率。与此同时,eRDMA与传统RDMA的区别在于它能承载更广泛的互联,从超节点内部延伸到跨可用区场景,这正是“算力网络”想象的物理基础。
应用侧,智能体将从可选项,变成行业标配。阿里云 E-HPC的产品逻辑已经从单点智能体走向体系化:各个行业Agent负责各自领域的专业执行,WorkAgent则把分散在各行业的智能体统一收敛成一套目标驱动的执行闭环,科研人员只需要说清“算什么、要多准、多久出结果、花多少钱”,WorkAgent负责翻译成计算任务和验收标准,自动拆解流程、引入专家经验、匹配算力网络与存储资源,最后解析日志收敛曲线与关键指标,定位失败并给出改进方向。“人只需要给目标、做判断,重复的工作流程交给智能体自主完成。这一步的意义在于:企业不必重复建设行业智能体和算力底座,而是像水电一样接入。”游亮如是说。
阿里云是超智融合领域最具代表性的企业之一,但并不是唯一家。传统超算阵营里,国家超算中心与硬件厂商正沿着各自平台推进HPC与AI方法的深度融合;AI厂商则以智算集群和万卡训练平台为主攻方向;运营商大规模建设智算中心。IDC数据显示智算集成服务市场正以超过150%的增速扩张,市场格局远未定型。
值得注意的是,各家的打法虽不同,但方向高度一致:让HPC与AI在同一个平台里协同工作,让智能体成为科研的默认操作系统。
本文由 AI图鉴自动抓取并翻译,版权归原作者所有;如需查看原文请点下方链接。
If scientific calculations are conceived of as a streaming line, the rhythm of this streaming line has been controlled by humans over the past few decades, and scientists have found tools, scripts, submitted jobs, lined up, ran back wrongly, and then summarized the results. The calculus itself is powerful, but the way to feed the calculus is still in the hands.
Alibaba Cloud High Performance & Accelerated Performance Monitors tell us that in the current scientific process, data preparation, environmental configuration, tasking, results analysis, etc., are still over 80 per cent of the time dependent on manual flows and less than 20 per cent of the time actually spent on calculating implementation.
It's an inevitable trend.
As AI moved into the scientific process on a large scale, the workload, which used to be based on HPC high-precision calculations, now overlapped with AI training, AI reasoning, AI agent generic calculations and even AI agent sandbox calculations.
In recent years, the Nobel Prize has produced several results directly related to AI, which is no longer an aid to scientific research, but a step-by-step approach to observations, simulations, guessworks and experiments, becoming a key component of the scientific process. At the same time, the traditional HPC high-precision scientific computation and AI model training reasoning are emerging in parallel with the same research project, which involves both high-precision physical simulation and large-scale modelling training. The IDC and Wave Information Joint publish Report on the Assessment of China ' s Artificial Intelligence Computer Development shows that China ' s intellectual calculus size reached 725.3 EFLOPS in 2024, an increase of 74.1 per cent over the same period, well above the General Calculus ' 20.6 per cent increase; the report projects that China ' s intellectual calculus size will reach 2781.9 EFLOPS in 2028, with sustained high growth. Another IDCpublish market tracking report on smart services also shows that the market for smart services in China grew by almost 80 per cent in the first half of 2024, with generating AI services growing at a rate of over 200 per cent.
In the light of the light, the various types of calculations flow simultaneously to the same calculator platform, and it is difficult for the traditional HPC architecture to allow them to perform at the same time with the highest performance and lowest cost. Even more problematic is movement control, the traditional HPC movement logic is operational, and Slurm or PBS knows how much core, how much memory and how long it is expected to run for a mission, but it does not know what the task is to solve scientifically. A bioinformatics researcher submitted a mutation detection exercise, and the dispatcher saw a calculator that could not see which link it was in in a complete chain from the FASTQ to the candidate mutation, failed to change the tools or adjust the parameters, and as a result, next step ran away, leaving all these judgements to people.
Now, the system is running HPC operations, AI training reasoning, AI agent missions and sandbox missions, which have very different characteristics, with high precision imitating double-precision floating point performance and memory bandwidth, AI training on interconnectivity bandwidth and VRAM capacity, and Agent ' s tool call and sandbox missions eating light, fast start and elasticity. At a time when traditional HPC clusters are structured according to a single load that does not allow each type of calculation to run to its best, it is clear that the traditional single type of movement is unsustainable.
Today, the scientific process is moving from a human-driven tool to a human-targeted, AI agent-self-executed process, and the new process demands that both HPC and AI should be able to run and AI agent must be centrally programmed and measured. Those who can feed these three different appetites to a table receive tickets to the next generation of scientific infrastructure, which is why super-intellectual integration has been pushed to the stage.
What's the problem with super-wise integration?
Super-wise integration seems to connect super-calculable servers to smart servers, working together, but it's not that simple. The most intuitive difficulty of hyper-intellectual integration lies in precision and data flow. Weather forecasting, fluid imitation, molecular dynamics usually require a FP64 double accuracy to contain cumulative errors, LLM training and reasoning make extensive use of FP16, BF16 and even FP8, while Agent's sandbox and tool call involve large amounts of debrisation and short life-cycle light calculations. The traditional computing system separates high-precision calculations from low-precision calculations and makes it difficult to meet complex business needs such as hybrid precision calculations and AI4S. Precision is not a question of “selecting a slot”, but a seamless transition between different slots in the same scientific chain.
We shared the solution of Alibaba Cloud, which he said could be detached into three layers.
The first level, where different algorithms come together, is a network of algorithms. The idea of Alibaba Cloud is the type of example that provides the best match for different work loads. For example, for the traditional HPC, examples of HPC optimization, examples of AI acceleration for AI training and reasoning, examples of generic calculations and containers for AI agent, etc. The key is that these isomeric examples are linked to a “calculative network” through a unified virtual network on the cloud, and that a non-virtually lost resource allocation is achieved through the CIPU 2.0 architecture, and that all algorithms can be created, deployed and organized in a unified open-ended OpenAPI.
At the CPU level, Alibaba Cloud has developed hpc9a, which carries the fifth generation AMD EPYC (Turin-C) processor, with a maximum of 5 GHz, 128 physical cores with memory of up to 3 TB, which eliminates virtualization losses based on the CIPU 2.0 framework, and the 200G low latencyeRDMA network, which increases performance by about 30% compared to the previous generation.
The groove presentation, hpc9a, is dedicated to scenarios such as CAE simulation, electronic design automation, which are extremely sensitive to computing performance, memory bandwidth and network latency. On the side of AI, a report card was handed over from Hiranko's self-study chip: M890P increased three times the performance of the previous generation, and on this, the new generation V 900 increased three times. E-HPC can provide an AI accelerator capability from single card to large-scale clusters for the scientific scene, in conjunction with hypernomous structures and elasticity.
In terms of intellectuals, Alibaba Cloud has gained some ground on the path of self-study chips, with M890P being introduced three times more than the performance of the previous generation, and a third. E-HPC can provide an AI accelerator capability from single card to large-scale clusters for the scientific scene, in conjunction with hypernomous structures and elasticity.
The balance between performance and cost is walking on two legs. The first is the multigenerational example of the client ' s specification of the highest selective price for the work load; the second is the elasticity of the cloud, where the client needs thousands of examples at the peak of the business, where only a few dozen are needed for the valley, where traditional models must reserve resources at the peak, and where the cloud can be created and destroyed as needed, allowing performance to be brought closer to cost at the same time. We shared a case: Alibaba Cloud has worked in depth with a front-line car company in the country to significantly improve efficiency through HPC optimization, flexible eRDMA low-delayed network, superseding CPFS parallel storage optimization.
The second level is to make data faster than calculations. If you can't feed the data, it's empty. The ERDMA is responsible for making communication between nodes faster and the CPFS is responsible for making HPC computing, distributed AI training and reasoning loads available as soon as possible, allowing the calculation to saturate, and minimizing the waste of expensive calculations. The optimization of the side of the storage includes cold thermal layers, the placement of commonly used data into the cache layer, the need to use them without being drawn from remote storage, and the use of the adjacent network to store adjacent computing nodes for relative and sexual movement.
The third level is for AI agent to complete the execution process. The E-HPCZ AI agent platform was launched by Alibaba Cloud and is structured on four levels: The top tier is AI agent, which is oriented towards the smart, life sciences, CAE, EDA, quantization, rendering, etc.; the industrial access layer deposits industrial knowledge and expert methods into Skills, which is available for AI agent call, while providing command line, Open API and SDK; the middle is Agent Runtme, which is responsible for understanding research objectives, planning tasks, actual execution of skills, re-planning based on logs and feedback of results; and the E-HPC implementation layer, which is the E-HPC implementation layer, which integrates work organization, isomer resource movement, environment and data preparation, and state log management, and moves CPU, GPU, eRDMA network, mirror software and security governance resources.
Through the construction of these three-tier structures, Alibaba Cloud has now achieved some success in hyper-intellectual integration. In the case of a smart scene, RoboAgent had access to end-end data acquisition, data-processing labelling, model training reasoning, a complete link of remote simulation to deployment back-to-back, and all-in-one had reduced the cost of data preparation; using this option, a front-end enterprise in a particular field had achieved rapid closure for multi-type algorithm training such as uploading from a real machine, multi-state labelling processing to QwenVL.
In the life sciences scene, BioAgent automatically organizes WDLworkflow according to research objectives, autonomously identifies data formats such as FASTQ, PDB and recommends molecular simulation parameters, uses tools such as GATK, GROMACS and intelligently matches CPU/GPU resources to allow re-use, interpretability and sustainability of credit and pharmaceutical research processes; the most representative of these are the “mixed sky”, which was developed under the umbrella of the Shanghai Institute of Smart Sciences, and which has a physical science intelligence development platform of more than 140 scientific skills, in conjunction with the elastic base of Alibaba CloudE-HPC, and which uses molecular screening tasks from data retrieval, calibration, molecular interface, and FEP calculations to automated synthesis, which were previously artificially linked and are now all autonomous by AI agent.
In the CAE scenario, engineers need only focus on design judgement, and pre-processing, solvency, diagnosis, reprocessing and parameter exploration are autonomously linked by Agent.
To achieve these, which are not instantaneous, and to be frank, the greatest challenge lies in the fact that the needs of different industries for computing, precision and delivery vary, requiring the deposition of expert experience and industry knowledge in various industries into skills, as well as the deposit of E-HPC expertise in movement control, organization, mission recognition and matching of resources into platforms to skills, to be followed by a unified Agent Runtime, and to address such challenges as industry skills and E-HPC skills synergies, the build-up of big and industry loads. His conclusion was that the industry Agent could be very stable, flexible, visible, and manageable to call on HPC services in Alibaba Cloud, and that this would require grinding every layer behind it.
Of even greater concern is the fact that E-HPC is no longer just a universal calculator platform, but has developed exclusive solutions for the three priority sectors of chip design, AI4S and life sciences, and intelligence. For chip-oriented design, the E-HPC hybrid cloud provides a seamless connection to the existing LSF, Slurm, PBS movement environment in the enterprise, unified movement under the cloud, automatic build-up of the queue, and engineers can complete their calculations at any time without changing their usage habits, at a simulation-validation peak; for large memory operations such as STA, PR and so forth, large memory examples of HPC can be taken directly, helping the team to maintain the flow cycle and optimize overall ownership costs. For AI4S and life sciences, the E-HPC pre-assessed AlphaFold3, OpenFold3, RFDiffusion, GROMACS, GATK, etc., are used in boxes, and Agent, the client-owned researcher, can directly use elastic computing through CLI, API or Skill to free drug discovery and genetic analysis from cumbersome environmental construction. For intelligence-oriented purposes, HPDA Robotics will collect, manage, label, train, imitate and delineate into a complete data closed loop, with an APAK acceleration engine built in, multi-modular data processing with a value increase of 2.1 to 5 times, significantly reducing data preparation and model iterative cycles. The three exclusive programmes share the same flexible calculator network and the same set of movement control, storage and security capabilities, which can be used both to open boxes and smooth access to the client ' s existing environment.
AI agent will be an operating system for scientific research.
The three-tier structure set up by Alibaba Cloud shows the current path of hyper-intellectual integration. On the technical side, the calculator will continue to grow and the transmission channel will continue to grow. Swim revealed that, in the Scale Up direction, the Alibaba Cloud hypernodes consist of dozens to hundreds of AI chips that will support a larger scale in the future; and in the Scale Out direction, cross-nodes, interconnections across available areas through the flexible RDMA network.
Supernodes, by their very nature, can be seen as a huge AI calculator node, and the constant increase in their size ceiling will directly determine large-scale training and imitation of real efficiency. At the same time, the difference between eRDMA and the traditional RDMA lies in its ability to carry wider interconnections, from inside the hypernodes to across-the-horizon scenarios, which are the physical basis of the “calculative network” imagination.
Application side, AI agent will transform from option to industry formulation. The product logic of Alibaba Cloud E-HPC has moved from single point AI agent to systematization: Agent in various industries is responsible for professional implementation in their respective fields, and WorkAgent has condenses the AI agent harvests scattered across industries into a set of target-driven implementation closed loops. Scientists simply have to say, “What to count, more accurate, how long to produce results, how much to spend”, WorkAgent is responsible for translating into computational tasks and acceptance criteria, self-dismantling processes, introducing expert experience, matching computing networks and storage resources, and finally deciphering the tectonic curves and key indicators, identifying and giving direction for improvement. “A person needs only to be targeted, to judge, and the duplicate workflow process is left to AI agent. The point of this step is that an enterprise does not have to repeat the construction of the industry AI agent and the substrate of power, but rather access as water and electricity.
Alibaba Cloud is one of the most representative enterprises in the area of hyper-intellectual integration, but is not the only one. In the traditional super-calculations camp, the National Super-calculator Centre and hardware manufacturers are moving forward with the deep integration of HPC and AI methodologies along their respective platforms; the AI manufacturer is targeting the IQ cluster and the Vanka training platform; and operators are building large-scale IQ centres. The IDC data indicate that the market for smart integrated services is expanding at a rate of over 150 per cent, and that market patterns are far from fixed.
It is worth noting that there are different approaches, but very different directions: HPC and AI work together on the same platform and AI agent become the default operating system for scientific research.
Auto-fetched and translated by AI Atlas. All rights belong to the original author.