Rack-scale AI：戴尔和AMD如何建设基础设施。
Dell Technologies Inc. 及其合作伙伴Advanced Micro Edities Inc. 面临类似的竞争压力。 这两座技术发电站都在商业环境下运作，现在通过中央处理单元、图形处理单元、架式系统、内存能力、网络带宽和每美元信使来衡量成功与否。
在这些措施之间竞争的压力凸显了CUBE研究所称的“工程速度 ”。 自从5月的戴尔技术世界活动以来，AMD和戴尔并没有放慢合作和公告的速度。
AMD和Dell正在努力建设企业AI基础设施，能够跟上不断变化的工作量和客户需求。
称是CUBE研究的Dave Vellante。 “工程系统比任何其他人更快地学习、验证和改进整个平台，这更可能是一个公司。”
这个特征是SiliconANGLE Media对建筑转向动力连续，生产级AI的探索的一部分。 请检查CUBE的「Dell AI数据平台事件：规模化的从野心到AI」, (* 下文披露。 )
小规模合并
Dell和AMD关系的基石是服务器巨头的PowerEdge组合。 戴尔于5月扩展了其PowerEdge服务器线，并随后又增加了AMD处理器集成的细节。
由于代理AI推动对通用计算的需求激增，这两家公司采取了一项旨在促进服务器合并的战略。 Agents严重依赖CPU，然而数据中心却没有太多额外空间
由AMD处理器提供动力的系统将带来70%的基因超原性能收益，并且可以将戴尔的第14个电源Edge世代的13个服务器折叠成一个，据戴尔的产品策划者和高级顾问Pradyumna Narendra说。
“每更换10到13个服务器，你就会得到如此多的生产率收益，因为现在你可以用一个服务器做到这一点，” 他告诉了CUBE。 “当你在数据中心中扩展它，这就像一个加速效应，你可以利用它。”
戴尔和AMD还必须满足大量的AI使用案例和伴随的计算需求。 戴尔的三机架单元服务器冷却所需的功率比两机单位等效的低，让客户将更多的计算功率装入每个机架，根据纳伦德拉的说法。
他解释说，“通过部署包体略大一些的服务器，在机架级别上交付密度更高，有时似乎反感”。 “随着大赦国际的加速、激化和代理性AI的出现，它将会加快和加快增长。 随着这些使用案例的扩大，计算需要的指数增长。”
模块部署和验证
除了服务器整合，戴尔和AMD一直在稳步加强模块化解决方案，以简化AI部署和缩放：戴尔AI平台与AMD.
立体企业AI解决方案将AMD Instinct加速器和EPYC处理器与Dell PowerEdge服务器，PowerSwitch网络和PowerScale存储器集成，用于模型训练，微调和高通量推论。 该平台旨在让客户从满足当前工作量并随时间推移扩展计算和GPU密度的配置开始，而无需重新抓取。
客户想要模块化解决方案。 称 Varun Chhabra在与CUBE讨论时，“计算、存储、联网[和]GPU。 上面的软件框架，模型：他们都接受了测试吗？是否都经过验证？”
Dell将AMD ROCm软件堆栈整合到平台中，支持培训，微调，并在-premise上服务大型语言模型。 根据Chhabra的说法，模块化平台让客户开始较小的部署，并在同样的基础设施上扩展。
“你可以从这些堆肥的存储、计算、联网、预先与AMD Instinct和EPYC CPU结合到ROCm软件以及其上方的开放生态系统中开始，” “我们测试它，我们验证它，客户可以使用它，一旦它们开始看到它的价值，它们就会知道它们可以用同样的框架以模块方式进行规模化。”
利用 PCIe 的好处
Dell 宣布在其PowerEdge XE7745和R7725服务器中支持AMD Instinct MI350P PCI Express GPU，从7月开始，为企业提供了另一种缩放代理和基因AI部署的选项。 这也代表了内存带宽的重要提升，这在时间到时间到时间到时间到时间到时间到时间到时都很重要。
MI350P提供144千兆字节的HBM3E内存，根据业务发展高级管理人员Instinct Data Center GPUs在CUBE上出现时的描述，这可以是推论性能的关键差异。
他的称表示，“主要用例是推论：这都是暂时性的
AMD和戴尔在PCIe整合上的合作还有另一个值得注意的角度。 AMD基于PCIe的GPU支持空气冷却部署，为用户提供了一个在现有的数据中心基础设施内部署AI的路径。
由于近几个月来技术行业在全美国建立大型数据中心方面遇到阻力，人们关注的中心是环境影响和使用水冷却系统。 另外，戴尔发现，当它部署下一代的PowerEdge服务器时，一些企业客户仍然没有准备好在数据中心内采用液冷。
“我们与戴尔合作，在GPU Instinct的可空气冷却路线图中增加了PCI卡，使之适合更多Dell PowerEdge服务器，这些服务器将进入更广泛的企业，”称 Robert Hormuth在与CUBE谈话时，担任位于AMD的数据中心解决方案组的架构和战略的公司副总裁。 “并不是每个人都采用直接液体冷却。”
开放市场的合作
Dell今年与AMD的合作已超出联合产品公告的范围。 6月，两家公司宣布与剑桥大学结成伙伴关系，在联合王国建立一个主权AI创新实验室。
合作旨在帮助推进下一代AI基础设施和AI带动的科学突破。 它以剑桥扩展的AI研究基础设施为基础，包括由第五代AMD EMYC处理器和AMD Instinct MI355X GPU加速器组成的Zenith AI超级计算机，集成于戴尔基础设施。
Dell和AMD最近几个月还就电信倡议与AT&T公司进行了合作。 电讯数据对于AI来说是一种重载运，AT&T已经处理过超过一兆兆的符号来训练模型。 AMD为通过Open Telco AI倡议开发的OTel 2.0模型的培训和推论工作量提供了高性能硬件基础。
Dell将这一技术与用于电信环境的AI-优化服务器相结合，为跨数据密集、真实世界网络环境的运营商创造了生产级基础。 OTel 2.0的训练基础设施使用戴尔的PowerEdge XE9785服务器，配置了8台MI355X GPU.
Dell和AMD的联盟凸显了不断发展的AI基础设施市场的关键发展：竞争机会很多。 Dell和AMD似乎都致力于建设在AI处理解决方案中提供企业选择的组合。
“AMD没有将它的新产品。作为 " 值得尊敬的替代品 "。 “它声称直接领导CPU、GPU、机架级性能、内存能力、网络带宽和每美元信物。 它将这一论点扩展到企业AI、机器人、超低相关性推论和迅速扩展的伙伴生态系统。”
(* 披露：CUBE是戴尔AI数据平台事件的付费媒体合作伙伴。 CUBE活动报道的发起人Dell或其他发起人对CUBE或SiliconANGLE上的内容都没有编辑控制。 )
图片：SiliconANGLE/ChatGPT
SiliconANGLE联合创始人John Furrier的留言：
支持我们的使命，通过与CUBE社区接触，保持内容的开放和自由。 加入CUBE的校友信托网络，其中技术领导者相互连接，分享情报，创造机会。
- 15M+观看CUBE视频，为AI、云层、网络安全等各种对话提供动力
- 11.4k+CUBE校友——通过独特的信任网络与11400多名技术和企业领袖建立联系，塑造未来
你是AWS客户吗？在财务上支持SiliconANGLE，从我们的市场门户页面和链接购买您的AWS服务：https://siliconangle.com/aws-marketplace/.
关于 SiliconANGLE 媒体
SiliconANGLE Media由技术视觉家约翰·弗瑞尔(John Furrier)和戴夫·韦兰特(Dave Vellante)创立，构建了产业领先数字媒体品牌的动态生态系统，达到1500+百万精英技术专业人士。 利用CUBEAi.com神经网络，帮助科技公司以数据为导向做出决策
本文由 AI图鉴自动抓取并翻译，版权归原作者所有；如需查看原文请点下方链接。
Rack-scale AI: How Dell and AMD are building infrastructure
Dell Technologies Inc. and its partner Advanced Micro Devices Inc. face similar competitive pressures. Both tech powerhouses operate in a business climate where success is now measured across central processing units, graphics processing units, rack-scale systems, memory capacity, network bandwidth and tokens per dollar.
The pressure to compete across those measures underscores what theCUBE Research has termed “Engineering Velocity.” Since the Dell Technologies World event in May, AMD and Dell have not slowed their pace of collaborations and announcements.
AMD and Dell are working to build enterprise AI infrastructure that can keep pace with changing workloads and customer requirements.
“In AI infrastructure, the winner may not be the company with the best benchmark on keynote day,” said theCUBE Research’s Dave Vellante. “It will more likely be the company whose engineering system learns, validates and improves the entire platform faster than anyone else.”
This feature is part of SiliconANGLE Media’s exploration of the architectural shifts powering continuous, production-grade AI. Be sure to check out theCUBE’s “Dell AI Data Platform Event: From Ambition to AI at Scale” and its past coverage of this year’s Dell Technologies World. (* Disclosure below.)
Rack-scale consolidation
A cornerstone of the Dell and AMD relationship is the server giant’s PowerEdge portfolio. Dell expanded its PowerEdge server line in May and has followed that up with additional details surrounding the integration of AMD processors.
The two companies have pursued a strategy designed to facilitate server consolidation as agentic AI drives a surge in demand for general-purpose compute. Agents rely heavily on CPUs, yet data centers don’t have a lot of extra room to add servers in rooms already stacked with GPUs.
Systems powered by AMD processors will deliver 70% gen-over-gen performance gains and can fold 13 servers from Dell’s 14th PowerEdge generation into one, according to Pradyumna Narendra, product planner and senior consultant at Dell.
“For every 10 to 13 servers you replace, you get so much of that productivity gain back because now you can do that with one server,” he told theCUBE. “When you expand that across data centers, that’s like an accelerating effect that you get to capitalize from.”
Dell and AMD must also accommodate a wealth of AI use cases and the accompanying demand for compute. Dell’s three-rack-unit server requires less power for cooling than a two-unit equivalent, allowing customers to pack more computing power into each rack, according to Narendra.
“By deploying servers that are slightly larger in package, you deliver more density at a rack level, which seems sometimes counterintuitive,” he explained. “As AI accelerates, inferencing and agentic AI around it will accelerate and grow faster. As these use cases scale out, you get exponential growth in the need for compute.”
Modular deployment and validation
Beyond server consolidation, Dell and AMD have been steadily enhancing a modular solution to simplify AI deployment and scaling: the Dell AI Platform with AMD.
The on-premises enterprise AI solution integrates AMD Instinct accelerators and EPYC processors with Dell PowerEdge servers, PowerSwitch networking and PowerScale storage for model training, fine-tuning and high-throughput inference. The platform is designed to let customers begin with configurations that meet current workloads and expand compute and GPU density over time, without rearchitecting.
“What we’re finding is customers want modular solutions. They want to think about [solutions] across the whole platform,” said Varun Chhabra, senior vice president of product marketing and infrastructure solutions group at Dell, in a discussion with theCUBE. “Compute, storage, networking [and] GPU. The software framework on top of it, the models: Have they all been tested? Have they all been validated?”
Dell has integrated the AMD ROCm software stack into the platform to support training, fine-tuning and serving large language models on-premises. The modular platform lets customers begin with smaller deployments and expand on the same infrastructure, according to Chhabra.
“You can start small with these composable units of storage, compute, networking, pre-integrated with AMD Instinct and EPYC CPUs with the ROCm software as well as the open ecosystem that sits on top of it,” he said. “We test it, we validate it, and the customers can use it and once they start seeing value in it, they know they can scale in a modular way with the same frameworks.”
Leveraging benefits of PCIe
Dell announced support for AMD Instinct MI350P PCI Express GPUs in its PowerEdge XE7745 and R7725 servers beginning in July, giving enterprises another option for scaling agentic and generative AI deployments. It also represents an important boost in memory bandwidth which matters in time-to-first-token latency.
The MI350P offers 144 gigabytes of HBM3E memory, which can be a key differentiator for inference performance, according to Mike Darby, senior manager of business development, Instinct Data Center GPUs, at AMD, during an appearance on theCUBE.
“It’s a lot of memory, but it’s also HBM3E, so [it has] an amazing amount of bandwidth,” he said. “The main use case being inference: It’s all about that latency, time to first token [and] the user experience, so that your attention span doesn’t move somewhere else while you’re waiting for your response.”
AMD and Dell’s collaboration on PCIe integration has another angle worth watching. AMD’s PCIe-based GPU supports air-cooled deployments, providing users with a path to deploy AI within existing data center infrastructure.
As the tech industry has encountered resistance in recent months to the buildout of large data centers across the U.S., concerns have centered on the environmental impact and use of water-based cooling systems. Separately, Dell found that, as it deployed its next-generation PowerEdge servers, some enterprise customers were still not prepared to adopt liquid cooling within data centers.
“Working with Dell, we added a PCI card to our air-coolable roadmap for GPU Instinct so that it fit in more Dell PowerEdge servers that go to the broader enterprise,” said Robert Hormuth, corporate vice president of architecture and strategy for the Data Center Solutions Group at AMD, during a conversation with theCUBE. “Not everybody is adopting direct liquid cooling.”
Collaboration in an open market
Dell’s work with AMD this year has extended beyond joint product announcements. In June, the two companies announced a partnership with the University of Cambridge to establish a Sovereign AI Innovation Lab (SAIL) in the United Kingdom.
The collaboration is designed to help advance the next generation of AI infrastructure and AI-powered scientific breakthroughs. It builds on Cambridge’s expanded AI research infrastructure, including the Zenith AI supercomputer, powered by fifth-generation AMD EPYC processors and AMD Instinct MI355X GPU accelerators integrated into Dell infrastructure.
Dell and AMD have also collaborated in recent months on a telecom initiative with AT&T Inc. Telecom data is a heavy lift when it comes to AI, with AT&T having processed over one trillion tokens to train models. AMD has provided the high-performance hardware foundation for both training and inference workloads involving the OTel 2.0 models developed through the Open Telco AI initiative.
Dell integrated this technology with AI-optimized servers for telecom environments to create a production-grade foundation for operators across data-intensive, real-world network environments. OTel 2.0’s training infrastructure uses Dell’s PowerEdge XE9785 server configured with eight MI355X GPUs.
The alliance between Dell and AMD highlights a key development in the growing AI infrastructure market: There are plenty of opportunities for competition. Both Dell and AMD appear committed to building portfolios that offer enterprise choices in AI processing solutions.
“AMD didn’t present its new products … as ‘respectable alternatives,’” Vellante said. “It claimed outright leadership across CPUs, GPUs, rack-scale performance, memory capacity, network bandwidth and tokens per dollar. It extended the argument across enterprise AI, robotics, ultra-low-latency inference and a rapidly expanding partner ecosystem.”
(* Disclosure: TheCUBE is a paid media partner for the Dell AI Data Platform Event. Neither Dell, the sponsor of theCUBE’s event coverage, nor other sponsors have editorial control over content on theCUBE or SiliconANGLE.)
Image: SiliconANGLE/ChatGPT
A message from John Furrier, co-founder of SiliconANGLE:
Support our mission to keep content open and free by engaging with theCUBE community. Join theCUBE’s Alumni Trust Network, where technology leaders connect, share intelligence and create opportunities.
- 15M+ viewers of theCUBE videos, powering conversations across AI, cloud, cybersecurity and more
- 11.4k+ theCUBE alumni — Connect with more than 11,400 tech and business leaders shaping the future through a unique trusted-based network
Are you an AWS customer? Support SiliconANGLE financially by buying your AWS services from our Marketplace portal page and links: https://siliconangle.com/aws-marketplace/
About SiliconANGLE Media
Founded by tech visionaries John Furrier and Dave Vellante, SiliconANGLE Media has built a dynamic ecosystem of industry-leading digital media brands that reach 15+ million elite tech professionals. Our new proprietary theCUBE AI Video Cloud is breaking ground in audience interaction, leveraging theCUBEai.com neural network to help technology companies make data-driven decisions and stay at the forefront of industry conversations.
Auto-fetched and translated by AI Atlas. All rights belong to the original author.