OCT. 8, 2026 (英语).
Google AI Edge Team兴奋地宣布开源发布ML Drift，我们的高性能，跨平台，端侧 GPU 计算引擎专门为端侧 AI/ML推论建造，在Apache 2.0许可下。 通过抽象硬件以及跨越OpenGL ES,OpenCL,Metal，以及WebGPU的端侧 GPU的低级API复杂性，ML Drift授权开发者构建实时，从高级视频效果到跨多个平台的基因AI的交互ML体验。 作为LitERT内部的核心GPU加速引擎，ML Drift也作为自定义图形和推论运行时间的独立库提供，为各地提供峰值性能提供统一的基础。
ML Drift 的 GPU API 支持矩阵跨平台
与数据中心推论不同的是，模型运行在同质，可预测的加速器集群上，将GPU-加速AI部署到边缘设备是由显著的硬件多样性定义的。 开发者面临着广泛的GPU架构，驱动程序版本，以及低级API，不知道应用程序将运行于哪个特定的硬件的先验。
TensorFlow Lite GPU代表建立了基础GPU加速，但生态系统后来有所演变：目前的端侧工作量现在包含一个宽谱，从实时计算机视觉，音频，深度处理模型到高参数的基因AI. 这些建筑将消费硅推向极限，制造出遗留的运行时间无法解决的计算和内存瓶颈。
这种转变需要有一个普遍的工程基础，以确保古典和下一代模型的可移植性和性能。 借鉴"Speed is all you need"中所讨论的优化原理，我们构建了一个单一，统一的框架，为传统的ML提供稳定性，为最先进的基因AI提供峰值性能，确保你的模型运行在我们最高效，最前瞻性的GPU计算引擎上。
为了在古典建筑和基因建筑中实现峰值性能和扩大模型覆盖，ML Drift 推出核心建筑变化和结构升级都超越了遗留下来的TFLite GPU代表。
通过Tensor虚拟化实现的统一阴影：历史上，为TFLite GPU代表维护优化的阴影，需要硬编码逻辑的拉伸绘图，直接通过OpenGL,OpenCL和Metal后端分别对物理GPU对象(如纹理和缓冲)进行硬编码。 ML Drift 推出 lamor virtualization，一种核心建筑范式，将一个lamor的逻辑代表与其在GPU上的物理分配脱钩。 通过在编译器初始化阶段拥有动态的遮荫器模板解决和翻译坐标，这种统一的遮荫器模型消除了维持不同，后端特定遮荫器代码库的需要，在保留跨平台模型可移植性的同时增加了最小的运行时间管理费。
可扩展自定义 Op框架：ML Drift采用现代化自定义 Op框架，提供低水平阴影语言访问的直接注册API. 为加快速度，ML Drift包含一个代理SKILL.md指南，允许编码agents在几分钟内编写，注册，并验证性能自定义阴影。 这为开发者提供了颗粒控制，可以直接将专门的模型块集成到执行图中，极大地减少了部署专有架构的障碍。
5D Tensor Support:TFLite GPU代表器结构硬码为4D的百分位数，迫使开发人员对需要5D百分位数的复杂模型使用布局黑客。 ML Drift在新的LiteRT ML Drift GPU加速器中反映了这个早就应该使用的功能，以启用5D 的 lasonor 支持出箱。 有了这个，LitERT执行复杂的工作量，如3D的演化网络，用于量子/空间AI和spatiopal模型(如YOLO 11n,MobileViT v2,Swin Transformer v2)，直接在边缘GPU上。
经典模型的性能升级：作为我们遗留下来的GPU后端的直接继承者，ML Drift为现有的古典工作量提供了严格的性能升级。 通过将运行时间与硬件特定限制脱钩，提高内核执行效率，已经运行在TFLite GPU上的模型获得了即时的性能改进。 移徙设计得直截了当，遗留的工作量维持或超过以前的基准测试。
阶段感知优化边缘LLMs：自动递归LLMs在推论中有两个不同的计算工作量：计算约束的KV缓存预填阶段和内存带宽的解码阶段，逐个生成令牌。 货币根据活动执行阶段，漂移动态切换内核和布局配置。 在解码阶段，它使用自定义，折叠式的KV缓存布局，并应用主动性的内核激活量子化绕过冗余内存圆盘。
预填512，解码128，上下文长度640,4位重(块大小32),fp16 KV缓存，argmax采样，同步每个令牌。
预填(固条；左键)和解码(厚条；右键。)的吞吐量被呈现出来。
Gemma4 E2B在支持跨KV共享的框架上表现较好。
跨平台扩展( Desktop 预览)：在当今新兴的代理编码时代，高度响应的桌面推论正变得对为本地编码agents和工作站工作流程提供动力至关重要。 虽然我们最初为浏览器内加速度设计了ML Drift的WebGPU后端，但Dawn(Chrimium的WebGPU执行)允许我们在浏览器外编译出完全相同的编码库。 这个统一的API模型使得ML Drift能够运行在Windows和Linux上，利用WebGPU的现代本土硬件抽象来绕过DirectX和Vulkan破碎，同时补充我们现有的高性能本土金属在macOS上的后端。
我们的首要重点仍然是在资源限制最紧的地方提供轻量级生产推断：移动和边缘设备。 然而，地方发展需要各开发商机器的灵活性。 以下是ML Drift在工作站硬件上运行的Gemma模型的早期快照，说明我们的统一运行时间尺度如何跨越环境。
桌面 LLM 性能。 预填8192，解码1024，上下文长度9216
4位重(块大小32), fp16 KV缓存，argmax采样，同步每个令牌。
跨越边缘和工作站环境，最高吞吐量只讲述了一半的故事：记忆足迹决定了哪些工作量可以同时运行。 在我们的Gemma 基准测试中，ML Drift实现了比其他框架低12%的内存管理费，将内存释放出来用于其他本地任务。
ML Drift已经每天在数百万个设备上运行，为Google生态系统上的关键功能提供动力，包括Chrome,YouTube Shorts,Photos，和Meet. 在前沿，它充当了开拓端侧能力的运行时间，为Google AI Edge apps提供动力，如 Gallery, Foresight，和Eloatent.
YouTube Shorts将基于分区的效应迁移到使用ML Drift. 这在Android和iOS两地的平均帧延时性上实现了高达40%的降低，使创作者能够实时应用高真实度的视觉效果而无需放下相机帧。
为了提供快速，无缝的端侧照片编辑，Google Photos团队将ML Drift纳入其计算摄影和分区管道。 与遗留下来的GPU代表相比，这提供了最多2秒的速度，使先进的照片增强感觉瞬间。
Google 通过ML漂流加速照片Unblur
Google Chrome将ML Drift整合到其AI运行时基础设施中，为Built-In APIs(包括即时，即时，即时，即作家APIs)的本地硬件加速Gemini Nano模型提供动力。 通过桌面平台，ML Drift 实现了高效端侧 AI，跨越网络应用不断增长的生态系统
Cafe24 使用 Express API 来通过 ML Drift 加速标记生成。
除了Google产品外，主要开发方合作伙伴还采用了ML Drift，为移动用户带来桌面级能力：
Adobe灯光室和Adobe Photoshop升级了关键AI功能，包括Select Object,Select Sky，以及适应性肖像，配有ML Drift，在手机上实现最快30%的端侧性能，用于支持质量的图片编辑。
Adobe 灯光室通过移动LitERT ML Drift GPU加速器运行端侧
Snap 集成的LitERT为GPU通过Android设备的端侧推论提供动力。 ML Drift GPU加速器为Snapchat镜头中的ML动力面部和样式生成器效果带来了30%的模型延迟性改进。 货币漂流不仅可以改善基于现有模型的用户体验，还可以提高扩展运行甚至大型扩散模型的能力。
实现边缘AI效率最大化需要软件架构与物理硅之间的深度对接。 在整个ML Drift开发过程中，Google与硅和IP合作伙伴密切合作，实现优化，使硬件在现代GPU的利用最大化：
- 武装：协同优化马里和Immortalis GPU的计算内核和纹理缓存位置，确保高通量、高功率的推论跨移动设备。
- Intel:ML Drift利用本地WebGPU计算和Xe Matrix扩展(XMX)来加速客户端LLM和Intel® CoreTM Ultra处理器上带有Xe3图形的视觉工作量。
- Qualcomm Technologies, Inc.:ML Drift优化了OpenCL内核，用于顶峰ALU和内存带宽利用，使得Qualcomm® AdrenoTM GPU上的有效边缘AI加速。
ML Drift通过与硅领导者携手工作，确保最先进模型优化转化为电池意识，高帧率在用户设备上的性能。
随着ML Drift的发布，遗留下来的TFLite GPU代表将不再收到新的功能更新。 我们鼓励所有开发人员向 LiteRT ML Drift GPU 加速器过渡，该加速器为您现有的模型提供完全的反向兼容性，同时立即解锁跨越 Android, iOS, Web，和桌面的现代性能收益。 对于Android开发者使用未捆绑的运行时间来最小化应用二进制大小，ML Drift加速在今天以独立的LiteRT软件包提供，并即将来到Google Play Services的LiteRT.
无论您正在将端侧模型集成到一个应用程序中还是正在编写自定义的 GPU 阴影器中，ML Drift 提供了两个直接路径：
- 对于应用程序开发者：使用LitERT ML 漂流加速器，获得出箱模式分区，拉诺虚拟化，并优化LLM加速。
- 对于GPU & System Engineers：通过ML Drift GitHub寄存器与ML Drift的独立C++ API直接构建，以构建自定义的‘GpuModel'计算图表并写入自定义的遮蔽器(通过Dawn快速启动开始在Android和WebGPU上的OpenCL).
社区反馈和贡献
ML漂流在Apache 2.0许可下为全开源。 我们欢迎各方的贡献、区域渔业委员会和社区反馈：
- 来源代码：github.com/google-ai-edge/ml-drift
- 问题跟踪器和特性请求：在 GitHub 上文件一个问题
- 讨论 & QQA: Google AI边缘社区论坛
- 贡献：审查我们的贡献指南，增加新的UCL操作器或硬件特定优化。
承认：
安德烈·库利克、安基特·戈雅尔、鲍勃·刘、金秉哲、陈冠英、长明孙、克里斯·麦克兰纳汉、克拉克·杜瓦尔、科马克·布里克、丹尼尔·霍、戴维·内托、埃卡捷琳娜·伊格纳谢瓦、冯武·姚、弗兰克·班、杰拉尔多·卡兰扎、格兰特·詹森、雅各布·多尔穆思、贾加德什·帕卡拉沃尔、詹姆斯·普莱斯、杰里米·肯普、静江·李、唐继强、开尔文·马、肯·罗素、肯吉·巴赫、林·陈、洛科·昆、卢旺、玛丽萨·伊科诺米迪斯、马特·克雷莱德尔、马蒂亚斯·格伦德曼、马蒂亚斯·西蒙松、帕韦拉夫·安德鲁什基维奇、拉曼·萨罗金、罗马里克·乔丁、萨钦·科特瓦尼、索姆达特
本文由 AI图鉴自动抓取并翻译，版权归原作者所有；如需查看原文请点下方链接。
OCT. 8, 2026
The Google AI Edge Team is excited to announce the open-source release of ML Drift, our high-performance, cross-platform, on-device GPU compute engine specifically built for on-device AI/ML inference, under the Apache 2.0 license. By abstracting hardware and low-level API complexities of on-device GPUs across OpenGL ES, OpenCL, Metal, and WebGPU, ML Drift empowers developers to build real-time, interactive ML experiences from advanced video effects to generative AI across multiple platforms. Serving as the core GPU acceleration engine within LiteRT, ML Drift is also available as a standalone library for custom graphics and inference runtimes providing a unified foundation to deliver peak performance everywhere.
ML Drift's GPU API support matrix across platforms
Unlike datacenter inference, where models run on homogeneous, predictable accelerator clusters, deploying GPU-accelerated AI to edge devices is defined by significant hardware diversity. Developers are faced with a wide range of GPU architectures, driver versions, and low-level APIs, not knowing a priori on which specific hardware the application will run on.
The TensorFlow Lite GPU delegate established foundational GPU acceleration, but the ecosystem has since evolved: current on-device workloads now encompass a broad spectrum, ranging from real-time computer vision, audio, and depth processing models to high-parameter generative AI. These architectures push consumer silicon to its limits, creating compute and memory bottlenecks that legacy runtimes weren't designed to solve.
This shift necessitates a universal engineering foundation of ensuring portability and performance across both classical and next-generation models. Drawing on optimization principles discussed in "Speed is all you need", we've built a single, unified framework that delivers stability for traditional ML and peak performance for the most advanced generative AI, ensuring that your models are running on our most efficient, forward-looking GPU compute engine.
To achieve both peak performance and expanded model coverage across classical and generative architectures, ML Drift introduces core architectural changes and structural upgrades over the legacy TFLite GPU delegate.
Unified Shaders via Tensor Virtualization: Historically, maintaining optimized shaders for the TFLite GPU delegate required hardcoding logical tensor mappings directly to physical GPU objects (such as textures and buffers) separately across OpenGL, OpenCL, and Metal backends. ML Drift introduces tensor virtualization, a core architectural paradigm that decouples a tensor's logical representation from its physical allocation on the GPU. By having dynamic shader templates resolve and translate coordinates during the compiler’s initialization phase, this unified shader model eliminates the need to maintain disparate, backend-specific shader codebases, adding minimal runtime overhead while preserving cross-platform model portability.
Extensible Custom Op Framework: ML Drift adopts a modernized custom op framework that provides direct registration APIs with low-level shading language access. To accelerate this, ML Drift includes an agentic SKILL.md guide, enabling coding agents to author, register, and verify performant custom shaders in minutes. This provides developers with granular control to integrate specialized model blocks directly into the execution graph, drastically reducing the barrier to deploying proprietary architectures.
5D Tensor Support: The TFLite GPU delegate is structurally hardcoded to 4D tensors, forcing developers to use layout hacks for complex models requiring 5D tensors. ML Drift reflected this long overdue functionality in the new LiteRT ML Drift GPU accelerator to enable 5D tensor support out-of-the-box. With this, LiteRT executes complex workloads like 3D convolutional networks for volumetric/spatial AI and spatiotemporal models (e.g., YOLO 11n, MobileViT v2, and Swin Transformer v2) directly on edge GPUs.
Performance Upgrades for Classical Models: As the direct successor to our legacy GPU backend, ML Drift provides a strict performance upgrade for existing classical workloads. By decoupling the runtime from hardware-specific constraints and improving kernel execution efficiency, models that were already running on TFLite GPU gain immediate performance improvements. Migration is designed to be straightforward, with legacy workloads maintaining or exceeding previous benchmarks.
Stage-aware Optimizations for Edge LLMs: Autoregressive LLMs have two different computational workloads during inference: the compute-bound KV cache prefill stage and the memory-bandwidth-bound decode stage, generating tokens one by one. ML Drift dynamically switches kernels and layout configurations depending on the active execution phase. During the decoding stage, it utilizes a custom, convolution-aligned KV cache layout and applies aggressive in-kernel activation quantization to bypass redundant memory roundtrips.
Prefill 512, decode 128, context length 640, 4-bit weights (block size 32), fp16 KV cache, argmax sampling, sync every token.
Prefill (solid bars; left ord.) and decode (hollow bars; right ord.) throughputs are presented.
Gemma4 E2B performs better on frameworks supporting cross KV sharing.
Broadening across Platforms (Desktop Previews): In today’s emerging agentic coding era, highly responsive desktop inference is becoming vital for powering local coding agents and workstation workflows. While we originally engineered ML Drift’s WebGPU backend for in-browser acceleration, Dawn (Chromium's WebGPU implementation) allowed us to compile that exact same codebase natively outside the browser. This unified API model enables ML Drift to run on Windows and Linux, leveraging WebGPU's modern, native hardware abstraction to bypass DirectX and Vulkan fragmentation, while complementing our existing high-performance native Metal backend on macOS.
Our primary focus remains delivering lightweight, production-grade inference where resource constraints are tightest: mobile and edge devices. However, local development requires flexibility across developer machines. Below is an early snapshot of ML Drift running Gemma models on workstation hardware, illustrating how our unified runtime scales across environments.
Desktop LLM performance. Prefill 8192, decode 1024, context length 9216,
4-bit weights (block size 32), fp16 KV cache, argmax sampling, sync every token.
Across edge and workstation environments alike, peak throughput only tells half the story: Memory footprint dictates what workloads can actually run concurrently. In our Gemma benchmarks, ML Drift achieves up to 12% lower memory overhead than other frameworks, freeing up memory for other local tasks.
ML Drift is already running in production across millions of devices daily, powering key features across Google's ecosystem, including Chrome, YouTube Shorts, Photos, and Meet. At the frontier, it acts as the runtime for pioneering on-device capabilities, powering Google AI Edge apps like Gallery, Foresight, and Eloquent.
YouTube Shorts migrated its segmentation-based effects to use ML Drift. This delivered up to a 40% reduction in average frame latency across both Android and iOS, enabling creators to apply high-fidelity visual effects in real time without dropping camera frames.
To deliver fast, seamless on-device photo editing, the Google Photos team integrated ML Drift across its computational photography and segmentation pipelines. This delivered up to 2 sec speedup compared to the legacy GPU delegate, making advanced photo enhancements feel instantaneous.
Google Photos Unblur accelerated via ML Drift
Google Chrome integrated ML Drift into its AI runtime infrastructure to power native hardware-accelerated Gemini Nano models for Built-In AI APIs (including the Prompt, Summarize, and Writer APIs). Across desktop platforms, ML Drift enables high-efficiency on-device AI across a growing ecosystem of web applications from e-commerce review summarization to enterprise customer workflows:
Cafe24 using Prompt API for tag generation accelerated via ML Drift.
Beyond Google products, leading developer partners have adopted ML Drift to bring desktop grade capabilities to mobile users:
Adobe Lightroom and Adobe Photoshop upgraded key AI features, including Select Subject, Select Sky, and Adaptive Portrait, with ML Drift, achieving up to 30% faster on-device performance for pro-quality photo editing on mobile.
Adobe Lightroom running on-device via mobile LiteRT ML Drift GPU accelerator
Snap integrated LiteRT to power on-device inference on GPU across Android devices. ML Drift GPU accelerator brought 30% model latency improvements for ML-powered face and style generator effects in Snapchat lenses. ML Drift not only improves user experiences based on existing models, but also expands capabilities to run even large diffusion models.
Maximizing edge AI efficiency requires deep alignment between software architectures and physical silicon. Throughout ML Drift's development, Google worked closely with silicon and IP partners to enable optimizations that maximize hardware utilization across modern GPUs:
- Arm: Co-optimized compute kernels and texture cache locality for Mali and Immortalis GPUs, ensuring high-throughput, power-efficient inference across mobile devices.
- Intel: ML Drift harnessed native WebGPU compute and Xe Matrix Extensions (XMX) to accelerate client LLM and vision workloads on Intel® Core™ Ultra processors with Xe3 graphics.
- Qualcomm Technologies, Inc.: ML Drift optimized OpenCL kernels for peak ALU and memory bandwidth utilization, enabling efficient edge AI acceleration on Qualcomm® Adreno™ GPUs.
By working hand-in-hand with silicon leaders, ML Drift ensures that state-of-the-art model optimizations translate into battery-conscious, high-frame rate performance on user devices.
With the launch of ML Drift, the legacy TFLite GPU delegate will no longer receive new feature updates. We encourage all developers to transition to the LiteRT ML Drift GPU accelerator which offers full backwards compatibility for your existing models while immediately unlocking modern performance gains across Android, iOS, Web, and desktop. For Android developers using unbundled runtimes to minimize app binary size, ML Drift acceleration is available today in standalone LiteRT packages and is coming soon to LiteRT in Google Play Services.
Whether you are integrating on-device models into an app or authoring custom GPU shaders, ML Drift provides two straightforward paths:
- For Application Developers: Use the LiteRT ML Drift accelerator to get out-of-the-box model partitioning, tensor virtualization, and optimized LLM acceleration.
- For GPU & System Engineers: Build directly with ML Drift's standalone C++ APIs via the ML Drift GitHub repository to construct custom `GpuModel` compute graphs and write custom shaders (get started with our OpenCL on Android and WebGPU via Dawn quickstarts).
Community feedback and Contributions
ML Drift is fully open source under the Apache 2.0 license. We welcome contributions, RFCs, and community feedback:
- Source Code: github.com/google-ai-edge/ml-drift
- Issue Tracker & Feature Requests: File an Issue on GitHub
- Discussions & Q&A: Google AI Edge Community Forum
- Contributing: Review our Contributing Guide to add new UCL operators or hardware-specific optimizations.
Acknowledgments:
Andrei Kulik, Ankit Goyal, Bob Liu, Byungchul Kim, Champ Yen, Changming Sun, Chris McClanahan, Clark Duvall, Cormac Brick, Daniel Ho, David Neto, Ekaterina Ignasheva, Fengwu Yao, Frank Ban, Gerardo Carranza, Grant Jensen, Jacob Dormuth, Jagadeesh Pakaravoor, James Price, Jeremy Kemp, Jingjiang Li, Jiuqiang Tang, Kelvin Ma, Ken Russell, Kenji Baheux, Lin Chen, Loko Kung, Lu Wang, Marissa Ikonomidis, Matt Kreileder, Matthias Grundmann, Mattias Simonsson, Paweł Andruszkiewicz, Raman Sarokin, Romaric Jodin, Sachin Kotwani, Somdat
Auto-fetched and translated by AI Atlas. All rights belong to the original author.