智东西（公众号：zhidxcom）
作者 | 孙小雯
编辑 | 程茜
智东西10月22日消息，10月20日，英国AI音频独角兽ElevenLabs发布实时语音系列模型Eleven Realtime，包括面向自然交流的Eleven Realtime v1 Mini，以及面向客服、多步骤任务的Eleven Realtime v1 Max。两款模型均支持超过90种语言，可使用ElevenLabs声音库中超过1.7万种音色。
此次发布主打简化语音智能体的搭建和调试。过去，开发者需要分别选择语音识别、大语言模型和语音合成等组件，再调整接话、打断等交互细节。Eleven Realtime统一配置和调优底层组件，开发者选择Mini或Max后，即可围绕具体业务设置角色、知识库和工具，无需再逐项选择底层模型。
实时语音的难点之一，是既要答得对，又不能让用户等太久。据第三方AI评测机构Artificial Analysis的独立测试，Max在Big Bench Audio语音推理评测中以98.8%的正确率位列第三，平均首次音频响应时间为0.57秒。
此次模型发布，智东西测试了其中的Architect搭建助手。从实测体验来看，用语音修改Agent设置比较方便，多项操作无需手动点击。不过，中文口音明显，回复偏长，延迟也较高。
ElevenLabs于2022年成立，从文本转语音起家，逐步将业务扩展至语音识别、音效、音乐和对话智能体。今年2月，公司完成5亿美元（约33.4亿元人民币）D轮融资，估值110亿美元（约734.2亿元人民币）；9月的一笔员工股份交易使其估值达到220亿美元（约1468.4亿元人民币）。投资方既有红杉资本、a16z，也包括英伟达旗下NVentures和LG Technology Ventures等产业资本。
其两位联合创始人Mati Staniszewski和Piotr Dabkowski分别在Palantir和谷歌工作多年，2025年已跻身《福布斯》亿万富翁行列。公司也在考虑IPO，CEO Mati Staniszewski此前透露，希望未来两年至两年半内具备上市条件。
目前，两款模型可通过ElevenAgents平台或Realtime API使用。用户可在平台中创建或打开Agent，选择Realtime模型；开发者也可通过API接入自己的应用。该API兼容OpenAI Realtime规范，目前仅支持WebSocket连接。
产品及演示入口：https://ElevenLabs.io/zh/realtime
Agent搭建入口：https://ElevenLabs.io/app/agents
API接入文档：https://ElevenLabs.io/docs/eleven-agents/guides/openai-realtime-compatibility
一、实测语音修改Agent设置，操作方便，但中文口音明显
为了测试语音搭建Agent是否方便，智东西通过ElevenLabs的Architect搭建助手，尝试用英文修改语言、切换音色，再让助手改用中文交流，并用中文继续修改设置。
首先，我们用英文要求修改Agent的语言设置。Architect理解指令后完成了修改，过程比较顺畅，不需要手动寻找菜单、切换选项。
接着，我们要求它换个声音。这次没能如愿：Architect自身的音色不支持切换，所创建Agent的音色则可以在设置中选择。打开音色库后，可以看到可选声音确实不少。
▲ElevenLabs音色库中的可选声音
随后，我们让Architect改用中文交流。它能听懂要求，但中文带着明显的外国口音，听起来有点“塑料”，部分表达需要仔细听才能明白。
不过，中文指令的执行比较顺利。我们用中文要求修改Agent的语言设置，它也完成了相应修改。
实测下来，通过语音直接修改设置，还是让人有些惊喜。用户说清楚要求，助手就能完成部分配置，省去了寻找菜单、填写表单的操作。
不过，Architect有些“话密”，一个简单要求也可能引出一长串解释，且不易被打断。
▲Architect执行指令时，有时会给出较长的解释
此外，测试中存在较明显的响应等待和偶发卡顿，可能受到网络影响。回复偏长、打断响应不够及时，以及中文语音不够自然，是本次体验中较突出的不足。
注：本次实测范围为Architect搭建助手。运行创建的Agent需要付费购买Token，因此我们没能继续测试Mini、Max的实际对话表现。
二、Mini用于教学和角色互动，Max处理客服及多步骤任务
ElevenLabs在官方演示中展示了两款模型的应用场景。
该公司称，Mini主打低成本的自然对话，适用于AI助手、教学辅导和互动角色；Max则面向客服、潜在客户筛选及多步骤工作流，更适合需要逐步推理、频繁调用工具的任务。
第三方AI评测机构Artificial Analysis对两款模型进行了独立测试。在语音推理评测Big Bench Audio中，Max的正确率为98.8%，Mini为75.9%。该测试通过语音提出1000道题，涉及逻辑判断、物品计数、路径推理等任务，再对模型的口头回答进行评分。
截至10月21日，Max在这一单项中位列第三，仅次于阶跃星辰的StepAudio 3 Realtime（99.7%）和阿里的Qwen Audio 3.0 Realtime Plus（99.2%）。
▲Big Bench Audio语音推理评测排名，Eleven Realtime v1 Max以98.8%的正确率位列第三（图源：Artificial Analysis）
Max的延迟也较低。据ElevenLabs发布公告，Max和Mini在Big Bench Audio测试中的平均首次音频响应时间分别为0.57秒、0.62秒。作为参考，Artificial Analysis官网所列Qwen Audio 3.0 Realtime Plus的同项指标为1.54秒。该指标衡量模型开始发声所需的时间。
在真人参与实时对话的Speech Agent Arena评测中，Max的工具使用任务完成率为87.1%，Mini为60.3%。该指标衡量模型通过正确的工具调用完成任务的比例，Max在这一项上比Mini高出26.8%。
不过，在停顿、打断等接话表现上，Mini的综合成绩略高。在Full-Duplex-Bench（全双工对话评测）中，Artificial Analysis选取停顿处理、轮次交接、用户打断和附和回应四类项目计算加权得分，Mini为95.8%，Max为94.1%。这些项目考察模型能否在用户停顿时继续等待、用户说完后及时接话，以及区分真正的打断和“嗯”“对”等附和声。
三、从配音、转写到实时交互，延续级联架构
此次Realtime发布前，ElevenLabs已分别推出语音合成、语音识别模型，并提供语音Agent搭建服务。
语音合成方面，今年9月，该公司发布文本转语音模型Eleven v4及其低延迟版本v4 Turbo，重点改善情绪表达、声音克隆和多角色对话效果。用户可以用自然语言或标签控制台词的语气、重音，并加入笑声等声音，v4 Turbo则针对语音Agent等低延迟场景优化。
语音识别方面，去年11月推出的Scribe v2 Realtime面向实时转写；今年1月发布的Scribe v2则重点处理长录音和复杂音频，支持说话人区分、词级时间戳，及热词提示。
在单项模型之外，ElevenLabs早在2024年11月就推出了Conversational AI，支持开发者结合知识库、函数和大语言模型创建语音Agent。
技术路线上，Eleven Realtime延续采用级联架构，由语音识别、大语言模型和语音合成等独立组件协同完成对话。该公司认为，这种方式便于分别测试和升级各环节，也能通过中间生成的文字排查错误、约束回答，并接入更强的大语言模型。
不过，该公司也在技术博客中提到，语音转成文字时会丢失部分语调、节奏和情绪信息，需要通过额外建模补充。其优化方向是在保留各组件可控性的同时，改善声音表现、延迟和轮次交接。
定价方面，Realtime API按音频连接时长和Token用量计费。超出套餐额度后，两款模型的音频连接费用均为每分钟0.08美元；Mini每百万输入Token收费0.20美元、输出Token收费6美元；Max分别为1.80美元、9美元。
结语：实时语音加速接入Agent，交互体验仍需打磨
ElevenLabs此次发布Realtime，将语音识别、推理和语音生成组合成可直接选用的实时交互方案。其在改善对话表现的同时，也降低了语音Agent的搭建门槛。
这一赛道已聚集多家大厂。OpenAI的GPT-Realtime、谷歌的Gemini Live，以及阿里的Qwen-Audio-Realtime系列等，都在强化实时语音交互与工具调用能力。尽管技术路线和产品侧重有所不同，但各家目标一致：让用户通过自然对话，推动AI完成具体任务。
要让对话顺畅、任务持续推进，还需要整套系统配合。网络延迟、接话时机和工具执行都会影响使用体验，OpenAI此前也披露了底层网络架构的优化，以减少停顿和打断延迟。
从智东西此次的体验来看，通过语音创建文档、修改设置已经让人惊喜，但响应速度、打断处理及页面切换时的语音连续性仍需打磨。随着模型和产品继续迭代，我们期待这类应用能在连续交流和任务执行中表现得更稳定，让语音成为用户日常工作中真正顺手的操作方式。
本文由 AI图鉴自动抓取并翻译，版权归原作者所有；如需查看原文请点下方链接。
Intellectual (WeChat Official Account:zhidxcom)
Author, Sun Xiaoxuan.
Editor, Concubine.
Wisdom News, October 22, 20, Eleven Labs publish Real-time Voice Series Model, British AI Audio Unicorn, Eleven Realtime, including Eleven Realtime v1 Mini for nature-oriented communication, and Eleven Realtime v1 Max for customer service, multistep missions. Both models support more than 90 languages and use more than 17,000 voices in Eleven Labs sound libraries.
This time, publish was used to streamline the set-up and debug the voice AI agent. In the past, developers were required to select components such as voice recognition, large-language models and speech synthesis, respectively, and to adjust the interactive details of answering calls, interrupting, etc. Eleven Realtime centrally configures and fine-tunes the bottom component, and when the developer selects Mini or Max, he or she sets the roles, knowledge base and tools around the specific business without having to select the bottom model on a case-by-case basis.
One of the difficulties with real-time voice is that the answer is right and the user cannot wait too long. According to independent tests conducted by the third-party AI rating agency Artificial Analysis, Max ranked third in the Big Bench Audio speech reasoning assessment at a 98.8 per cent correct rate, with an average first audio response time of 0.57 seconds.
This model, publish, tested the Architec builder. From an empirical experience, it is easier to change the Agent settings with voice, and multiple operations do not require manual hits. However, the Chinese accent is clear, response is long and delays are high.
Eleven Labs was established in 2022 to gradually expand its operations to voice recognition, sound, music and dialogue from text to voice. In February this year, the company completed the D round of financing of $500 million (approximately RMB 3.34 billion), valued at $11 billion (approximately RMB 73.42 billion); and in September, a trade in employee shares brought its valuation to $22 billion (approximately RMB 14.684 billion). The investors include Redwood capital, a16z, and industrial capital such as NVentures and LG Technology Ventures, under the British flag.
Two of its co-founders, Mati Staniszewski and Piotr Dabkowski, have worked for many years in Palantir and Google, respectively, and were among the billionaires of Forbes in 2025. The company was also considering IPO, CEO Mati Staniszewski, who had previously revealed that he hoped to be listed within the next two to two and a half years.
Currently, two models can be used through the ElevenAgents Platform or Realtime API. Users can create or open Agent in the platform and select the Realtime model; developers can also access their applications through API. The API is compatible with Open AI Realtime and currently supports only WebSocket connections.
Product and presentation portal: https://ElevenLabs.io/zh/realtime
Agent's portal: https://ElevenLabs.io/app/agents
API access document: https://ElevenLabs.io/docs/eleven-ages/guides/openai-realtime-compatibility
i. Audio sound modification of the Agent set-up is easy to operate, but the Chinese accent is clear
In order to test the convenience of speaking Agent, intellectuals are using the Architec assistant of Eleven Labs to try to change the language in English, to switch to voice, and to change the assistant to Chinese and continue to change the settings in Chinese.
First of all, we asked in English to change Agent's language settings. Architect understands the command and completes the modification. The process is smooth and does not require manual search for menus and switching options.
Then we asked for a different voice. This time Architect's own voice did not support the switch, and Z108Z, created by Agent, could be selected in the settings. When you open the voice library, you can see a lot of optional voices.
An optional voice in the Eleven Labsvoice library
Then we changed Architect to Chinese. It understands the requirements, but the Chinese language, with a clear foreign accent, sounds a little “plastic” and in part speaks of the need to listen carefully.
However, the implementation of the Chinese directive has been relatively smooth. In Chinese, we asked for changes to the language set-up of Agent, and it completed the corresponding changes.
It's a surprise to actually change the settings directly by voice. Users stated clearly that the assistant would be able to complete a partial configuration, leaving out the search for menus and filling in forms.
However, Architect has some “close talk”, and a simple requirement may also lead to a long line of explanations that are not easily interrupted.
Architect sometimes gives a longer explanation when he executes the instructions.
In addition, there is a more visible response waiting and occasional carton in the test, which may be affected by the network. Longer responses, less timely interruptions and less natural Chinese voice are some of the more salient deficiencies of this experience.
Note: The scope of this exercise is Architect's builder. Run the created Agent needed paid to buy Token, so we couldn't continue testing Mini, Max's actual conversation.
ii. Mini for teaching and role interaction, Max for customer service and multistep tasks
In an official presentation, Eleven Labs presented the application of two models.
It states that Mini's low-cost natural dialogue applies to AI's assistants, teaching tutors and interactive roles, while Max is more suited to the task of requiring gradual reasoning and frequent calls for tools for customer service, potential customer screening and multistep workflow.
The third-party AI evaluation agency Artificial Analysis conducted independent tests of the two models. In the speech reasoning assessment of Big Bench Audio, the correct rate for Max was 98.8 per cent and for Mini 75.9 per cent. The test raised 1,000 questions through voice, involving tasks such as logical judgement, item count, path reasoning, and then rated the model ' s oral responses.
As at 21 October, Max was third in the list, after StepAudio 3 Realtime (99.7 per cent) and QwenAudio 3.0 Realtime Plus (99.2 per cent) of Ali.
▲Big Bench Audio voice reasoning ranking, with Eleven Realtime v1 Max ranking third at 98.8% (source: Artificial Analysis)
Max's delay is also low. According to the Eleven Labspublish announcement, the average first audio response times for Max and Mini were 0.57 and 0.62 seconds, respectively, in Big Bench Audio tests. For reference purposes, the same indicator for Qwen Auto 3.0 Realtime Plus in the Artificial Analysis network is 1.54 seconds. The measurement model of the indicator takes time to begin speaking.
In the assessment of Speech Agent Arena, who was involved in real-time dialogue, Max had 87.1 per cent of tasks performed using tools, compared to 60.3 per cent for Mini. The indicator measures the percentage of the correct tool call missions in which Max is 26.8 per cent higher than Mini.
However, Mini's combined performance was slightly higher in terms of pauses, interruptions, etc. In Full-Duplex-Bench (full-work dialogue assessment), Artificial Analysis calculated a weighted score of 95.8 per cent for Mini and 94.1 per cent for Max, for four categories of items: stop-and-off, turn-over, user interruption and attachment. These projects look at whether the model can continue to wait while the user is at a standstill, to respond in a timely manner after the user has spoken, and to distinguish between true interruptions and the “mmm” echoes.
III. Continuation of cascade structures from voiceover, transliteration to real-time interaction
Before this time Realtimepublish, ElevenLabs had launched a speech synthesis and voice recognition model and provided voice Agent set-up services.
In terms of speech synthesis, in September this year, the company publish text-to-speech model Eleven v4 and its low-lated version v4 Turbo focused on improving emotional expression, voice cloning and multi-role dialogue. Users can control the dialogue line tone, accent, and add laughter, while v4 Turbo optimises low-delayed scenes such as voice Agent.
In terms of voice recognition, Scribe v2 Realtime, which was introduced in November last year, is rewritten in real time; Scribe v2 of publish in January this year focuses on long audio and complex audio, supporting voice differentiation, word-level punctuation, and hot-word tips.
In addition to individual models, Eleven Labs launched Conversational AI as early as November 2024, supporting developers to create voice Agent in combination with knowledge banks, functions and large language models.
On the technical route, Eleven Realtime continues to use cascade structures, with independent components such as voice recognition, large language models and voice synthesis completing the dialogue. In its view, such an approach would allow for separate testing and upgrading of the components, as well as for checking errors of words generated through the intermediate, restraining answers and access to larger language models.
However, the company has also mentioned in its technical blog that some tone, rhythm and emotional information may be lost when a voice is translated into text and need to be supplemented by additional modelling. Its optimal direction is to improve sound performance, delay and rotation while retaining the controllability of components.
In terms of pricing, Realtime API is charged for the length of the connection to the audio and token. After exceeding the package of quota, the audio connection costs for both models were $0.08 per minute; $0.20 per million for Mini input Token and $6 for output Token; and $ 1.80 and $9 respectively.
Conclusion: Real-time voice speeds access to Agent, interactive experience still needs to be sharpened
This time, Eleven Labs publish Realtime combines voice recognition, reasoning and voice generation into a direct, interactive, real-time option. While improving the performance of the dialogue, it also lowered the threshold of the voice of Agent.
This track has gathered many large factories. The GPT-Realtime in Open AI, Gemini Live in Google and the Qwen-Audio-Realtime series in Ali are reinforcing real-time voice interaction with tool call capabilities. Despite the different focus on technology routes and products, there is a convergence of goals: to enable users to contribute to AI ' s specific tasks through natural dialogue.
A coherent set of systems is needed to keep the dialogue smooth and the mandate moving forward. The use experience is affected by network delays, response times and implementation of tools, and Open AI has previously disclosed the optimization of the bottom network architecture to reduce pauses and interruption delays.
From this experience of intellectuals, the creation of documents and modifications to settings through voice are already surprising, but voice continuity in response speed, disruption and page switching still needs to be sharpened. As models and products continue to evolve, we expect such applications to be more stable in continuous communication and in the performance of tasks, making voice a truly smooth way for users to operate in their daily work.
Auto-fetched and translated by AI Atlas. All rights belong to the original author.