不认字的 AI，却很会说话
用语音和 AI 聊天早已不是新鲜事，但市面上绝大多数语音助手的底层逻辑其实是在「念稿」——先把人说的话转成文字，交由文本大模型思考，最后再用语音合成技术朗读出来。
由前推特联合创始人推出的语音模型 Dessa，彻底绕开了文字这座桥梁，试图证明机器的知识与对话能力可以直接从声音本身生长出来。
「既然人类婴儿在识字之前就已经学会了说话，机器为什么不能走同样的路？」Jack Dorsey 抱着这样的决心于是这样做了一个 AI 实验室。Jack Dorsey 是现在的X、从前的推特的创始人之一，现在有一家自己的公司 Block，核心业务是 Square 和 Cash App 等金融支付产品。Dessa 是旗下的研究团队，只关注 AI 研究，直接向公司创始人 Jack Dorsey 汇报。
把文字从训练过程中剔除，是因为文字天然会过滤掉大量信息。文字把一句话固定成一串静态的字符，而声音每一次响起都千差万别。Block 团队表示，完全浸泡在声音里长大的 Dessa 不仅能边听边说、自然插话，随时给出「嗯」「是的」这类附和，甚至能直接在一台配备 M4 Max 芯片的 MacBook Pro 上本地运行。
受限于尚未公开的训练细节与模型规模，外界还难以评估这套技术路线的上限，但它确实指向了一个长期被文本大模型掩盖的「非主流」路线：从语音到语音。
会说话，然后会接电话
如果说 Dessa 代表了实验室里对语音本质的纯粹探索，那么在同一时期，语音模型的另一条务实路线正加速驶入日常工作流。
10 月中旬，OpenAI 更新了语音模型 GPT-Live，不仅在 ChatGPT 客户端里改善了多语言口音的自然度，还在面向开发者的 API 中一口气新增了包括日韩口音英语在内的九款音色，并支持直接用文字描述来定制声音。
几乎同时，xAI 也推出了同样主打实时端到端语音交互（speech-to-speech）的 Grok Voice 3.0。
与 Dessa 那种「完全不碰文字」的洁癖式训练不同，以 Grok 为代表的主流商业模型更在意如何将大模型的文本推理底子与原生语音能力结合，让声音真正派上用场。
以特斯拉为例，车主可以直接在车机上和 Grok 对话，而不用单独打开手机、点开 app。
有趣的是，这批主流语音模型的进化方向，都围绕着一个极其微妙的时间展开：半秒钟。
在键盘上打字时，多等半秒几乎不会改变交流的性质；但在电话里，半秒钟的沉默就足以让对话完全走向另一个极端。这也是为什么，哪怕 AI 给出的答案无懈可击，只要它每次都死板地等人把话说完才接茬、在不合时宜的地方发笑，或者在被打断后依然固执地念完原稿，用户就会瞬间出戏。Grok Voice 3.0 的评测中，半秒钟就是它的优化目标。
毫秒间的差异，会直接影响实际的商业应用。目前车载空间与客服热线，构成了语音 AI 最能发挥的应用场景。根据 xAI 披露的内部运营数据，这套语音系统除了每天应付海量的特斯拉车内对话外，还承担了 Starlink 每天约 2 万通的客服来电，其中约七成问题无需转接人工即可办结。
除了速度，模型能否在夹杂着口音、背景噪音和频繁打断的通话中听清诉求，能否在后台顺畅地检索资料、调用工具、处理故障，最后礼貌地挂断电话。这一系列的操作都需要丝滑完成，再加上自然的声音，才真正算得上「上岗」。
科幻片里的声音，为什么还不是电脑的主界面
人类对语音电脑的迷恋由来已久。早在《银翼杀手》里，主角 Deckard 就曾对着图像分析仪一声声下达口令，让画面平移、放大，直至锁定隐藏的线索。
到了《Her》中，操作系统 Samantha 更进一步，从执行指令的机器变成了随时陪伴左右的灵魂伴侣。不仅能完成所有的日常任务，声线还极其灵动，因此才能和男主建立感情。
现实中的技术演进也的确实现了原先的想象。早年的手机语音助手只能处理设闹钟、查天气这类固定句式的简单指令，智能音箱走进客厅后，最常干的活也不过是放音乐和倒计时。直到大语言模型与语音 Agent 结合，机器才算真正听懂了自由散漫的口语，并开始帮用户收发短信、预约餐厅，听着开发者的口头描述去调用编程工具修改代码。
▲ Apple HomePod
然而，哪怕语音 AI 已经能处理远比「把灯打开」复杂的任务，我们面前，屏幕始终有一席之地，敲键盘和点鼠标，现在还不能被语音使用动摇。
这并非因为模型还不够聪明，而是受制于声音作为信息媒介的物理属性。
声音是线性的，在调度 AI 方面，它尤其适合用来表达一个模糊的意图。比如编程中的想法， 比如指挥 Agent 安排任务。
▲ Claude Code 的语音模式
但是在工作生产的场景，它并不擅长让人同时审阅多个并列的选项。写文章时，我们需要一目了然地看到段落结构并随时回溯修改；写代码时，我们必须盯着屏幕比对代码差异与运行结果。
哪怕只是订一张机票，把几个航班的价格、起降时段和退改签规则同时铺在屏幕上扫一眼，也远比听语音助手逐条念出来要高效得多。
AI 替人代劳的权限越大，人类往往越需要一块可视化的界面，去核实它到底干了什么，并且在出错之前就及时停下和回撤。
物理环境与社交礼仪同样在暗中划定语音的边界。在开车、做饭或双手抱着孩子的时候，视线与肢体被双双占用，开口说话无疑是最省力的救星；可一旦场景切换到安静的开放式办公室，一个人对着电脑持续不断地说话，还是显得太诡异了。整个办公室都一起说话，更有如大型客户呼叫中心。
这些场景，关乎信息带宽与隐私边界的制约，无论工程师把模型延迟再往下压多少毫秒，也不能解决。
科幻片里「对电脑说句话，一切就完成了」的预言正在被逐渐兑现。只不过，机器越是学会了如何像人一样说话，我们反而越清楚地意识到，哪些时刻可以说话，哪些时刻还是得闭嘴。
本文由 AI图鉴自动抓取并翻译，版权归原作者所有；如需查看原文请点下方链接。
I can't read, AI, but I can talk.
It is no longer new to talk with voice and AI, but the bottom logic of most of the voice assistants on the market is actually "reading" — turning words into words, giving them to LLM for reflection, and finally reading them out using speech synthesis.
The voice model, Dessa, launched by the former co-founder of Twitter, completely bypassed the bridge of words in an attempt to prove that the machine ' s knowledge and ability to talk can grow directly from the sound itself.
"If human babies have learned to speak before they can read, why can't the machine go the same way? Jack Dorsey, with such determination, did this to an AI laboratory. Jack Dorsey is one of the founders of the current X, former Twitter, and now has one of his own companies, Block, whose core business is financial payment products such as Square and Cash App. Dessa is a research team under the flag, focusing only on AI research and reporting directly to Jack Dorsey, the founder of the company.
Text is removed from the training because it naturally filters out a lot of information. The text fixes a sentence into a series of static characters, and the sound is very different each time. The Block team says that Dessa, who grew up in a sound completely immersed, can not only listen to the sound, naturally intervene, give "um" and "yes" adjuncts at any time, but even run locally on a MacBook Pro with M4 Max chips.
It is difficult for the outside world to assess the upper limit of this technology route, which is limited by the training details and size of the model that have not yet been made public, but it does point to a “non-mainstream” route that has long been masked by text LLM: from voice to voice.
He talks, he answers.
If Dessa represents a pure exploration of the nature of the voice in the laboratory, another pragmatic route of the voice model is accelerating into daily workflow during the same period.
In mid-October, Open AI updated the voice model GPT-Live, which not only improved the naturality of the multilingual accent in ChatGPT client, but also added nine voices, including Japanese and Korean accent English, to the API for developers, and supported the customization of the sound by direct text description.
Almost simultaneously, x AI also introduced the same root-to-end voice interaction (speech-to-speech) Grok Voice 3.0.
Unlike Dessa's "no words at all" eclectic training, the dominant business model, represented by Grok, is more concerned with how to combine the textal reasoning of LLM with the original speech ability, so that the sound really works.
In Tesla, for example, the owner can talk directly to the Grok on the engine instead of turning on his cell phone and clicking on an app.
Interestingly, the evolution of the mainstream voice models revolved around an extremely delicate time: half a second.
When typing on the keyboard, the length of half a second will hardly change the nature of the communication; but in the telephone, the silence of half a second is enough to allow the dialogue to reach another extreme. And that's why, even if the answer given by AI is impeccable, if it's always taken up by people who say what they say, laugh where they don't fit in, or remain stubborn to read the original after being interrupted, then the user comes out in no time. In the evaluation of Grok Voice 3.0, half a second is the goal of its optimization.
Differences in milliseconds directly affect actual commercial applications. At present, the vehicle-borne space and passenger-service hotlines constitute the most effective voice AI applications. According to the internal operating data disclosed by XAI, the voice system, in addition to dealing with massive Tesla intra-car conversations on a daily basis, also assumed the call of Starlink's approximately 20 million daily passenger service, of which about 70 per cent of the problems could be closed without a hand-over.
In addition to speed, the model is able to hear claims in conversations with accents, background noises and frequent interruptions, and is able to easily retrieve information, call tools, deal with malfunctions and finally hang up the telephone politely. This series of operations requires silk slips and natural voices to be truly "on board".
The sound of science fiction. Why is it not the main computer? Noodles.
The human obsession with voice computers has long existed. Back in Silver Wing Killer, the leading actor, Deckard, used to give a password to the image analyser, to flatten and zoom in until the hidden clues were locked.
In Her, the operating system Samantha went further, from the machine that executed the command to the soul mate who was always there. Not only are all day-to-day tasks carried out, but the sound lines are also extremely lively, so that it is possible to build feelings with the man.
The technological evolution in reality has indeed brought about the original vision. Early cell phone voice assistants can only handle simple instructions such as alarm clocks, weather check, and the most common work when smartphones enter the living room is to play music and count down. It was not until the Big Language model was combined with the Voice that the machine really understood free speech and started helping users send text messages, book restaurants, and listen to the developer's oral description to call programming tools to change the code.
▲ Apple HomePod
However, even if voice AI has been able to handle tasks that are far more complex than turning on the light, there is always room on the screen, knocking on the keyboard and clicking on the mouse, which cannot be shaken by the use of the voice.
This is not because models are not smart enough, but because they are subject to the physical properties of sound as an information medium.
Sound is linear, and it is particularly suitable for a vague intent in terms of dispatch AI. Like the idea of programming, like commanding Agent on mission.
Voice mode for Claude Code
However, in the workplace, it is not very good at allowing multiple parallel options to be examined at the same time. When writing, we need to look at the structure of the paragraph and change it back at any time; when writing the code, we have to look at the difference between the screen and the running results.
It is far more efficient than listening to voice assistants to read out on each page, even if it is just a ticket that sets the prices, departures and departures of several flights simultaneously on screen.
The larger AI permission, the larger it is, the more human beings often need a visual interface to verify what it does and to stop and withdraw in time before it makes a mistake.
The physical environment and social ceremonialism also define the boundaries of voice in secret. When driving, cooking or holding children with their hands, eyes and limbs are taken over, and speaking is undoubtedly the most energy-saving; but once the scene is switched to a quiet, open office, it's weird that a person continues to talk to a computer. The entire office speaks together, more like a large customer call centre.
These scenarios, which relate to information bandwidth and privacy boundaries, cannot be resolved regardless of how many milliseconds engineers delay the model.
In science fiction, the prophecy "Say to the computer, everything is done" is being fulfilled. Only the more the machine learns how to talk like a human being, the clearer we are about when we can talk and when we have to shut up.
Auto-fetched and translated by AI Atlas. All rights belong to the original author.