面谈
亚历克斯·斯莫拉，博森AI联合创始人 – 访谈系列
在 Google 上的首选来源中添加 Unite.AI
亚历克斯·斯莫拉是博森AI的联合创始人，他在那里致力于对话语音AI和多式联运系统。 他的职业生涯跨越卡内基梅隆大学的学术研究和AWS的大规模AI倡议。 He 带来将研究和产品经验结合到建设助手上，这些助手能够理解对话提示，处理中断，并进行有益的工作。
Boson AI开发了语音，音频和多模式互动的基础模型和工具。 其Higgs Realtime模型支持现场语音对语音对话，而其更广泛的平台包括表达式语音生成，avatars，以及企业语音agents. 在此次采访中，斯莫拉讨论了对话智能，多语言性能，推论经济学，以及将语音agents移动到真正的业务工作流程所需的保障。
你的职业生涯将你从卡内基梅隆大学的学术研究带到了AWS的大规模AI倡议，现在又共同创立了博森AI. 是什么激励你专注于对话的语音AI，以及你希望克服的人与AI互动中哪些根本的局限性？
人类的交流远不止于言语。 你可以听到有人的声音中的不确定性，或者看到他们不明白一个解释。 一个小的承认可以告诉你继续说话。 我们毫无顾虑地接过这些提示，但教计算机识别这些提示仍然是一项正在进行中的工作。
这使这个领域变得如此有趣。 我们现在的系统拥有相当的智能，但与它们沟通仍然会感到尴尬。 我希望人们能够用自己的话来解释自己需要什么，而不必想出如何与软件交谈。
在AWS时，我学会了密切关注客户试图做什么，以及产品是否帮助他们做。 随着研究的进展，我们把这种方法带到了博森，并有迅速行动的自由。
声音和视频给我们更多的工作。 如果有人看起来很困惑或犹豫，助理有机会在谈话偏离轨道之前作出回应。 使这些日常互动正确，是我们野心的一大部分。
Boson AI近期的推出 Higgs Realtime，旨在实现自然，实时语音对语音交互。 是什么使得这种方法与传统系统根本不同，这些系统结合了语音识别，语言模型，文本转语音，为什么这种区分对于AI助手的未来很重要？
在传统的管道中，语音识别将你的话变成文字，语言模型生成一个答案，语音模型大声读出来。 每一步都取决于之前的一步，这可以让对话感到缓慢和脱节。
希格斯实时(Higgs Realtime)是一种音频本地化的语音对语音模型，为现场互动而构建。 其培训涉及对话行为，包括转身和回应不断变化的请求。 这与在文本聊天台上放声音大不相同。
想想当你问一个人一个困难的问题。 他们可能会说，“让我查查，”他们正在研究。 他们已经听到了，人工智能助理在获取信息或使用工具时也需要这样做
随着任务的延长，这一点变得更加重要。 助理需要随时通知你，如果您问问题或改变主意，请回答。
对话AI的最大挑战之一是知道何时说话，何时聆听，如何处理中断或方向变化。 Boson AI如何应对这些挑战，在AI对话真正自然前还需要什么技术突破？
承认和中断听起来相当相似，但它们需要不同的答复。 若有人在您说话时，如果他们更正了电话号码中的一个数字，你需要停下来使用正确的号码。 AI助手应该能够辨别出不同之处。
Higgs Realtime用称来帮助决定他们是否已经发言完毕，迅速作出反应是重要的，但知道如何作出反应也是重要的。
始终如一地实现这一权利仍然相当困难。 人们不会轮流说话，助手必须了解所发生的变化，并记住自己在做什么。
Boson AI开发了IH Bench等基准测试，以测量语音agents如何从中断中恢复。 这些评价揭示了当今主要语音模型的弱点，产业界应该如何重新考虑如何衡量对话智能？
我们建造了IH Bench来测试在有人打断一个声音后会发生什么如果在订票时中途改正某物，agent应接受改正，从正确点继续。 你不应该重新开始，或者坐视它已经给你的信息。 其结果因模式和中断类型而异。 一个我们并不特别喜欢出版的发现：闭重系统一般能更好地处理中断，它们能更深入地保持中断的谈话。
我们宁愿公开衡量，也不愿假装不存在。 这也是基准测试存在的大部分原因。 你无法弥合一个没人同意如何衡量的差距。 我们公布了基准测试没有我们自己的模型。 当我们真的加了一个时，希格斯实时在恢复质量上打出71.2分，在65.5分领先GPT实时1.5分，62.4分领先GPT实时2分，61.1分领先Gemini闪光直播3.1分。 因此，空隙是可阻塞的，一个开放量级模型将其关闭。
我宁愿在测试中表明，我们设计得非常困难，在有东西显示之前发表。 更广泛地说，我认为实地衡量错误的东西。 一个声音可以听起来完全自然，在工作上仍然失败，因为重要的测试是agent是否完成了任务。 如果你需要不断提醒它你在做什么，如何好的声音听起来停止重要性非常快。
令人印象深刻的语音AI演示与能够处理真实客户互动的自主agent有很大的区别。 在企业在业务关键工作流程中自信地部署语音agents之前，什么是两者的区别，哪些能力必须提高？
你上次打电话是什么时候？对大多数人来说，答案永远不是。 你被从一个人递到另一个人，每次解释你的名字，出生日期和问题再次交代，这样下一个人就可以再次要求你全部交代。 所以，接受第四个电话关于同样的问题。 你想让agent知道前三场比赛发生了什么如果重启路由器没有成功，或许是时候派技术员了，agent要做到这一点，它需要你的账户历史和访问记录访问的系统。
它也必须应付预订失败，并知道什么时候把你交给一个能真正修复的人。 这些是企业在给agent打电话前需要测试的情况。 那就有运行它的成本。 处理一个优美的演示的系统可能太昂贵，无法每天运行上千个电话，所以效率是做好商业准备的一部分，而不是单独的关注。
我们与一家全球保险公司合作了几个月。 这不是一个与真正的制度并列的审判。 它处理客户现场谈话，进出，在他们的人类销售agents旁边，在他们的联络中心。 数以千计的同步通话大约是人类agent的三分之一
我发现比成本更有趣的是，它得到了改进。 我们开始谈话时，他们自己的评价评分为30分 5个月后是80分，这是他们用来做一个合格的人类电话营销员的分数。 GPT-4o Realtime在同一评价上得55分。 我们没有在实验室里调头系统比它实际处理的流量要好这是联系中心唯一关心的改进
随着语音agents从回答问题转向执行任务，获取客户信息，以及与企业系统互动，开发者应该如何平衡自主与人的监督？哪些保障措施对建立对这些系统的长期信任至关重要？
我首先要问您相信agent做什么。 让它查清账户余额与允许它转移资金或取消服务大不相同。 权限应反映这一点，其连接的系统必须执行。 无法马上让这些都听从指令。
人们也应该知道他们同意什么。 在agent做出难以推翻的改变之前，它应该解释后果并要求确认。 这样，当它被卡住，让一个人参与应该是直截了当的。
事情还是会出问题当他们这样做的时候，你需要能够看到agent做了什么，在哪里停止了，所以有人可以把它正确。 一个听起来自信的声音可以让我们假设一个系统知道它在做什么，但我们需要通过它如何可靠的行为来赢得这种信心。
Higgs Realtime支持超过100种语言，包括在对话中交换语言的能力。 在提供不同语言、口音和交流风格的一致对话智能方面，特别是在理解情感背景和用户意图时，最大的技术挑战是什么？
语言报导听起来像一个列表，不久前有个顾客需要泰文我们得到了部分报道，我们发现的第一件事是我们对泰文本身一无所知，从写泰文基本上没有尖锐意义开始。 这打破了句子界限的假设大部分语音系统都是建立在。 我们可以推迟市场。 相反，我们找到了讲泰语的计算机科学家，他们重建了处理程序。
大约六个星期后，我们在曼谷进行了示范。 在演示期间，我无法分辨情况如何，我听到人们用我不说的语言测试一个系统，公约与词汇不同。 在日语中，听众在演讲时会小声承认，如果助手每次都停下来交出转盘，就会破坏对话的节奏。
话语可以完美，对话仍然失败。 你说的话和你说的话一样重要你可以用完全相同的词亲切或讽刺，适当的反应是完全不同的。 坚持语调和语境，而语言改变中刑，是使我感兴趣的一大部分。
OpenAI和Google等公司正在大量投资实时多式联运AI. 博森AI这样的专业公司如何与具有显著较大计算资源的组织竞争，在模型架构，性能，推论经济学方面，你在哪里看到最大的差异机会？
更多的计算并不自动意味着你使用得很好。 在现场交谈中，你可以花很多钱经营一个非常强大的模型仅仅处理一个人说“嗯，”或“是的，也许。” 萨
本文由 AI图鉴自动抓取并翻译，版权归原作者所有；如需查看原文请点下方链接。
Interviews
Alex Smola, Co-Founder of Boson AI – Interview Series
Add Unite.AI to your preferred sources on Google
Alex Smola is co-founder of Boson AI, where he works on conversational voice AI and multimodal systems. His career spans academic research at Carnegie Mellon University and large-scale AI initiatives at AWS. He brings that combination of research and product experience to building assistants that can understand conversational cues, handle interruptions, and carry out useful work.
Boson AI develops foundation models and tools for voice, audio, and multimodal interaction. Its Higgs Realtime model supports live speech-to-speech conversations, while its broader platform includes expressive speech generation, avatars, and enterprise voice agents. In this interview, Smola discusses conversational intelligence, multilingual performance, inference economics, and the safeguards needed to move voice agents into real business workflows.
Your career has taken you from academic research at Carnegie Mellon University to leading large-scale AI initiatives at AWS, and now to co-founding Boson AI. What inspired you to focus on conversational voice AI, and what fundamental limitations in human-AI interaction are you hoping to overcome?
Humans communicate through much more than words. You can hear uncertainty in someone’s voice or see that they haven’t understood an explanation. A small acknowledgement can tell you to keep talking. We pick up on these cues without thinking, but teaching computers to recognize them is still a work in progress.
That is what makes this area so interesting to me. We now have systems with considerable intelligence, but communicating with them can still feel awkward. I want people to be able to explain what they need in their own words, without having to figure out how to talk to the software.
At AWS, I learned to pay close attention to what customers were trying to do and whether the product helped them do it. We’ve brought that approach to Boson, with the freedom to move quickly as the research advances.
Voice and video give us more to work with. If someone looks confused or hesitates, the assistant has an opportunity to respond before the conversation goes off course. Getting those everyday interactions right is a big part of our ambition.
Boson AI recently introduced Higgs Realtime, designed to enable natural, real-time speech-to-speech interactions. What makes this approach fundamentally different from traditional systems that combine speech recognition, language models, and text-to-speech, and why does that distinction matter for the future of AI assistants?
In a traditional pipeline, speech recognition turns your words into text, a language model generates an answer, and a speech model reads it aloud. Each step depends on the one before it, which can make the conversation feel slow and disjointed.
Higgs Realtime is an audio-native speech-to-speech model, built for live interaction. Its training addresses conversational behavior, including turn-taking and responding to changing requests. That’s very different from putting a voice on a text chatbot.
Think about when you ask someone a difficult question. They might say, “Let me look that up,” while they work on it. You know they’ve heard you and are working on it. An AI assistant needs to do the same while it retrieves information or uses a tool, so you’re not left wondering whether it understood you or stopped working.
That becomes even more important with longer tasks. The assistant needs to keep you informed as it works and respond if you ask a question or change your mind.
One of the biggest challenges in conversational AI is knowing when to speak, when to listen, and how to handle interruptions or changes in direction. How is Boson AI addressing these challenges, and what technical breakthroughs are still needed before AI conversations feel genuinely natural?
An acknowledgment and an interruption can sound quite similar but they need different responses. If someone says “yeah” while you’re talking, they may want you to keep going. If they correct a digit in their phone number, you need to stop and use the right number. An AI assistant should be able to tell the difference.
Higgs Realtime uses the meaning of what someone says to help decide whether they’ve finished speaking or are taking a pause. Responding quickly is important, but so is knowing how to respond.
Getting that right consistently is still quite difficult. People don’t speak in neat turns and the same words can mean different things depending on the conversation. The assistant has to understand what changed and remember what it was doing.
Boson AI has developed benchmarks such as IHBench to measure how voice agents recover from interruptions. What have these evaluations revealed about the weaknesses of today’s leading voice models, and how should the industry rethink the way it measures conversational intelligence?
We built IHBench to test what happens after someone interrupts a voice agent. If you correct something halfway through a booking, the agent should take the correction and carry on from the right point. You should not have to start again, or sit through information it has already given you. The results varied a lot depending on the model and the type of interruption. One finding we did not especially enjoy publishing: the closed-weight systems generally handled interruptions better, and they held up better the deeper into a conversation the interruption landed.
That is a real gap in the open ecosystem, and we would rather measure it in public than pretend it isn’t there. It is also most of the reason the benchmark exists. You cannot close a gap that nobody has agreed how to measure. We published that benchmark with no model of our own in it. When we did put one in, Higgs Realtime scored 71.2 on recovery quality, ahead of GPT Realtime 1.5 at 65.5, GPT Realtime 2 at 62.4 and Gemini 3.1 Flash Live at 61.1. So the gap is closable, and an open-weight model closed it.
I would rather show that on a test we designed to be hard and published before we had anything to show on it. More broadly, I think the field measures the wrong things. A voice can sound completely natural and still fail at the job, because the test that matters is whether the agent finishes the task. If you have to keep reminding it what you were doing, how good the voice sounds stops mattering very quickly.
There is a significant difference between an impressive voice AI demonstration and an autonomous agent capable of handling real customer interactions. What separates the two, and which capabilities must improve before enterprises can confidently deploy voice agents in business-critical workflows?
When was the last time you enjoyed calling your cable company? For most people the answer is never. You get handed from one person to the next, and each time you explain your name, your date of birth and your problem again, so that the next person can ask you for all of it again. So take the fourth call about the same problem. You want the agent to know what happened on the first three. If restarting the router hasn’t worked, perhaps it is time to send a technician, and you should not have to make that case yourself. For the agent to do that, it needs your account history and access to the system that books the visit.
It also has to cope when the booking fails, and know when to hand you to a person who can actually fix it. Those are the situations a business needs to test before it puts an agent on customer calls. Then there is the cost of running it. A system that handles a demo beautifully can be far too expensive to run for thousands of calls a day, so efficiency is part of being ready for business rather than a separate concern.
We have been running this exact scenario with a global insurance company for several months. It is not a trial sitting alongside the real system. It handles live customer conversations, inbound and outbound, next to their human sales agents, in their contact centre. Thousands of concurrent calls, at roughly a third of the cost of a human agent.
The part I find more interesting than the cost is that it improved in place. Their own evaluation scored our conversation quality at 30 when we started. Five months later it was at 80, which is the score they use for a qualified human telemarketer. GPT-4o Realtime scored 55 on the same evaluation. We did not tune our way to that in a lab. The system got better against the traffic it was actually handling, which is the only kind of improvement a contact centre cares about.
As voice agents move beyond answering questions to executing tasks, accessing customer information, and interacting with enterprise systems, how should developers balance autonomy with human oversight? What safeguards are essential to establishing long-term trust in these systems?
I’d start by asking what you’re trusting the agent to do. Letting it look up an account balance is very different from allowing it to move money or cancel a service. The permissions should reflect that, and the systems it connects to need to enforce them. You can’t leave all of that to an instruction in a prompt.
People should also know what they’re agreeing to. Before an agent makes a change that would be tough to undo, it should explain the consequences and ask for confirmation. That way when it gets stuck, getting a person involved should be straightforward.
Things will still go wrong. When they do, you need to be able to see what the agent did and where it stopped, so someone can put it right. A voice that sounds confident can make us assume a system knows what it’s doing but we need to earn that confidence through how reliably it behaves.
Higgs Realtime supports more than 100 languages, including the ability to switch languages during a conversation. What are the biggest technical challenges in delivering consistent conversational intelligence across different languages, accents, and communication styles, particularly when understanding emotional context and user intent?
Language coverage sounds like a list, and it isn’t. A while ago a customer needed Thai. We had partial coverage, and the first thing we discovered was that we knew absolutely nothing about Thai proper, starting with the fact that written Thai essentially has no punctuation. That breaks the sentence-boundary assumptions most speech systems are built on. We could have deferred the market. Instead we found computer scientists who spoke Thai and rebuilt the handling properly.
About six weeks later we were in Bangkok demonstrating it. During the demo I could not tell how it was going — I was listening to people test a system in a language I don’t speak — and afterwards they said it was good, which was a considerable relief. Conventions differ as much as vocabulary does. In Japanese, listeners make small acknowledgments while you are speaking to show they are following along. If the assistant stops every time to hand over the turn, it destroys the rhythm of the conversation.
The words can be perfect and the conversation still fails. And how you say something matters as much as what you say. You can use exactly the same words affectionately or sarcastically, and the appropriate response is completely different. Holding on to tone and context while the language changes mid-sentence is a large part of what makes this interesting to me.
Companies such as OpenAI and Google are investing heavily in real-time multimodal AI. How can a specialized company like Boson AI compete against organizations with significantly greater computational resources, and where do you see the greatest opportunities for differentiation in model architecture, performance, and inference economics?
Having more compute does not automatically mean you are using it well. In a live conversation you can spend a great deal of money running a very powerful model just to process someone saying “hmm,” or “yeah, probably.” Tha
Auto-fetched and translated by AI Atlas. All rights belong to the original author.