Jake Dorsey的Block 发布是公司研究实验室的一个新声音模型，该型号的称公司称为Dessa，全面接受人语训练，并拥有单张NVIDIA GPU.
德萨是一个全复式模型，意思是它可以同时说话和聆听，就像今年早些时候的GPT-Live模型OpenAI 发布一样。 通常，语音模型与大型语言模型配对，用于知识检索。 Dessa的知识来源于对话数据。
Block 称认为，目前的模型可以听起来不自然和令人烦恼，因为它们的文本在管道中驱动语音生成。 ElevenLabs v4或Fish Audio Drama 3等许多模型在语音生成中使用语音标记来驱动情感。 Dorsey的声称公司将情感植入其模式。 并不清楚这种情绪是否受到某种标签的左右。
“与当前声音的交流，AI会很烂。 将你的声音变成文字，世界上最强大的AI无法听到你的感受。 称公司在博客文章中表示
agent有不同的声音、节奏和情感。 例如，在谈话中，机器人称由于工作而感到疲惫，这时听起来很疲惫。
在下面的例子中，博特在谈论一个人让自己感到舒服，同时发出情感和几乎崩溃的声音。
又一个显示是Block团队成员与语音助理一起玩游戏。
该型号可在当地运行于消费级计算机如MacBook M4 Max或更好的公司称。 然而，目前公众无法进入德萨，布洛克已发出呼吁，要求安全研究人员测试该模型。 这很可能会检验一个声音agent在另一端可以欺骗人的情况。
Block从事金融业务，并向小企业出售其解决方案。 Square已经使用语音AI来处理订单。 但凭借自己的能运行端侧的语音模式，它可以以较低的成本为更多的商业和消费者体验提供动力。
本文由 AI图鉴自动抓取并翻译，版权归原作者所有；如需查看原文请点下方链接。
Jack Dorsey’s Block released a new voice model out of the company’s research lab that runs locally and sounds strangely human with its expressions like laughs, pauses, and cries. The company said the model, called Dessa, is trained fully on human speech and a single NVIDIA GPU.
Dessa is a full-duplex model, which means it can talk and listen at the same time, just like the GPT-Live model OpenAI released earlier this year. Typically, speech models are paired with a large language model for knowledge retrieval. Block said that Dessa’s knowledge stems from the conversational data it has.
Block said that the current models can sound unnatural and annoying because they have text in the pipeline that drives speech generation. A lot of models like ElevenLabs v4 or Fish Audio Drama 3 use voice tags to drive emotion in speech generation. Dorsey’s company claims that emotion is built into its model. It’s not clear if that emotion is being steered by some kind of tags.
“Interacting with current voice AI can just plain suck. It turns your voice into text and throws away an incredible amount of information text alone can’t carry. The most powerful AIs in the world can’t hear how you feel. We think that’s ridiculous, leads to a worse experience, and is potentially dangerous,” the company said in a blog post.
The company’s researchers posted multiple examples of conversations with Dessa where the agent had different voices, cadences, and emotions. For instance, in the conversation, the bot said it was tired from work, and at that point it sounded tired.
In the example below, the bot was talking about the person making it comfortable while sounding emotional and almost breaking down.
Another one shows a Block team member playing a game with the voice assistant.
The company said that the model can run locally on a consumer-grade computer such as a MacBook M4 Max or better. However, there is no public access to Dessa at the moment, and Block has put out a call for safety researchers to test the model. This is likely to test out cases where a voice agent could trick people on the other end.
Block is in the business of finance and selling its solutions to small businesses. Square already uses voice AI to handle ordering. But with its own voice model that can run on-device, it can power a lot more business and consumer experiences at a lower cost.
Auto-fetched and translated by AI Atlas. All rights belong to the original author.