在AWS上建立超低频语音AI应用程序

中文版正在随流水线自动整理，稍后刷新即可；可先阅读下方英文原文。

本文由 AI图鉴自动抓取并翻译，版权归原作者所有；如需查看原文请点下方链接。

Artificial Intelligence

Building ultra-low-latency voice AI applications on AWS

Conversational AI is transforming how humans interact with technology. From virtual assistants and AI-powered contact centers to real-time translation and accessibility tools, voice-first interfaces are rapidly becoming the default mode of interaction for millions of users worldwide.

Yet delivering a truly natural voice experience remains one of the hardest challenges in AI infrastructure. Users expect human-like responsiveness. Excess latency can break conversational flow and erode trust. To feel natural, AI-generated speech must arrive within the same temporal window as human turn-taking.

This post introduces how Cartesia, an AWS Partner specializing in real-time voice AI, uses purpose-built State Space Models (SSMs) and AWS infrastructure to deliver sub-100ms text-to-speech and speech-to-text at global scale. Developers can build voice experiences that feel indistinguishable from human conversation.

The rise of conversational AI and latency expectations

The demand for real-time voice AI is growing rapidly across industries:

- Contact centers: AI agents handling millions of customer interactions require instantaneous speech generation to maintain natural dialogue.

- Telehealth: Remote patient consultations demand low-latency voice for accurate, empathetic communication.

- Gaming and virtual worlds: Immersive experiences require real-time voice synthesis for non-player characters (NPCs) and player interactions.

- Accessibility: Screen readers and voice interfaces must respond instantly to serve users with visual impairments.

- Inbound and outbound sales: AI sales agents must respond without awkward pauses to qualify leads, handle objections, and keep prospects on the line.

Traditional transformer-based text-to-speech (TTS) models struggle with streaming latency because they process entire sequences before producing output. This architectural limitation creates a fundamental tension: higher quality often means higher latency. A new class of models, State Space Models, resolves this trade-off.

AI for voice: State Space Models (SSM)

State Space Models (SSMs) take a fundamentally different approach to processing sequences than transformers, the architecture behind most of today’s AI. Transformers use a mechanism called attention, which compares every element in a sequence to every other element. As a result, the amount of computation grows quadratically as the sequence gets longer. SSMs instead treat a sequence as a system that evolves continuously over time, processing it step by step and carrying a compact internal state forward. This can be converted into a form computers handle efficiently, keeping the cost low for certain long sequences. This efficiency makes SSMs well suited to real-time, streaming data like live speech.

Key advantages of SSMs for audio processing include:

- High-quality long-range dependencies: Structured SSMs are a form of linear recurrence, but their state parameterization preserves information over long audio sequences, avoiding the long-range signal decay that limits classic recurrent neural networks (RNNs) such as long short-term memory (LSTM) networks.

- Constant memory footprint: Unlike transformers that cache all previous tokens, SSMs maintain a fixed-size hidden state.

- Native streaming support: The recurrent structure processes input token-by-token, so it supports true streaming without architectural modifications.

These properties make SSMs uniquely suited for real-time voice applications where both quality and latency are non-negotiable.

How it works: SSMs for real-time voice

Cartesia puts these SSM advantages to work in two production voice models: Sonic for speech generation and Ink for transcription.

Introducing Cartesia

Cartesia is a real-time multimodal intelligence company that has built a voice AI application, purpose-built on State Space Model architecture. Founded by researchers who pioneered SSM advances, Cartesia’s application is designed for streaming inference at the edge of human perception.

The application features two core models, Sonic for text-to-speech and Ink for speech-to-text, both optimized for sub-100ms first-byte latency while maintaining studio-quality output.

Sonic (text-to-speech)

Sonic is Cartesia’s flagship streaming text-to-speech model, delivering ultra-realistic speech synthesis with sub-100ms latency:

- First audio byte in approximately 90ms: Users hear the voice respond almost instantaneously.

- Ultra-realistic voice quality: Natural prosody, emotion, and expressiveness across over 44 languages.

- Voice cloning with minimal data: Create natural-sounding custom voices from 10-second audio samples.

- Streaming-native architecture: Audio begins generating before the full text input is received.

- Context-aware synthesis: Maintains consistent tone and style across long passages.

Sonic is recommended for teams that want the fastest time-to-production with minimal operational overhead. Call the API and receive streaming audio in real time.

Ink (speech-to-text)

Ink is Cartesia’s streaming speech-to-text model, purpose-built for conversational AI applications:

- Native turn detection: Automatically identifies when a speaker has finished their turn, alleviating awkward pauses without a separate voice activity detection (VAD) or end pointing model.

- Context-aware end pointing: Uses semantic understanding to distinguish between mid-sentence pauses and true conversation boundaries.

- Streaming transcription: Emits partial transcripts in real time as the user speaks, so a downstream large language model (LLM) can begin forming its response before the turn ends.

- Noise robustness: Maintains accuracy in noisy environments common in real-world deployments.

- Multi-speaker support: Handles overlapping speech and speaker transitions gracefully.

Together, Sonic and Ink provide the complete voice AI stack needed to build conversational agents that feel human.

Deploying Cartesia on AWS

If you need data sovereignty, latency control, or compliance, you can deploy Cartesia models directly into your own AWS account. Sonic is delivered as an Amazon SageMaker AI model package through AWS Marketplace, which you deploy to a SageMaker AI endpoint running entirely within your virtual private cloud (VPC). Inference data doesn’t leave your account.

The deployment patterns in this section cover Sonic. Ink is currently available through the Cartesia API, which you can purchase through the Cartesia Voice AI Platform listing in AWS Marketplace. Two primary patterns are available for Sonic.

Option 1: Amazon SageMaker AI (AWS Marketplace model package)

Deploying on Amazon SageMaker AI is the fastest path to a managed, in-VPC Sonic endpoint. Sonic ships as an AWS Marketplace model package, so there’s a one-time subscribe step that grants your account an AWS Region-specific model-package ARN. You then create the SageMaker AI endpoint from that ARN, either from the console or with the SageMaker or Boto3 SDK. The following steps follow Cartesia’s official deployment notebook.

Prerequisites

Run from an Amazon SageMaker Notebook Instance or SageMaker Studio.

The AWS Identity and Access Management (IAM) execution role needs AmazonSageMakerFullAccess.

To subscribe, your role (and you) need aws-marketplace:Subscribe, aws-marketplace:Unsubscribe, and aws-marketplace:ViewSubscriptions. Otherwise, the account must already be subscribed to the listing.

Before deployment, choose a supported instance type from the model package’s SupportedRealtimeInferenceInstanceTypes field:

- ml.g6e.xlarge or larger

- ml.g7e.2xlarge or larger

- ml.g7.2xlarge or larger

- ml.p5.48xlarge

- ml.p5en.48xlarge

If your chosen instance type’s applied value is below your current usage plus your planned endpoint instance count, you may need to request a service quota increase. See SageMaker AI quotas for details.

Deploy Sonic 3.6 to a SageMaker endpoint

In Amazon SageMaker JumpStart, you can subscribe to its AWS Marketplace model package, create a SageMaker model, and host it on a real-time endpoint. The following four steps show the deployment path.

- In SageMaker AI, open AWS Marketplace resources, then All products. Search for Sonic 3.6 SageMaker and choose the Cartesia listing.

- Choose View purchase options, review the pricing and terms, and subscribe. The subscription grants your AWS account access to the model package. It doesn’t deploy resources.

- Open Deployments & inference > Deployable models, choose Create model, select Use a model package subscription from AWS Marketplace, and select Sonic 3.6 SageMaker. Choose an execution role, then create the model.

- Create a real-time endpoint from the model with a supported GPU instance, such as ml.g6e.2xlarge. The endpoint first appears with Creating status. Wait for InService before invoking it.

The model-package ARN is Region-specific. To verify the Sonic 3.6 package in US East (N. Virginia) with the SageMaker API, run:

After the endpoint reaches InService, invoke it with the SageMaker Runtime InvokeEndpointWithResponseStream API and process the returned audio stream in your application.

Option 2: Self-managed on EC2 GPU instances

Some teams have existing GPU infrastructure, strict networking requirements, or a need to co-locate Cartesia with other services. For these cases, you can run the model container yourself on Amazon Elastic Compute Cloud (Amazon EC2) GPU instances. This gives you full control over the runtime, scaling, and network path. It uses Cartesia’s container image, obtained from Cartesia, rather than the AWS Marketplace model package.

Obtain the container image. Request the Sonic container image and licensing terms from Cartesia. Replicate it into your own private Amazon Elastic Container Registry (Amazon ECR) repository so your compute plane can pull it with IAM-scoped credentials rather than reaching out to an external registry at scale.

Choose your GPU instance family. Sonic runs on NVIDIA GPU instances. The G6 (L4) and G6e (L40S) families are the optimal balance for real-time TTS. The earlier G5 (A10G) also works. Match GPU memory and vCPU to your target concurrency and prefer a single-GPU instance per model replica for simplest scaling.

Provision the compute plane (Amazon Elastic Container Service (Amazon ECS) or Amazon Elastic Kubernetes Service (Amazon EKS)). For ECS: register a GPU task definition (set the container’s GPU resource requirement, for example, resourceRequirements with type GPU) and run it as a service behind an Application Load Balancer. For EKS: create a GPU node group (or Karpenter provisioner) using the EKS GPU-optimized AMI, install the NVIDIA device plugin, and deploy the container as a Deployment fronted by a Service/Ingress.

Configure networking for low latency. Place inference nodes in a cluster placement group to minimize inter-node latency, keep replicas in a single Availability Zone for the tightest path, and use VPC endpoints for ECR/S3 pulls. For hybrid or on-premises callers, terminate traffic over AWS Direct Connect or a private Network Load Balancer to avoid public-internet hops.

Load model weights efficiently. Store model weights on a shared, low-latency volume (Amazon Elastic File System (Amazon EFS) or Amazon FSx for Lustre) mounted across replicas so new pods/tasks start without each re-downloading multi-GB artifacts. Warm the model at container start and expose a health check so the load balancer only routes to ready replicas.

Tune batching and concurrency. Set the container’s max batch size and concurrent-stream limits to match your traffic profile. Higher batch size improves GPU utilization but adds first-byte latency, so tune against your sub-100ms target. Benchmark streams-per-GPU to find the replica capacity you will scale on.

Implement custom auto scaling. Scale on a signal that reflects real load, active audio-stream count or GPU utilization, rather than CPU. On Amazon ECS, use A

Auto-fetched and translated by AI Atlas. All rights belong to the original author.