Create AI Voice Agents with LiveKit

Production-grade STT→LLM→TTS pipeline. Sub-second latency. Works over SIP for inbound PSTN calls.

Response Latency
Sub-second
Open Source
100%
AI Providers
10+
Ready
SIP + WebRTC

AI Voice Pipeline

PSTN Call
↓
SIP Bridge
↓
LiveKit Room
↓
Speech-to-textstreaming
↓
Language model + toolsstreaming
↓
Text-to-speechstreaming
↓
Audio Out

Quick answer

What is a LiveKit voice agent?

A LiveKit voice agent is an AI agent built with the open-source LiveKit Agents framework that joins WebRTC rooms or phone calls through LiveKit SIP, listens with speech-to-text, decides with a language model and its tools, and replies with text-to-speech. CelloIP builds and self-hosts production LiveKit voice agents in any language.

Voice Agent Architecture

STT → LLM → TTS pipeline

An AgentSession with separate speech-to-text, language model and text-to-speech, all streaming. The most control over each step, voices and languages. Best for most phone agents.

Realtime speech-to-speech

A realtime model such as the OpenAI Realtime API or Gemini Live hears and speaks directly. Very low latency, with less control over voice, transcripts and each step.

Custom Pipeline

Full control over audio processing, custom VAD, tool calling, multi-modal input.

STT / LLM / TTS Providers

ProviderTypeLatencyQualitySelf-HostBest For
Deepgram Nova-3STTLow★★★★★NoMost English phone agents
Whisper (faster-whisper)STTMedium★★★★YesSelf-hosted, many languages
AssemblyAISTTLow★★★★NoAccuracy, diarisation
Azure or Google SpeechSTTLow★★★★NoEnterprise, wide language list
OpenAI GPT modelsLLMLow (small tiers)★★★★★NoReliable tool calling
Anthropic ClaudeLLMLow (Haiku tier)★★★★★NoCareful instruction following
Google GeminiLLMLow (Flash tier)★★★★NoSpeed and cost
Llama or QwenLLMDepends on GPUs★★★★YesData stays on your servers
ElevenLabsTTSLow★★★★★NoMost natural voices
CartesiaTTSVery low★★★★★NoFastest replies
OpenAI TTSTTSMedium★★★★NoSimple set-up

Indicative only: latency and quality vary by language, region and model version. We test candidates on your own calls before choosing.

Key Agent Capabilities

Semantic Turn Detection

Transformer model reduces interruptions and improves natural conversation flow.

Streaming Pipeline

STT→LLM→TTS overlap enables sub-second response times.

Tool/Function Calling

CRM lookup, calendar booking, DB queries, MCP tools integration.

Interruption Handling

VAD detects user barge-in, gracefully stops TTS and resets state.

Multi-language Support

Automatic language detection, seamless language switching mid-call.

DTMF + IVR Fallback

Handle keypress input when speech recognition fails.

Minimal Python Voice Agent

from livekit import agents
from livekit.agents import Agent, AgentSession
from livekit.plugins import deepgram, elevenlabs, openai, silero

class SupportAgent(Agent):
    def __init__(self) -> None:
        super().__init__(instructions=(
            "You are a friendly customer service agent. Keep replies short. "
            "If you can't help, offer to transfer the caller to a person."
        ))

async def entrypoint(ctx: agents.JobContext):
    await ctx.connect()
    session = AgentSession(
        vad=silero.VAD.load(),
        stt=deepgram.STT(model="nova-3"),
        llm=openai.LLM(model="gpt-4o-mini"),
        tts=elevenlabs.TTS(),
    )
    await session.start(room=ctx.room, agent=SupportAgent())
    await session.generate_reply(instructions="Greet the caller.")

if __name__ == "__main__":
    agents.cli.run_app(agents.WorkerOptions(entrypoint_fnc=entrypoint))

SIP + Telephony Integration

Inbound Flow

PSTN Caller → SIP Carrier (Twilio/Telnyx) → LiveKit SIP Bridge → LiveKit Room → Voice Agent

Asterisk/FreeSWITCH Bridge

Caller → Asterisk PBX → SIP trunk to LiveKit → Voice Agent

Outbound Flow

Agent → LiveKit SIP → SIP Carrier → PSTN Number

Real-World Use Cases

Customer Service Bot

Handle tier-1 support 24/7, escalate complex issues to humans.

AI Sales Qualifier

Pre-screen leads, qualify intent, book qualified meetings.

Healthcare Front Desk

Appointment booking, reminders and common patient questions, with handover to staff.

Appointment Scheduler

Answer inbound calls, check calendar, confirm bookings.

A Real LiveKit Project

CelloIP built a live AI voice agent for a contact centre in North America on LiveKit Agents. The centre's FreeSWITCH platform sends calls to LiveKit over SIP; the agent handles inbound support, outbound calls and appointment booking in English and Spanish, works with the CRM, booking system and helpdesk, and hands calls back to human agents with a summary.

Read the case study →
Managed-platform MVP
$8,000–$20,000
One use case, 4–6 weeks
Custom production agent on LiveKit
$25,000–$60,000
Telephony, integrations, 8–16 weeks
Enterprise or on-premise
Scoped after discovery
Self-hosted models, several call types

Guide ranges, matching our AI voice agent development page. Speech, model and hosting usage are separate. Step-by-step build: LiveKit voice agents guide · LiveKit with Asterisk and FreeSWITCH · self-hosted LiveKit · LiveKit vs Pipecat · LiveKit with Gemini Live

Frequently Asked Questions

What is a LiveKit voice agent?

An AI agent built with the open-source LiveKit Agents framework. It joins a WebRTC room or a phone call through LiveKit SIP, listens with streaming speech-to-text, decides what to say and do with a language model and its tools, and replies with text-to-speech, with turn detection and interruption handling built in.

Can LiveKit voice agents answer and make real phone calls?

Yes. LiveKit SIP connects to SIP trunks from carriers such as Twilio or Telnyx, or to your own Asterisk or FreeSWITCH PBX. Dispatch rules send each incoming number to the right agent, and agents can place outbound calls the same way.

How fast can a LiveKit voice agent reply?

With streaming speech-to-text, a fast language model and streaming text-to-speech hosted close together, replies typically start in about a second or less. Realtime speech-to-speech models can be faster still, with less control over each step.

Can we self-host everything?

Yes. LiveKit server, the agents and the models can all run on your own servers: for example self-hosted Whisper for speech-to-text, an open-weight language model served with vLLM, and a local text-to-speech voice. That keeps call audio and transcripts inside your infrastructure.

How long does it take and what does it cost?

As a guide, a managed-platform MVP for one use case is typically $8,000–$20,000 over 4–6 weeks, and a custom production agent on a self-hosted framework such as LiveKit $25,000–$60,000 over 8–16 weeks, depending on integrations, languages and telephony. Speech, model and hosting usage are separate running costs.

Has CelloIP built LiveKit voice agents in production?

Yes. For example, we built a live AI voice agent for a North American contact centre on LiveKit Agents, connected over SIP to the centre's FreeSWITCH platform, handling inbound support, outbound calls and appointment booking in English and Spanish. The case study describes it.

Ready to Build AI Voice Agents?

Let CelloIP's expert team integrate LiveKit voice agents into your VoIP infrastructure.

Explore SIP IntegrationHire Our DevelopersAI Voice Agent DevelopmentHire AI DevelopersDeveloper FAQs & Pricing