What is LiveKit Agents SDK?
LiveKit Agents SDK is an open-source Python framework for building real-time, low-latency AI voice agents. It connects speech-to-text (STT), large language models (LLM), and text-to-speech (TTS) into a pipeline that handles inbound or outbound phone calls via SIP integration. CelloIP Technologies specialises in building production LiveKit voice agents for customer service automation, appointment scheduling, sales outreach, and AI-powered IVR systems.
LiveKit vs VAPI vs Retell AI — Which is Better?
LiveKit is best for HIPAA-compliant, high-volume (50K+ minutes/month) deployments because it charges infrastructure costs only with no per-minute fees. VAPI is best for rapid prototyping with managed infrastructure. Retell AI offers a no-code interface for simple agents. CelloIP builds on all three platforms but recommends LiveKit for enterprise production deployments needing full data control and cost efficiency at scale.
How to Build a Voice Agent with LiveKit
A production LiveKit voice agent requires: (1) a LiveKit SIP trunk connecting to your telephony carrier, (2) an Agent worker running the Agents SDK pipeline with Deepgram STT, GPT-4o LLM, and ElevenLabs TTS, (3) a dispatch rule to route inbound calls to the agent, and (4) a monitoring dashboard for latency and transcription quality. CelloIP delivers fully production-ready LiveKit voice agent systems in 4–8 weeks.
What does CelloIP build with LiveKit Voice Agents?
CelloIP Technologies builds: inbound customer service voice bots, outbound AI sales and appointment-booking agents, multilingual IVR replacements, real-time call transcription systems, HIPAA-compliant healthcare intake agents, and LiveKit + Asterisk/FreeSWITCH hybrid telephony platforms. Engagements start from $25/hr with 48-hour onboarding.
LiveKit Voice Agents: Build Production-Ready AI Voice Bots
CelloIP builds production-grade AI voice agents using the LiveKit Agents SDK. Our STT→LLM→TTS pipeline integrates with Deepgram, GPT-4o, Claude, ElevenLabs, and Cartesia. Achieve sub-500ms latency, SIP trunk integration for inbound PSTN calls, 100% open-source design, and support for 10+ AI providers. Self-hosted on-premise with Python SDK and MCP tool support.
Create AI Voice Agents with LiveKit
Production-grade STT→LLM→TTS pipeline. Sub-500ms latency. Works over SIP for inbound PSTN calls.
AI Voice Pipeline
Voice Agent Architecture
VoicePipelineAgent
Sequential STT→LLM→TTS, streaming, interruption handling. Best for most use cases.
MultimodalAgent
OpenAI Realtime API or Gemini Live speech-to-speech. Ultra-low latency, no transcription.
Custom Pipeline
Full control over audio processing, custom VAD, tool calling, multi-modal input.
STT / LLM / TTS Providers
| Provider | Type | Latency | Quality | Self-Host | Best For |
|---|---|---|---|---|---|
| Deepgram Nova-2 | STT | ~200ms | ★★★★★ | No | Best overall |
| Whisper | STT | ~400ms | ★★★★ | Yes | On-premise |
| AssemblyAI | STT | ~250ms | ★★★★ | No | Accuracy |
| Azure Speech | STT | ~300ms | ★★★★ | No | Enterprise |
| GPT-4o | LLM | ~300ms | ★★★★★ | No | Best quality |
| Claude 3.5 Haiku | LLM | ~200ms | ★★★★★ | No | Fast + smart |
| LLaMA 3 | LLM | ~100ms | ★★★★ | Yes | On-premise |
| ElevenLabs | TTS | ~200ms | ★★★★★ | No | Voice quality |
| Cartesia Sonic | TTS | ~100ms | ★★★★★ | No | Fastest |
| OpenAI TTS | TTS | ~300ms | ★★★★ | No | Simple |
Key Agent Capabilities
Semantic Turn Detection
Transformer model reduces interruptions and improves natural conversation flow.
Streaming Pipeline
STT→LLM→TTS overlap enables sub-500ms response times.
Tool/Function Calling
CRM lookup, calendar booking, DB queries, MCP tools integration.
Interruption Handling
VAD detects user barge-in, gracefully stops TTS and resets state.
Multi-language Support
Automatic language detection, seamless language switching mid-call.
DTMF + IVR Fallback
Handle keypress input when speech recognition fails.
Minimal Python Voice Agent
from livekit import agents
from livekit.agents import AutoSubscribe, JobContext, WorkerOptions, cli
from livekit.agents.voice import Agent, AgentSession
from livekit.plugins import deepgram, openai, elevenlabs, silero
async def entrypoint(ctx: JobContext):
await ctx.connect(auto_subscribe=AutoSubscribe.AUDIO_ONLY)
session = AgentSession(
vad=silero.VAD.load(),
stt=deepgram.STT(model="nova-2"),
llm=openai.LLM(model="gpt-4o"),
tts=elevenlabs.TTS(voice="Rachel"),
)
await session.start(ctx.room, agent=Agent(
instructions="You are a helpful customer service agent for CelloIP Technologies."
))
if __name__ == "__main__":
cli.run_app(WorkerOptions(entrypoint_fnc=entrypoint))SIP + Telephony Integration
Inbound Flow
PSTN Caller → SIP Carrier (Twilio/Telnyx) → LiveKit SIP Bridge → LiveKit Room → Voice Agent
Asterisk/FreeSWITCH Bridge
Caller → Asterisk PBX → SIP trunk to LiveKit → Voice Agent
Outbound Flow
Agent → LiveKit SIP → SIP Carrier → PSTN Number
Real-World Use Cases
Customer Service Bot
Handle tier-1 support 24/7, escalate complex issues to humans.
AI Sales Qualifier
Pre-screen leads, qualify intent, book qualified meetings.
Healthcare Triage Bot
Initial symptom collection, appointment scheduling.
Appointment Scheduler
Answer inbound calls, check calendar, confirm bookings.
Frequently Asked Questions
What is a LiveKit voice agent?
A LiveKit voice agent uses the VoicePipelineAgent architecture: speech input flows through STT (Deepgram), then to LLM (GPT-4o), then to TTS (ElevenLabs), with streaming overlap for <500ms response times.
What is the minimum latency possible?
Streaming pipeline achieves <500ms end-to-end latency. OpenAI Realtime API and Gemini Live (multimodal) can achieve <300ms with no transcription step.
Can LiveKit voice agents answer inbound phone calls?
Yes. Connect a SIP trunk (Twilio, Telnyx, etc.) to LiveKit. Use dispatch rules to route incoming DIDs to specific AI agent rooms.
How do agents handle interruptions?
VAD (Voice Activity Detection) detects when the user starts speaking (barge-in). The agent gracefully stops TTS playback and resets the conversation state.
Can I run agents on-premise?
Yes. Self-host LiveKit, use Whisper for STT, Ollama/LLaMA for LLM, and Coqui for TTS. No cloud API calls required.
Ready to Build AI Voice Agents?
Let CelloIP's expert team integrate LiveKit voice agents into your VoIP infrastructure.