LiveKit Agents

A LiveKit Agents program is an AgentServer process that registers with a LiveKit server and receives jobs through dispatch. For each job, an entrypoint joins a LiveKit room and starts an AgentSession, which wires audio and video to a pipeline of VAD, speech-to-text, LLM and text-to-speech plugins, or to a realtime speech model. The agent is a participant in the room, so clients connect through LiveKit's WebRTC SDKs or by phone over SIP.

Within a session, one Agent is active at a time. An agent holds instructions, tools and optionally its own models; a tool can return another agent to hand off control, and the docs describe passing chat_ctx to keep history or starting fresh. Tasks take temporary control to complete one objective and return a typed result, and task groups chain tasks with backtracking. The docs present these as combinable patterns: single agent with tools, supervisor with tasks, subagent delegation to a background model, and handoffs.

Conversation history lives in the session's ChatContext, and custom per-session state goes in a typed userdata object. Long-term knowledge and actions come from tools, RAG or MCP servers. The same framework exists for Node.js in a separate repository (AgentsJS).

Because the user is live on the call, the main human controls are conversational: turn detection lets the user interrupt the agent mid-sentence, and adaptive interruption handling separates real interruptions from backchannel sounds. WarmTransferTask hands a call to a human operator over SIP, plays hold music, and gives the operator a context summary before connecting the caller. Tool calls can be forwarded to the user's frontend over RPC, so the client app fulfils or confirms them, and tasks can collect explicit consent before the flow continues. The docs do not describe a separate approval queue for tool calls outside the conversation.

pip install "livekit-agents[openai,deepgram,cartesia]"

from livekit.agents import Agent, AgentServer, AgentSession, JobContext, RunContext, cli, function_tool, inference
class BillingAgent(Agent):
    def __init__(self) -> None:
        super().__init__(instructions="You handle billing questions. Keep answers short.")
class FrontDesk(Agent):
    def __init__(self) -> None:
        super().__init__(instructions="Greet the caller and find out what they need.")
    @function_tool
    async def transfer_to_billing(self, context: RunContext):
        """Called when the caller asks about invoices or payments."""
        return BillingAgent(chat_ctx=self.chat_ctx), "Transferring you to billing."
server = AgentServer()
@server.rtc_session()
async def entrypoint(ctx: JobContext):
    session = AgentSession(vad=inference.VAD(), stt="deepgram/nova-3", llm="google/gemma-4-31b-it", tts="cartesia/sonic-3:9626c31c-bec5-4cca-baa8-f8ba9e84c8bc")
    await session.start(agent=FrontDesk(), room=ctx.room)
if __name__ == "__main__":
    cli.run_app(server)

• Requires Python >= 3.10 (requires-python = ">=3.10,<3.15"); the Node.js edition is a separate package and repository.
• The agent needs a LiveKit server: set LIVEKIT_URL, LIVEKIT_API_KEY and LIVEKIT_API_SECRET for LiveKit Cloud or a self-hosted server.
• Model strings such as deepgram/nova-3 use LiveKit Inference on LiveKit Cloud; to call providers with your own keys, use plugin classes (for example openai.LLM) instead.
• MCP needs the mcp extra (livekit-agents[mcp]); the older mcp_servers parameter is deprecated in favour of MCPToolset in tools.
• 1.8.0 changed OpenTelemetry output (conversation content moved from span events to attributes), which breaks dashboards built on the old events.

Yes, in Python. mcp.MCPToolset wraps an HTTP or stdio MCP server and passes its tools to an agent. The docs mark MCP as unavailable in the Node.js SDK.

The framework is Apache-2.0 and can run against a self-hosted LiveKit server. LiveKit Cloud has a free Build plan and paid plans for deployment, inference and observability at larger scale.

This repository is the Python SDK (Python 3.10 to 3.14). A separate AgentsJS repository provides the Node.js edition, with some features, such as MCP, missing there.

One agent is active per session. A tool can return another agent to hand off control, tasks take temporary control and return typed results, and a supervisor agent can route work to tasks.

The README points coding assistants to a LiveKit Docs MCP server and an installable agent skill. These are documentation aids, not an integration that runs coding agents.