WebRTC & LiveKit
Build custom voice interfaces using WebRTC and LiveKit for real-time audio streaming with your AI agent.
WebRTC & LiveKit
For custom voice experiences beyond the widget, connect directly to the thinnestAI voice engine using WebRTC via LiveKit. Build your own call UI, integrate into existing apps, or create unique voice-first experiences.
Overview
thinnestAI uses LiveKit as its real-time infrastructure. LiveKit handles WebRTC connection management, audio routing, and scaling — you just connect and stream audio.
Your App (browser/mobile)
↓ WebRTC
LiveKit Server (thinnestAI hosted)
↓
Voice Engine: STT → Agent → TTS
↓ WebRTC
Audio response streamed backGetting a Session Token
Before connecting, request a session token from the thinnestAI API:
curl -X POST "https://api.thinnest.ai/voice/start" \
-H "Authorization: Bearer YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"agent_id": "agent_xyz",
"context": {
"user_name": "Jane Doe",
"purpose": "support"
}
}'Response:
{
"session_id": "session_abc123",
"token": "eyJhbGciOiJIUzI1NiIs...",
"websocket_url": "wss://voice.thinnest.ai/ws/session_abc123",
"livekit_url": "wss://livekit.thinnest.ai"
}Connecting with LiveKit SDK
JavaScript / Browser
npm install livekit-clientimport { Room, RoomEvent } from 'livekit-client';
// Get token from your backend
const { token, livekit_url } = await fetch('/api/start-voice-session', {
method: 'POST',
}).then(r => r.json());
// Connect to LiveKit room
const room = new Room();
await room.connect(livekit_url, token);
// Publish microphone
const stream = await navigator.mediaDevices.getUserMedia({ audio: true });
const track = stream.getAudioTracks()[0];
await room.localParticipant.publishTrack(track);
// Listen for agent audio
room.on(RoomEvent.TrackSubscribed, (track, publication, participant) => {
if (track.kind === 'audio') {
const element = track.attach();
document.body.appendChild(element);
}
});
// Disconnect when done
function hangUp() {
room.disconnect();
}React
import { LiveKitRoom, useVoiceAssistant } from '@livekit/components-react';
function VoiceCall({ token, serverUrl }) {
return (
<LiveKitRoom token={token} serverUrl={serverUrl} connect={true}>
<VoiceUI />
</LiveKitRoom>
);
}
function VoiceUI() {
const { state, audioTrack } = useVoiceAssistant();
return (
<div>
<p>Status: {state}</p>
{audioTrack && <audio autoPlay ref={(el) => {
if (el) audioTrack.attach(el);
}} />}
</div>
);
}Python
pip install livekitimport asyncio
from livekit import rtc
async def connect_voice(token: str, url: str):
room = rtc.Room()
await room.connect(url, token)
# Handle incoming audio from agent
@room.on("track_subscribed")
def on_track(track, publication, participant):
if track.kind == rtc.TrackKind.KIND_AUDIO:
audio_stream = rtc.AudioStream(track)
# Process audio frames...
# Publish microphone audio
source = rtc.AudioSource(sample_rate=16000, num_channels=1)
track = rtc.LocalAudioTrack.create_audio_track("mic", source)
await room.local_participant.publish_track(track)Mobile (React Native)
npm install @livekit/react-nativeimport { Room } from '@livekit/react-native';
const room = new Room();
await room.connect(livekitUrl, token);
// Publish microphone
await room.localParticipant.setMicrophoneEnabled(true);Session Parameters
| Parameter | Type | Required | Description |
|---|---|---|---|
agent_id | string | Yes | Voice agent to connect to |
context | object | No | User/session context for the agent |
recording | boolean | No | Record the session (default: agent setting) |
max_duration | integer | No | Max session length in seconds |
Events
Monitor session state via LiveKit room events:
| Event | Description |
|---|---|
connected | WebRTC connection established |
disconnected | Session ended |
track_subscribed | Agent audio track available |
track_unsubscribed | Agent audio track removed |
data_received | Metadata from the voice engine (transcripts, etc.) |
Receiving Transcripts
The voice engine sends real-time transcripts via LiveKit data channels:
room.on(RoomEvent.DataReceived, (payload, participant) => {
const data = JSON.parse(new TextDecoder().decode(payload));
if (data.type === 'transcript') {
console.log(`${data.role}: ${data.text}`);
// role: "user" or "agent"
}
if (data.type === 'call_ended') {
console.log('Call ended, duration:', data.duration);
}
});Architecture
┌──────────────────────────────────┐
│ Your App (Browser/Mobile/IoT) │
│ LiveKit Client SDK │
└────────────┬─────────────────────┘
│ WebRTC (audio + data)
▼
┌──────────────────────────────────┐
│ LiveKit Server │
│ (thinnestAI hosted) │
│ - Room management │
│ - Audio routing │
│ - Scaling & load balancing │
└────────────┬─────────────────────┘
│
▼
┌──────────────────────────────────┐
│ thinnestAI Voice Engine │
│ - STT (Deepgram / Google) │
│ - Agent Processing (LLM) │
│ - TTS (Aero / ElevenLabs / etc) │
│ - Tool execution │
└──────────────────────────────────┘vs Voice Widget
| Feature | WebRTC/LiveKit | Voice Widget |
|---|---|---|
| UI | Build your own | Pre-built floating button |
| Flexibility | Full control | Configuration-based |
| Setup time | Hours | Minutes |
| Mobile | Native SDKs available | Browser only |
| Best for | Custom apps, kiosks, IoT | Websites |
Use WebRTC/LiveKit when you need a custom voice interface. Use the Voice Widget for quick website integration.
Troubleshooting
| Issue | Solution |
|---|---|
| Connection fails | Check firewall allows WebRTC (UDP ports 443, 10000-20000) |
| No audio from agent | Verify you're subscribing to remote tracks |
| Echo | Enable echo cancellation in audio constraints |
| High latency | Use a server region close to your users |
| Token expired | Tokens are valid for 10 minutes — request a new one |