Deploy Your Agent

WebRTC & LiveKit

Build custom voice interfaces using WebRTC and LiveKit for real-time audio streaming with your AI agent.

WebRTC & LiveKit

For custom voice experiences beyond the widget, connect directly to the thinnestAI voice engine using WebRTC via LiveKit. Build your own call UI, integrate into existing apps, or create unique voice-first experiences.

Overview

thinnestAI uses LiveKit as its real-time infrastructure. LiveKit handles WebRTC connection management, audio routing, and scaling — you just connect and stream audio.

Your App (browser/mobile)
        ↓ WebRTC
LiveKit Server (thinnestAI hosted)

Voice Engine: STT → Agent → TTS
        ↓ WebRTC
Audio response streamed back

Getting a Session Token

Before connecting, request a session token from the thinnestAI API:

curl -X POST "https://api.thinnest.ai/voice/start" \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "agent_id": "agent_xyz",
    "context": {
      "user_name": "Jane Doe",
      "purpose": "support"
    }
  }'

Response:

{
  "session_id": "session_abc123",
  "token": "eyJhbGciOiJIUzI1NiIs...",
  "websocket_url": "wss://voice.thinnest.ai/ws/session_abc123",
  "livekit_url": "wss://livekit.thinnest.ai"
}

Connecting with LiveKit SDK

JavaScript / Browser

npm install livekit-client
import { Room, RoomEvent } from 'livekit-client';

// Get token from your backend
const { token, livekit_url } = await fetch('/api/start-voice-session', {
  method: 'POST',
}).then(r => r.json());

// Connect to LiveKit room
const room = new Room();
await room.connect(livekit_url, token);

// Publish microphone
const stream = await navigator.mediaDevices.getUserMedia({ audio: true });
const track = stream.getAudioTracks()[0];
await room.localParticipant.publishTrack(track);

// Listen for agent audio
room.on(RoomEvent.TrackSubscribed, (track, publication, participant) => {
  if (track.kind === 'audio') {
    const element = track.attach();
    document.body.appendChild(element);
  }
});

// Disconnect when done
function hangUp() {
  room.disconnect();
}

React

import { LiveKitRoom, useVoiceAssistant } from '@livekit/components-react';

function VoiceCall({ token, serverUrl }) {
  return (
    <LiveKitRoom token={token} serverUrl={serverUrl} connect={true}>
      <VoiceUI />
    </LiveKitRoom>
  );
}

function VoiceUI() {
  const { state, audioTrack } = useVoiceAssistant();

  return (
    <div>
      <p>Status: {state}</p>
      {audioTrack && <audio autoPlay ref={(el) => {
        if (el) audioTrack.attach(el);
      }} />}
    </div>
  );
}

Python

pip install livekit
import asyncio
from livekit import rtc

async def connect_voice(token: str, url: str):
    room = rtc.Room()
    await room.connect(url, token)

    # Handle incoming audio from agent
    @room.on("track_subscribed")
    def on_track(track, publication, participant):
        if track.kind == rtc.TrackKind.KIND_AUDIO:
            audio_stream = rtc.AudioStream(track)
            # Process audio frames...

    # Publish microphone audio
    source = rtc.AudioSource(sample_rate=16000, num_channels=1)
    track = rtc.LocalAudioTrack.create_audio_track("mic", source)
    await room.local_participant.publish_track(track)

Mobile (React Native)

npm install @livekit/react-native
import { Room } from '@livekit/react-native';

const room = new Room();
await room.connect(livekitUrl, token);

// Publish microphone
await room.localParticipant.setMicrophoneEnabled(true);

Session Parameters

ParameterTypeRequiredDescription
agent_idstringYesVoice agent to connect to
contextobjectNoUser/session context for the agent
recordingbooleanNoRecord the session (default: agent setting)
max_durationintegerNoMax session length in seconds

Events

Monitor session state via LiveKit room events:

EventDescription
connectedWebRTC connection established
disconnectedSession ended
track_subscribedAgent audio track available
track_unsubscribedAgent audio track removed
data_receivedMetadata from the voice engine (transcripts, etc.)

Receiving Transcripts

The voice engine sends real-time transcripts via LiveKit data channels:

room.on(RoomEvent.DataReceived, (payload, participant) => {
  const data = JSON.parse(new TextDecoder().decode(payload));

  if (data.type === 'transcript') {
    console.log(`${data.role}: ${data.text}`);
    // role: "user" or "agent"
  }

  if (data.type === 'call_ended') {
    console.log('Call ended, duration:', data.duration);
  }
});

Architecture

┌──────────────────────────────────┐
│  Your App (Browser/Mobile/IoT)   │
│  LiveKit Client SDK              │
└────────────┬─────────────────────┘
             │ WebRTC (audio + data)

┌──────────────────────────────────┐
│  LiveKit Server                  │
│  (thinnestAI hosted)             │
│  - Room management               │
│  - Audio routing                 │
│  - Scaling & load balancing      │
└────────────┬─────────────────────┘


┌──────────────────────────────────┐
│  thinnestAI Voice Engine         │
│  - STT (Deepgram / Google)       │
│  - Agent Processing (LLM)        │
│  - TTS (Aero / ElevenLabs / etc) │
│  - Tool execution                │
└──────────────────────────────────┘

vs Voice Widget

FeatureWebRTC/LiveKitVoice Widget
UIBuild your ownPre-built floating button
FlexibilityFull controlConfiguration-based
Setup timeHoursMinutes
MobileNative SDKs availableBrowser only
Best forCustom apps, kiosks, IoTWebsites

Use WebRTC/LiveKit when you need a custom voice interface. Use the Voice Widget for quick website integration.

Troubleshooting

IssueSolution
Connection failsCheck firewall allows WebRTC (UDP ports 443, 10000-20000)
No audio from agentVerify you're subscribing to remote tracks
EchoEnable echo cancellation in audio constraints
High latencyUse a server region close to your users
Token expiredTokens are valid for 10 minutes — request a new one

On this page