> ## Documentation Index
> Fetch the complete documentation index at: https://docs.thinnest.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# WebRTC & LiveKit

> Build custom voice interfaces using WebRTC and LiveKit for real-time audio streaming with your AI agent.

# WebRTC & LiveKit

For custom voice experiences beyond the widget, connect directly to the thinnestAI voice engine using WebRTC via LiveKit. Build your own call UI, integrate into existing apps, or create unique voice-first experiences.

## Overview

thinnestAI uses [LiveKit](https://livekit.io) as its real-time infrastructure. LiveKit handles WebRTC connection management, audio routing, and scaling — you just connect and stream audio.

```
Your App (browser/mobile)
        ↓ WebRTC
LiveKit Server (thinnestAI hosted)
        ↓
Voice Engine: STT → Agent → TTS
        ↓ WebRTC
Audio response streamed back
```

## Getting a Session Token

Before connecting, request a session token from the thinnestAI API:

```bash theme={null}
curl -X POST "https://api.thinnest.ai/v1/voice/session" \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "agent_id": "YOUR_AGENT_PUBLIC_ID",
    "metadata": {
      "user_name": "Jane Doe",
      "purpose": "support"
    }
  }'
```

Response:

```json theme={null}
{
  "session_id": "session_abc123",
  "status": "active",
  "agent_id": "ag_pub_xyz789",
  "web_call_url": "https://app.thinnest.ai/call/session_abc123",
  "token": "eyJhbGciOiJIUzI1NiIs...",
  "livekit_url": "wss://livekit.thinnest.ai",
  "room_name": "api-123-abcde",
  "created_at": "2024-03-01T12:00:00Z",
  "expires_in": 300
}
```

## Connecting with LiveKit SDK

### JavaScript / Browser

```bash theme={null}
npm install livekit-client
```

```javascript theme={null}
import { Room, RoomEvent } from 'livekit-client';

// Get token from your backend endpoint that calls thinnestAI API
const { token, livekit_url } = await fetch('/api/start-voice-session', {
  method: 'POST',
}).then(r => r.json());

// Connect to LiveKit room
const room = new Room();
await room.connect(livekit_url, token);

// Publish microphone
const stream = await navigator.mediaDevices.getUserMedia({ audio: true });
const track = stream.getAudioTracks()[0];
await room.localParticipant.publishTrack(track);

// Listen for agent audio
room.on(RoomEvent.TrackSubscribed, (track, publication, participant) => {
  if (track.kind === 'audio') {
    const element = track.attach();
    document.body.appendChild(element);
  }
});

// Disconnect when done
function hangUp() {
  room.disconnect();
}
```

### React

```jsx theme={null}
import { LiveKitRoom, useVoiceAssistant } from '@livekit/components-react';

function VoiceCall({ token, serverUrl }) {
  return (
    <LiveKitRoom token={token} serverUrl={serverUrl} connect={true}>
      <VoiceUI />
    </LiveKitRoom>
  );
}

function VoiceUI() {
  const { state, audioTrack } = useVoiceAssistant();

  return (
    <div>
      <p>Status: {state}</p>
      {audioTrack && <audio autoPlay ref={(el) => {
        if (el) audioTrack.attach(el);
      }} />}
    </div>
  );
}
```

### Python

```bash theme={null}
pip install livekit
```

```python theme={null}
import asyncio
from livekit import rtc

async def connect_voice(token: str, url: str):
    room = rtc.Room()
    await room.connect(url, token)

    # Handle incoming audio from agent
    @room.on("track_subscribed")
    def on_track(track, publication, participant):
        if track.kind == rtc.TrackKind.KIND_AUDIO:
            audio_stream = rtc.AudioStream(track)
            # Process audio frames...

    # Publish microphone audio
    source = rtc.AudioSource(sample_rate=16000, num_channels=1)
    track = rtc.LocalAudioTrack.create_audio_track("mic", source)
    await room.local_participant.publish_track(track)
```

### Mobile (React Native)

```bash theme={null}
npm install @livekit/react-native
```

```jsx theme={null}
import { Room } from '@livekit/react-native';

const room = new Room();
await room.connect(livekitUrl, token);

// Publish microphone
await room.localParticipant.setMicrophoneEnabled(true);
```

## Session Parameters

| Parameter       | Type   | Required | Description                             |
| --------------- | ------ | -------- | --------------------------------------- |
| `agent_id`      | string | Yes      | Voice agent public ID                   |
| `metadata`      | object | No       | User/session context data for the agent |
| `stt_provider`  | string | No       | Override STT engine                     |
| `tts_provider`  | string | No       | Override TTS engine                     |
| `voice_id`      | string | No       | Override bot voice ID                   |
| `first_message` | string | No       | Override first greeting message         |

## Events

Monitor session state via LiveKit room events:

| Event                | Description                                        |
| -------------------- | -------------------------------------------------- |
| `connected`          | WebRTC connection established                      |
| `disconnected`       | Session ended                                      |
| `track_subscribed`   | Agent audio track available                        |
| `track_unsubscribed` | Agent audio track removed                          |
| `data_received`      | Metadata from the voice engine (transcripts, etc.) |

### Receiving Transcripts

The voice engine sends real-time transcripts via LiveKit data channels:

```javascript theme={null}
room.on(RoomEvent.DataReceived, (payload, participant) => {
  const data = JSON.parse(new TextDecoder().decode(payload));

  if (data.type === 'transcript') {
    console.log(`${data.role}: ${data.text}`);
    // role: "user" or "agent"
  }

  if (data.type === 'call_ended') {
    console.log('Call ended, duration:', data.duration);
  }
});
```

## Architecture

```
┌──────────────────────────────────┐
│  Your App (Browser/Mobile/IoT)   │
│  LiveKit Client SDK              │
└────────────┬─────────────────────┘
             │ WebRTC (audio + data)
             ▼
┌──────────────────────────────────┐
│  LiveKit Server                  │
│  (thinnestAI hosted)             │
│  - Room management               │
│  - Audio routing                 │
│  - Scaling & load balancing      │
└────────────┬─────────────────────┘
             │
             ▼
┌──────────────────────────────────┐
│  thinnestAI Voice Engine         │
│  - STT (Deepgram / Google)       │
│  - Agent Processing (LLM)        │
│  - TTS (Aero / ElevenLabs / etc) │
│  - Tool execution                │
└──────────────────────────────────┘
```

## vs Voice Widget

| Feature     | WebRTC/LiveKit           | Voice Widget              |
| ----------- | ------------------------ | ------------------------- |
| UI          | Build your own           | Pre-built floating button |
| Flexibility | Full control             | Configuration-based       |
| Setup time  | Hours                    | Minutes                   |
| Mobile      | Native SDKs available    | Browser only              |
| Best for    | Custom apps, kiosks, IoT | Websites                  |

Use WebRTC/LiveKit when you need a custom voice interface. Use the [Voice Widget](/docs/deploy/voice-widget) for quick website integration.

## Troubleshooting

| Issue               | Solution                                                  |
| ------------------- | --------------------------------------------------------- |
| Connection fails    | Check firewall allows WebRTC (UDP ports 443, 10000-20000) |
| No audio from agent | Verify you're subscribing to remote tracks                |
| Echo                | Enable echo cancellation in audio constraints             |
| High latency        | Use a server region close to your users                   |
| Token expired       | Tokens are valid for 5 minutes — request a new one        |