Multilingual Auto-Switch

The Multilingual Auto-Switch tool detects when a caller speaks a different language and automatically switches the agent’s TTS voice and STT model mid-call. The agent continues the conversation seamlessly in the new language.

Automatic language following

When your agent’s speech-to-text language is set to auto-detect (for example Sarvam Saaras, or any transcriber set to “multi”), the agent follows the caller’s language on its own — no tool, no configuration:

It detects the language of each caller message and replies in that same language.
It handles mid-call switches turn by turn — a caller can open in English, move to Hindi, and switch back, and the agent keeps up every time.
It mirrors code-mixing. A caller speaking Hinglish (Hindi and English blended in one sentence) — or Tanglish, Tenglish, and similar mixes — gets a reply in the same mixed style, rather than being pushed into one pure language.
Hindi and Marathi are told apart correctly even though they share the Devanagari script, so the agent answers in the language the caller actually spoke.

This runs on the agent’s existing voice. Use the Multilingual Auto-Switch tool below only when you also want a different voice per language — for example, a dedicated Spanish voice for Spanish callers.

How It Works

Caller (English): "Hi, I need help with my account."
Agent (English): "Of course! What's your account number?"
Caller (Spanish): "Perdona, puedo hablar en espanol?"
-> Agent detects language switch
-> Calls switch_language tool with target "es"
-> TTS switches to Spanish voice, STT to Spanish model
Agent (Spanish): "Claro que si! Cual es su numero de cuenta?"

Configuration

{
  "multilingualEnabled": true,
  "multilingualPrimaryLanguage": "en",
  "languageVoiceMap": {
    "es": {
      "ttsVoice": "aura-2-thalia-es",
      "ttsModel": "aura-2",
      "sttLanguage": "es"
    },
    "hi": {
      "ttsVoice": "aero-ananya",
      "ttsModel": "aero",
      "sttLanguage": "hi"
    },
    "fr": {
      "ttsVoice": "aura-2-thalia-fr",
      "ttsModel": "aura-2",
      "sttLanguage": "fr"
    }
  }
}

Setting	Type	Default	Description
`multilingualEnabled`	boolean	`false`	Enable multilingual auto-switch
`multilingualPrimaryLanguage`	string	`en`	Primary language code
`languageVoiceMap`	object	`{}`	Map of language code to TTS/STT config

Language Voice Map Entry

Each entry in languageVoiceMap configures the voice for that language:

Field	Type	Description
`ttsVoice`	string	Voice ID to use for this language
`ttsModel`	string	TTS model to use (e.g., `aura-2`, `aero`, `sonic-3`)
`sttLanguage`	string	STT language code

LLM Tool

Tool name: switch_language

Parameters:
  target_language: string (language code, e.g., "es", "hi", "fr")

Returns: "Switched to {language}. Continue the conversation in {language}."

When called, the tool:

Looks up the language in languageVoiceMap
Creates a new TTS instance with the target voice
Updates the STT language
Swaps the agent’s TTS mid-session via session.update_agent()

Supported Languages

Any language supported by your TTS/STT providers. Common combinations:

Language	TTS Provider	Example Voice
English	Deepgram Aura-2	`aura-2-thalia-en`
Spanish	Deepgram Aura-2	`aura-2-thalia-es`
French	Deepgram Aura-2	`aura-2-thalia-fr`
German	Deepgram Aura-2	`aura-2-thalia-de`
Hindi	Aero	`aero-ananya`, `aero-kavya`
Hindi	Sarvam	`anushka`, `arya`
Japanese	Deepgram Aura-2	`aura-2-thalia-ja`

Example — Create Multilingual Agent

curl -X POST https://api.thinnest.ai/v1/agents \
  -H "Authorization: Bearer $THINNESTAI_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "Multilingual Support",
    "model": "gpt-4o",
    "instructions": "You are a multilingual support agent. Your primary language is English. When the caller speaks in a different language, detect it and switch your voice using the switch_language tool.",
    "voiceEnabled": true,
    "transcriber": { "provider": "deepgram", "model": "nova-2-conversationalai" },
    "voice": { "provider": "deepgram", "voiceId": "aura-2-thalia-en" },
    "multilingualEnabled": true,
    "multilingualPrimaryLanguage": "en",
    "languageVoiceMap": {
      "es": { "ttsVoice": "aura-2-thalia-es", "ttsModel": "aura-2", "sttLanguage": "es" },
      "hi": { "ttsVoice": "aero-ananya", "ttsModel": "aero", "sttLanguage": "hi" },
      "fr": { "ttsVoice": "aura-2-thalia-fr", "ttsModel": "aura-2", "sttLanguage": "fr" }
    }
  }'

Introduction

Getting Started

Voice Agents

Agent Capabilities

Channels

Quality & Oversight

Platform

Multilingual Auto-Switch

Multilingual Auto-Switch

Automatic language following

How It Works

Configuration

Language Voice Map Entry

LLM Tool

Supported Languages

Example — Create Multilingual Agent

​Multilingual Auto-Switch

​Automatic language following

​How It Works

​Configuration

​Language Voice Map Entry

​LLM Tool

​Supported Languages

​Example — Create Multilingual Agent

Multilingual Auto-Switch

Automatic language following

How It Works

Configuration

Language Voice Map Entry

LLM Tool

Supported Languages

Example — Create Multilingual Agent