Skip to main content

​
Multilingual Auto-Switch

The Multilingual Auto-Switch tool detects when a caller speaks a different language and automatically switches the agent’s TTS voice and STT model mid-call. The agent continues the conversation seamlessly in the new language.

​
Automatic language following

When your agent’s speech-to-text language is set to auto-detect (for example Sarvam Saaras, or any transcriber set to “multi”), the agent follows the caller’s language on its own — no tool, no configuration:
  • It detects the language of each caller message and replies in that same language.
  • It handles mid-call switches turn by turn — a caller can open in English, move to Hindi, and switch back, and the agent keeps up every time.
  • It mirrors code-mixing. A caller speaking Hinglish (Hindi and English blended in one sentence) — or Tanglish, Tenglish, and similar mixes — gets a reply in the same mixed style, rather than being pushed into one pure language.
  • Hindi and Marathi are told apart correctly even though they share the Devanagari script, so the agent answers in the language the caller actually spoke.
This runs on the agent’s existing voice. Use the Multilingual Auto-Switch tool below only when you also want a different voice per language — for example, a dedicated Spanish voice for Spanish callers.

​
How It Works

Caller (English): "Hi, I need help with my account."
Agent (English): "Of course! What's your account number?"
Caller (Spanish): "Perdona, puedo hablar en espanol?"
-> Agent detects language switch
-> Calls switch_language tool with target "es"
-> TTS switches to Spanish voice, STT to Spanish model
Agent (Spanish): "Claro que si! Cual es su numero de cuenta?"

​
Configuration

{
  "multilingualEnabled": true,
  "multilingualPrimaryLanguage": "en",
  "languageVoiceMap": {
    "es": {
      "ttsVoice": "aura-2-thalia-es",
      "ttsModel": "aura-2",
      "sttLanguage": "es"
    },
    "hi": {
      "ttsVoice": "aero-ananya",
      "ttsModel": "aero",
      "sttLanguage": "hi"
    },
    "fr": {
      "ttsVoice": "aura-2-thalia-fr",
      "ttsModel": "aura-2",
      "sttLanguage": "fr"
    }
  }
}
SettingTypeDefaultDescription
multilingualEnabledbooleanfalseEnable multilingual auto-switch
multilingualPrimaryLanguagestringenPrimary language code
languageVoiceMapobject{}Map of language code to TTS/STT config

​
Language Voice Map Entry

Each entry in languageVoiceMap configures the voice for that language:
FieldTypeDescription
ttsVoicestringVoice ID to use for this language
ttsModelstringTTS model to use (e.g., aura-2, aero, sonic-3)
sttLanguagestringSTT language code

​
LLM Tool

Tool name: switch_language 
Parameters:
  target_language: string (language code, e.g., "es", "hi", "fr")

Returns: "Switched to {language}. Continue the conversation in {language}."
When called, the tool:
  1. Looks up the language in languageVoiceMap
  2. Creates a new TTS instance with the target voice
  3. Updates the STT language
  4. Swaps the agent’s TTS mid-session via session.update_agent()

​
Supported Languages

Any language supported by your TTS/STT providers. Common combinations:
LanguageTTS ProviderExample Voice
EnglishDeepgram Aura-2aura-2-thalia-en
SpanishDeepgram Aura-2aura-2-thalia-es
FrenchDeepgram Aura-2aura-2-thalia-fr
GermanDeepgram Aura-2aura-2-thalia-de
HindiAeroaero-ananya, aero-kavya
HindiSarvamanushka, arya
JapaneseDeepgram Aura-2aura-2-thalia-ja

​
Example — Create Multilingual Agent

curl -X POST https://api.thinnest.ai/v1/agents \
  -H "Authorization: Bearer $THINNESTAI_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "Multilingual Support",
    "model": "gpt-4o",
    "instructions": "You are a multilingual support agent. Your primary language is English. When the caller speaks in a different language, detect it and switch your voice using the switch_language tool.",
    "voiceEnabled": true,
    "transcriber": { "provider": "deepgram", "model": "nova-2-conversationalai" },
    "voice": { "provider": "deepgram", "voiceId": "aura-2-thalia-en" },
    "multilingualEnabled": true,
    "multilingualPrimaryLanguage": "en",
    "languageVoiceMap": {
      "es": { "ttsVoice": "aura-2-thalia-es", "ttsModel": "aura-2", "sttLanguage": "es" },
      "hi": { "ttsVoice": "aero-ananya", "ttsModel": "aero", "sttLanguage": "hi" },
      "fr": { "ttsVoice": "aura-2-thalia-fr", "ttsModel": "aura-2", "sttLanguage": "fr" }
    }
  }'