Voice Tools
Multilingual Auto-Switch
Detect when a caller switches language and automatically change TTS voice and STT model mid-call.
Multilingual Auto-Switch
The Multilingual Auto-Switch tool detects when a caller speaks a different language and automatically switches the agent's TTS voice and STT model mid-call. The agent continues the conversation seamlessly in the new language.
How It Works
Caller (English): "Hi, I need help with my account."
Agent (English): "Of course! What's your account number?"
Caller (Spanish): "Perdona, puedo hablar en espanol?"
-> Agent detects language switch
-> Calls switch_language tool with target "es"
-> TTS switches to Spanish voice, STT to Spanish model
Agent (Spanish): "Claro que si! Cual es su numero de cuenta?"Configuration
{
"multilingualEnabled": true,
"multilingualPrimaryLanguage": "en",
"languageVoiceMap": {
"es": {
"ttsVoice": "aura-2-thalia-es",
"ttsModel": "aura-2",
"sttLanguage": "es"
},
"hi": {
"ttsVoice": "aero-ananya",
"ttsModel": "aero",
"sttLanguage": "hi"
},
"fr": {
"ttsVoice": "aura-2-thalia-fr",
"ttsModel": "aura-2",
"sttLanguage": "fr"
}
}
}| Setting | Type | Default | Description |
|---|---|---|---|
multilingualEnabled | boolean | false | Enable multilingual auto-switch |
multilingualPrimaryLanguage | string | en | Primary language code |
languageVoiceMap | object | {} | Map of language code to TTS/STT config |
Language Voice Map Entry
Each entry in languageVoiceMap configures the voice for that language:
| Field | Type | Description |
|---|---|---|
ttsVoice | string | Voice ID to use for this language |
ttsModel | string | TTS model to use (e.g., aura-2, aero, sonic-3) |
sttLanguage | string | STT language code |
LLM Tool
Tool name: switch_language
Parameters:
target_language: string (language code, e.g., "es", "hi", "fr")
Returns: "Switched to {language}. Continue the conversation in {language}."When called, the tool:
- Looks up the language in
languageVoiceMap - Creates a new TTS instance with the target voice
- Updates the STT language
- Swaps the agent's TTS mid-session via
session.update_agent()
Supported Languages
Any language supported by your TTS/STT providers. Common combinations:
| Language | TTS Provider | Example Voice |
|---|---|---|
| English | Deepgram Aura-2 | aura-2-thalia-en |
| Spanish | Deepgram Aura-2 | aura-2-thalia-es |
| French | Deepgram Aura-2 | aura-2-thalia-fr |
| German | Deepgram Aura-2 | aura-2-thalia-de |
| Hindi | Aero | aero-ananya, aero-kavya |
| Hindi | Sarvam | anushka, arya |
| Japanese | Deepgram Aura-2 | aura-2-thalia-ja |
Example — Create Multilingual Agent
curl -X POST https://api.thinnest.ai/v1/agents \
-H "Authorization: Bearer $THINNESTAI_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"name": "Multilingual Support",
"model": "gpt-4o",
"instructions": "You are a multilingual support agent. Your primary language is English. When the caller speaks in a different language, detect it and switch your voice using the switch_language tool.",
"voiceEnabled": true,
"transcriber": { "provider": "deepgram", "model": "nova-2-conversationalai" },
"voice": { "provider": "deepgram", "voiceId": "aura-2-thalia-en" },
"multilingualEnabled": true,
"multilingualPrimaryLanguage": "en",
"languageVoiceMap": {
"es": { "ttsVoice": "aura-2-thalia-es", "ttsModel": "aura-2", "sttLanguage": "es" },
"hi": { "ttsVoice": "aero-ananya", "ttsModel": "aero", "sttLanguage": "hi" },
"fr": { "ttsVoice": "aura-2-thalia-fr", "ttsModel": "aura-2", "sttLanguage": "fr" }
}
}'