Skip to content
Requires: Developer and above Last verified: 2026-09-21

Agents That Speak Several Languages

Many callers prefer to speak in their own language, and some switch language during a call. This page explains how to build an agent for that, and which pipeline to choose.

Where to set it: the Default Language on the General Details tab, and the pipeline on the AI Models tab.

The short answer

You want Choose
An agent that starts in one language and moves to another when the caller does, or that serves callers of several languages on the same number Speech-to-Speech
An agent that always speaks one language, especially an Arabic dialect, an Indian language with a specific voice, or a provider of your choice Cascaded

Speech-to-Speech is the better choice for a multilingual agent. A single model both listens and speaks, so it can follow the caller into another language. Cascaded agents can also change language during a call, but they have limits, explained below.

Default Language

The Default Language is the language the agent starts in. It applies to both pipelines.

Language Code
English (US), English (India) en, en-in
Hindi, Tamil, Telugu, Kannada hi, ta, te, kn
Arabic: Egyptian, Saudi, UAE / Gulf, Iraqi, Levantine, Moroccan, Algerian, Sudanese ar-eg, ar-sa, ar-ae, ar-iq, ar-levant, ar-ma, ar-dz, ar-sd
Uzbek uz

Choosing a language does two things:

  • It tells the agent to reply in that language (and, for Arabic, in that dialect).
  • On Cascaded agents, it sets the language the speech recogniser listens for.

It does not choose a voice. The voice is a separate choice on the AI Models tab.

Speech-to-Speech: several languages in one call

  1. On AI Models, choose Speech-to-Speech Model and a Gemini Live model.
  2. On General Details, set the Default Language to the language callers will most often use.
  3. Write the Agent Greeting in that language.
  4. In the Agent Definition, tell the agent what to do when the caller uses another language. For example:
You are Ava, an appointment scheduler for Vyom Demo Company.
Start in English. If the caller speaks Hindi or asks to speak Hindi,
continue the whole conversation in Hindi. If they switch back, switch back too.
Always answer in the language the caller is using.
Keep every answer short. Do not use lists.
  1. Save and test in Agent Preview in each language you expect.

How it works during a call:

  • The model hears the caller and replies in the same language. Your instructions steer it.
  • The agent also has a built-in language switch tool. When the caller asks to change language, it moves to that language and adjusts its instructions. This happens without dropping or restarting the call, so there is no gap.
  • The voice you chose speaks every language. The Gemini Live voices are built to speak many languages.

Tell the agent which languages it may use

Name the languages in the Agent Definition, as in the example. An agent with clear instructions switches more reliably than one left to guess.

What to watch for with Speech-to-Speech

  • Arabic dialects. Gemini Live treats all Arabic as one language. The agent can be told to answer in Egyptian or Saudi wording, but the listening side does not tell dialects apart. If dialect accuracy matters, use a Cascaded agent with Munsit. See below.
  • Fewer provider choices. You use Gemini Live models and their voices.
  • Test each language. A language that is supported still needs a real test call with a real accent before you go live.

Cascaded: one language, chosen carefully

A Cascaded agent is built from three parts, and each part must handle your language.

Part What must be true
STT (listening) The provider must recognise the language. The Default Language is passed on to it.
LLM (thinking) Writes the reply in the language named in your instructions. Large models handle most languages.
TTS (voice) The voice you choose must be able to speak the language. Vyom uses that one voice for every reply.

Points that catch people out:

  • Telugu and Kannada need Deepgram Nova 3. Nova 2 does not support them. Choose Deepgram (Nova 3) as the STT model.
  • Uzbek needs a speech recogniser that supports it. Deepgram does not. Test carefully before you rely on it, or use Speech-to-Speech.

  • Voices by language.

    Language Voices in Vyom today
    Saudi and UAE / Gulf Arabic Munsit has native voices
    Hindi and Telugu Eterna Edge has voices for both
    Other Arabic dialects, Indian English, Kannada No matching native voice yet. The agent speaks through the voice you select, which will not sound native
    Hindi, Tamil, Telugu, Kannada, Indian English Vakyam AI: every voice speaks all five. The voice sample plays in your agent's Default Language
    Every agent language, including Uzbek ElevenLabs Eleven v4 and v4 Turbo, and Gemini 3.8 Flash TTS and Flash-Lite TTS, where every voice speaks all of them. Replies start a little later than with other voices
    Any language Cartesia, ElevenLabs and Sarvam offer voices too. Listen to a sample in Preview before you decide

  • Arabic dialects. For genuine dialect recognition, choose Munsit for STT. Munsit states that one model covers all eight dialects. Pick a dialect as the Default Language so the agent replies in the right wording.

  • Sarvam listens for Indian languages only. Use it for Hindi, Tamil, Telugu, Kannada and Indian English.

Switching language during a Cascaded call

A Cascaded agent can change language during a call, but it is more limited than Speech-to-Speech.

  • The agent switches when the caller asks to ("speak in Hindi"), or when the agent decides to use the language switch tool. Vyom also listens directly for a few phrases: Hindi, Tamil and English.
  • When it switches, Vyom rebuilds the listening and speaking parts for the new language. The agent says a short confirmation in the new language, for example "जी बिल्कुल, अब हम हिंदी में बात करेंगे।" You may notice a short pause.
  • The voice does not change. The one voice you chose must be able to speak the new language.
  • Only the languages in the table above can be switched to.

Use this for simple cases, such as an English agent that can move to Hindi on request. If callers regularly move between languages, choose Speech-to-Speech.

Building an agent for a language, step by step

  1. Decide whether the agent is one language or several. Several means Speech-to-Speech.
  2. Choose the pipeline on AI Models, and the models and voice.
  3. Set the Default Language and write the Agent Greeting in it.
  4. Write the Agent Definition. You can write it in English, and tell the agent which language to speak. For the most natural result, add a few example phrases in the target language.
  5. Test in Agent Preview, speaking each language the way your callers do.
  6. Fix what you hear, then set the agent to Active.

If something goes wrong

What you see What it means and what to do
The agent does not understand the caller (Cascaded) The STT provider or model may not support the language. Check the points above, for example Nova 3 for Telugu and Kannada.
The agent understands but sounds wrong The voice does not speak that language well. Choose another voice, or another TTS provider.
The agent answers in English when the caller speaks another language Add a line to the Agent Definition telling it to reply in the caller's language, and name the languages.
The agent does not switch when asked Name the language in the Agent Definition. On a Cascaded agent, check the language is in the list above.
Arabic sounds like the wrong dialect Set the right Arabic Default Language. For the best accuracy use a Cascaded agent with Munsit.
The greeting looks wrong for Arabic Write it right to left as normal. The box handles the direction for you.
Need help or clarification? Email support@eternascisense.com, or call the support number shown on vyom.eternascisense.com.