Hindi Isn't Enough: Building Voice AI for Real Hinglish Conversations

Building a Hindi Voice AI agent sounds straightforward.

Choose Hindi STT.

Write Hindi prompts.

Select a Hindi voice.

Start calling.

Then a real customer says:

“Mera service request already raise hua tha, technician abhi tak assign nahi hua.”

Another says:

“AC cooling nahi kar raha, but compressor ka sound aa raha hai.”

And another:

“Kal afternoon mein callback kar dena, abhi meeting mein hoon.”

These are not clean Hindi conversations.

They are not English conversations either.

They are Hinglish—and for many Indian customer interactions, that is the real production language.

Language Selection Is Not Enough

A traditional multilingual system often begins by choosing a language:

Hindi
or
English

Real conversations behave differently.

Hindi → English → Hindi
↓
Product terms
↓
English numbers / dates
↓
Back to Hindi

The switch can happen inside a single sentence.

Consider:

“Technician kal visit karega ya appointment reschedule karna padega?”

Words such as technician, visit, appointment, and reschedule naturally remain English while the grammatical structure is Hindi.

Trying to force this conversation entirely into Hindi can make the agent sound unnatural.

Trying to force it into English can reduce understanding.

The architecture needs to support the mixture.

STT Must Preserve Meaning, Not Just Language

The first challenge is Speech-to-Text.

A customer might say:

“Mera complaint number 48321 hai.”

The important output is not whether the sentence was classified as Hindi or English.

The important information is:

intent: existing_complaint
complaint_number: 48321

Production Voice AI must preserve critical entities while handling code-switching:

• names
• mobile numbers
• product models
• ticket numbers
• amounts
• dates
• addresses
• brand names

A multilingual transcript that sounds linguistically correct but corrupts the ticket number is still a failed interaction.

Do Not Translate Everything

One tempting architecture is:

Hinglish Speech
↓
Translate to English
↓
LLM Reasoning
↓
Translate to Hindi
↓
TTS

That adds unnecessary transformation.

Every translation step can alter:

• customer intent
• terminology
• product names
• tone
• contextual meaning

For structured enterprise workflows, the better objective is often:

Understand the customer's meaning directly and convert it into structured intent.

For example:

“Engineer Friday ko aa sakta hai but morning mein chahiye.”

can become:

intent: schedule_visit
date: Friday
preferred_period: morning

The execution layer does not need a perfectly translated English sentence.

It needs the correct meaning.

The Agent Should Not Suddenly Become a Translator

Another common failure is unnatural language purity.

Imagine the customer says:

“Warranty status check kar do.”

A technically correct Hindi response might translate every term.

But that may not be how the customer naturally speaks.

A better response could be:

“Sure. Main aapka warranty status check karti hoon.”

The objective is not to demonstrate perfect Hindi.

It is to maintain the conversational register the customer is already using.

A good Hinglish agent should be able to adapt, not just translate.

Business Terms Need Special Treatment

Enterprise conversations contain vocabulary that often remains English even inside regional-language speech.

Examples include:

service request

ticket

warranty

AMC

technician

appointment

payment

OTP

serial number

invoice

callback

Forcing these into translated equivalents can make the conversation harder rather than easier.

The Voice AI layer therefore needs awareness of:

Natural language
+
Business vocabulary
+
Product terminology
+
Customer context

Language handling cannot be isolated from the workflow.

Code-Switching Can Happen Mid-Turn

Consider:

“Haan Friday fine hai, but technician 2 PM se pehle bhejna.”

This contains confirmation, date acceptance and a time constraint.

A useful runtime interpretation is:

appointment_date: Friday
date_confirmed: true
time_constraint: before 14:00

The customer should not have to repeat the sentence in one language.

The agent should understand the business meaning across languages.

The Response Should Follow the Customer

Language adaptation should work in both directions.

If the customer is speaking mostly English:

“I already raised the complaint yesterday, but technician assign nahi hua.”

the agent does not necessarily need to switch into formal Hindi.

Similarly, if the customer moves toward Hindi during the conversation, the agent should be able to follow.

The goal is not:

Detect language once at the beginning of the call.

It is:

Continuously understand how the customer is communicating.

Hinglish Is Also a TTS Problem

Understanding mixed-language speech is only half the challenge.

The response must also sound natural.

Consider:

“Aapka service request SR-4821 successfully register ho gaya hai.”

The TTS system has to correctly handle:

• Hindi pronunciation
• English words
• numbers
• abbreviations
• product names
• sentence rhythm

An STT system may understand Hinglish correctly while the TTS layer still makes the conversation sound artificial.

So multilingual Voice AI should be evaluated end to end, not provider by provider.

Workflow State Helps More Than Bigger Prompts

Context can dramatically improve mixed-language understanding.

Suppose the agent has just asked:

“Aapko technician kis din chahiye?”

and the customer replies:

“Friday chalega.”

The workflow already knows that it is expecting a date.

It does not need a large model to reconstruct the entire conversation.

current_step: select_date
expected_value: date

Customer:
"Friday chalega"

↓

selected_date: Friday

Structured workflow state makes multilingual conversations easier to interpret and more predictable to execute.

Test the Language Customers Actually Speak

A Hindi demo script is not enough.

Testing should include utterances such as:

“Complaint close dikha raha hai but issue resolve nahi hua.”

“Kal call kar lena, abhi office mein busy hoon.”

“Warranty khatam ho gayi kya?”

“Technician ka number share kar sakte ho?”

“Payment ho gaya tha but system mein pending show kar raha hai.”

And then add the real complications:

• background noise
• different accents
• interruptions
• numbers
• names
• addresses
• English product terminology
• mid-sentence switching

That is much closer to production than asking the agent to conduct a perfectly scripted Hindi conversation.

Multilingual Voice AI Is a Runtime Problem

There may not be one STT, LLM or TTS provider that is optimal for every language and every use case.

A production architecture should therefore allow language intelligence to sit above individual providers.

Conceptually:

Customer Speech
↓
Language + Context
↓
STT / Understanding
↓
Structured Workflow
↓
Reasoning when needed
↓
Response Language
↓
TTS

This makes it possible to evolve individual components without redesigning the entire Voice AI application.

How VARTA Thinks About Hinglish

At VARTA, multilingual conversation should not be treated as a translation feature.

It is part of the execution architecture.

The runtime needs to understand:

• what the customer said
• what business information was provided
• what workflow state they are in
• whether they changed language
• which terms should remain untranslated
• what the next action should be
• how the response should naturally be spoken

The goal is not to build an agent that speaks perfect Hindi.

It is to build an agent that can handle the way customers actually speak.

Because in India, multilingual Voice AI does not mean Hindi + English.

It means Hindi, English—and everything customers naturally do between them.

VARTA Engineering

The execution layer for production-grade Voice AI.

VARTA is designed to work across multiple LLM, STT and TTS ecosystems, including providers such as Sarvam, OpenAI, ElevenLabs and other AI services, enabling different language and voice technologies to be used within the same enterprise Voice AI architecture.