# How does the VARTA voice agent engine answer a customer turn?

How VARTA runs a live voice-agent call — a fixed cascade of rule-first layers with Hindi names, where the LLM is tried last and pre-built audio clips answer most turns.


VARTA is a voice-agent engine built rule-first. One customer turn arrives as speech-to-text (STT) output. A fixed cascade of layers tries to answer it, cheapest and most deterministic first, and pre-synthesised audio clips come out. The large language model (LLM) is the last layer tried, and even then it can only choose from options the code has already built.

The [interactive version of this page](/architecture/varta-engine) replays a sample Bank KYC Update call turn by turn: which layer answers each turn, when a turn reaches the LLM, when a reply needs live text-to-speech (TTS), and how much waiting and provider spend the rule-first design avoids compared with an agent that calls an LLM on every turn. How the workflow data this engine reads gets written is covered in [DAKSH Architecture](/architecture/daksh).

## A turn in one line

- **In:** the final STT transcript for the turn, plus call context.
- **Decide:** the layer cascade, from `LOOP_GUARD` through to `L4 BUDDHI`.
- **Commit:** the step result — the cursor moves, slot values are written, completion is recorded.
- **Out:** the response, played as pre-built audio clips wherever possible.

## Six rules the engine is built on

1. **Determinism ladder.** Cheap rule-based layers decide first. The LLM layer, L4, runs only when they cannot, and it must answer in structured JSON or pick from a list.
2. **Language-neutral engine.** The engine contains no language-specific strings or checks. Every per-language line, keyword and number word is workflow data written by DAKSH, so adding a language never changes the engine.
3. **Cache-first audio.** Pre-built clips play in about 0 ms at no marginal cost. Live TTS is only a fallback, and filler clips cover the wait whenever L4 runs.
4. **Re-ask, never silently advance.** A layer that cannot decide re-asks or passes the turn down. An LLM answer that does not validate triggers a re-ask; it never moves the call forward.
5. **Every LLM call is metered.** At call time, every reasoning call is costed onto that call's record. At build time, every generation step is costed onto the workflow it built.
6. **Silent boot.** Starting the server never triggers LLM work. Warm-up reads caches only.

## Three transports, one engine

Browser test calls from the designer, live phone calls over SIP through LiveKit workers, and a benchmark simulator in which an LLM plays the customer all run the same turn engine. They differ only in how speech becomes text before the engine sees it, so a fix to the engine reaches every transport at once.

The engine reads the workflow built by DAKSH and the audio builder. It calls an LLM provider (OpenAI or Gemini) only for L4, falls back to live TTS (ElevenLabs or Sarvam) only when no pre-built clip matches, and persists call state to a session store. The step index is rebuilt on every turn and never cached, so a call never acts on a stale copy of the flow.

## The layers, by name

Each layer is named in Hindi after what it does in a conversation.

| Layer | Name | Meaning | What it does |
|---|---|---|---|
| L1 | DHADKAN (धड़कन) | Heartbeat | Instant reflexes. Matches yes, no, wait, repeat, goodbye and handoff phrases against a compiled regex cache that includes phonetic variants. Fallback yes/no matching is whole-word only, so "hmm what" is not a yes. Also runs the confirmation gate after a readback ("haan, sahi hai"). No LLM; microseconds. |
| L2 | PEHCHAAN (पहचान) | Recognition | Cleans up the transcript (spoken numbers and letters, filler words), extracts entities such as phone numbers, dates, pincodes and codes from anywhere in the sentence, and matches the utterance to a step. A confident match with a valid value can jump ahead, but never while a confirmation or re-ask is pending. |
| L3 | MAARG (मार्ग) | Path | Chooses the branch with a pure function over the step's authored branches: keywords, digit counts and regex. Returns advance, ambiguous, no match or skip. Only an ambiguous result reaches the LLM, as a constrained pick of one option from a list. It never changes call state. |
| L4 | BUDDHI (बुद्धि) | Intellect | The LLM, used only as a last resort. It shortlists the relevant rules with embeddings, builds the prompt and requires a structured JSON reply. It can route, handle drift, extract a value, generate a line or validate. A pick below 0.6 confidence passes the turn on, and an invalid result means re-ask. A filler clip covers the wait. |
| L5 | NITI (नीति) | Policy / rules | Validates and captures the answer with per-step and global rules: regex, keywords, slot aliases and phonetic variants. The outcome is accept, reprompt, confirm the value, fall back to the LLM, defer or escalate. Also handles condition branches and the pincode and address check. No LLM. |
| L5·a | PRASANG (प्रसंग) | Context / episode | Runs first inside L5: a weighted-keyword matcher with a winning margin over side-question scenario groups that apply across the whole workflow. Takes 0–2 ms, answers from cached clips, and can combine up to 3 scenarios in one reply. Negated triggers ("not that") go to L4 instead of getting a wrong answer. |
| L6 | VANI (वाणी) | Speech / voice | Builds the reply: acknowledgement plus next prompt, captured values filled into templates, and readback with SSML. Plays the pre-built clip and uses live TTS only for text that cannot be built in advance, such as reading back a customer's own address. |
| L7 | DRISHTI (दृष्टि) | Sight / observation | Watches patterns across turns: hearing problems ("hello? hello?"), dead ends where a step makes no progress, and repeated STT failure. It re-prompts, offers a human, or ends the call politely. |
| — | VIRAM (विराम) | Pause | The empathy specialist, with about 12 scenarios such as a past complaint, a price negotiation or scam suspicion. It moves through four phases — acknowledge, resolve, steer, escalate. Its first turn plays a DAKSH-written clip; later turns reason live through L4. It exits back to the flow, to a callback, to a close or to escalation. |
| build | DAKSH (दक्ष) | Skilled / expert | The design-time LLM author. It is not a runtime layer and never runs during a customer call; every layer above reads what it wrote. |

### Supporting layers

| Layer | Name | What it does |
|---|---|---|
| guard | LOOP_GUARD (stuck-loop guard) | After 7 or more agent turns on the same step, the call ends politely. A safety rail, not a fix. |
| guard | HANDOFF (human handoff) | When a "talk to a colleague?" offer is waiting, takes the customer's yes or no and moves the handoff from offered to connecting, to Q&A, to listening. |
| guard | POLICY (conduct and pauses) | Handles abuse, please-wait requests and resuming afterwards, using DAKSH-written phrase lists. |
| L1.6 | STT GATE (confidence gate) | Blocks transcripts below the workflow's minimum STT confidence and re-asks. After 2 blocks in a row it lets the text through, but that turn can only fill an empty slot, never overwrite one that has a value. |
| L1.7 | OOS (out of scope) | Custom off-topic keywords, compiled once per workflow. Politely brings the customer back to the current step. |
| L1.5 | INTERJECT (interjection) | Answers meta-requests such as "what number did I give?" without moving the cursor, then asks the current question again. |
| tool | TOOL (API action step) | Runs the workflow's API action steps during the call. |
| match | FUZZY (sample fuzzy match) | Fuzzy string matching against authored samples, before the rule engine. |

## Layers in dispatch order

Each layer either answers the turn and returns, or passes it to the next layer. These are the labels that appear in a VARTA layer trace.

**Guards — before understanding**

1. `LOOP_GUARD` — ends the call politely after 7 or more turns stuck on one step.
2. `HANDOFF` — if a human-handoff offer is waiting, this turn is the customer's yes or no.
3. `L7 DRISHTI` — detects gibberish, "I can't hear you", and dead ends.
4. `VIRAM` — handles empathy scenarios in four phases; turn 1 is a cached clip, then L4.
5. `POLICY` — handles abuse, "please wait" and resume.

**Reflexes — fixed patterns**

6. `L1 DHADKAN` — yes, no, wait and repeat reflexes from a compiled regex cache with phonetic variants.
7. `STT GATE` — blocks low-confidence transcripts.
8. `L1 CONFIRM` — answers a pending confirmation ("did you say…").
9. `L1.7 OOS` — catches out-of-scope keywords.

**Understanding — rules and recognition**

10. `L2 PEHCHAAN` — recognises the step, jumps ahead, or captures a slot value directly.
11. `L1 INTERJECT` — handles interjections in the middle of the flow.
12. `TOOL` — runs API action steps.
13. `FUZZY` — fuzzy-matches the utterance against authored samples.
14. `L5 NITI` — rule-based validation, slot capture and condition branches, starting with the 0–2 ms PRASANG side-question matcher.

**Escalation — reasoning**

15. `L3 MAARG` — a routing shortcut that can hand the turn to the full L4.
16. `L4 BUDDHI` — the LLM step: rule shortlist with embeddings, then one structured-JSON call. Picks below 0.6 confidence are re-asked.

**Expression — always last**

17. `L6 VANI` — builds the response: picks the pre-built clip, reads values back, summarises the address, stamps the cost and attaches the trace.

## Where a turn's time goes

Most turns spend microseconds inside the engine; the time goes to what happens around it. The endpointing gap is paid on every turn, while L4 and live TTS are paid only when the rule ladder or the clip cache misses. Indicative ranges, not p95, from network traces measured in India in September 2026:

| Component | Typical time | When it is paid |
|---|---|---|
| Endpointing gap | 600 ms | Every turn |
| Deterministic layers, L1 to L5 | under 5 ms | Every turn |
| PRASANG matcher | 0–2 ms | Whenever it checks |
| L4 rule shortlist (embeddings) | 600–1100 ms | Only when L4 runs |
| L4 LLM call | 300–1500 ms | Only when L4 runs |
| Live TTS | 200–800 ms | Only when no clip matches |
| Cached clip | about 0 ms | The normal case |

Of each LLM call, about 420–570 ms is network backhaul that stays roughly constant whatever the prompt size, so connection pooling cannot remove it. Embeddings are about 60 ms of compute and roughly 520 ms of transit.

## One turn, start to finish

1. Rebuild the step index (every turn, never cached).
2. Compile the step graph if it is not compiled yet.
3. Start the layer trace.
4. Run the cascade: `LOOP_GUARD` → `HANDOFF` → `L7` → `VIRAM` → … → `L5 NITI` → `L3 MAARG` → `L4 BUDDHI`.
5. Commit the step result: the cursor moves, slot values are written, completion is recorded.
6. Build the response: check the audio, attempt one re-synthesis rescue if needed, stamp the cost, attach the trace.

A call remembers its current step, the steps completed, the values captured per step, a return stack of drift bookmarks (up to 5 deep) and the steps invalidated by a revisit, plus per-call flags for the closing grace turn, handoff, low-confidence turns and billing.

## DAKSH writes what the engine reads

The engine can stay language-neutral and cheap because the language-specific and costly work happens before the call. DAKSH writes the per-language data: runtime system lines, slot aliases and phonetic variants. The builder synthesises every clip that can be known in advance and fingerprints each one over its text, SSML, voice, model, language, provider and speech rate, so a rebuild re-synthesises only the clips whose inputs changed. At call time, the L1 and L5 matchers also match on phonetic variants, every system line comes from DAKSH-written text, and L6 resolves the pre-built clip, using live TTS only on a miss.

## What the layers call into

- **VIRAM** — the empathy specialist. Its first turn plays a pre-built clip; later turns reason live. It exits with handover, callback, close or escalate.
- **Drift** — handles REPEAT (says the captured value again, cursor stays put), REVISIT (bookmarks the step, resets the target and invalidates everything after it) and CROSS_TOPIC (answers, then returns). A keyword pre-check runs before any LLM list-pick.
- **PRASANG** — weighted-keyword side-intents answered from cached clips, keeping side-intents that would otherwise run through large L4 rule sets off the LLM. Anything it cannot match falls through to L4.
- **Human handoff** — can be triggered by L1, an L4 request for a human, an L7 dead end, or a VIRAM or NITI escalation. It dials a human into the LiveKit room and hands over an LLM-written brief; the human's follow-up Q&A skips L1–L7 entirely.
- **Navigator** — the step graph is the source of truth for flow: explicit next steps and branches plus completion tracking. Every goto target is checked against the current set of steps before it is used.
- **FAQ, address and tone** — authored FAQ answers; address verification with a 3-second wall-clock budget, where a timeout counts as an inconclusive soft pass; and tone detection that shapes delivery.
- **LLM providers** — OpenAI and Gemini chat behind one factory, with the provider and model chosen per workflow. Embeddings and moderation use OpenAI. Every client reuses pooled connections.
- **Observability** — per-layer timings and a cumulative clock on every turn, network timing per request (DNS, TLS, time to first byte, server versus transit), a per-call SIP timeline, a searchable call index and spreadsheet reports.

## Questions this page answers

**Does VARTA call an LLM on every turn?**
No. Rule-based layers (L1, L2, L3, L5 and PRASANG) answer first. L4, the LLM layer, runs only when none of them can decide, and even then it answers in structured JSON or picks from a list the code built.

**Why are VARTA's layers named in Hindi?**
Each layer is named after what it does in a conversation: DHADKAN (heartbeat) for instant reflexes, PEHCHAAN (recognition), MAARG (path), BUDDHI (intellect), NITI (policy), VANI (speech) and DRISHTI (sight).

**How does VARTA add a new language?**
By running DAKSH again. Every line the engine speaks, recognises or parses is workflow data DAKSH writes, so the engine itself does not change.

**What happens when the LLM returns an invalid answer?**
The turn is re-asked. An answer that fails validation, or a pick below 0.6 confidence, never advances the call.

**How fast is a cached reply?**
About 0 ms. Pre-built clips are synthesised at build time, so live TTS is only needed for text that cannot be known in advance, such as reading back a customer's own address.