> ## Documentation Index
> Fetch the complete documentation index at: https://brosbots.mintlify.site/llms.txt
> Use this file to discover all available pages before exploring further.

# Turn detection and STT timing

> How the platform decides that the caller has finished speaking: VAD, Deepgram parameters (endpointing/utterance_end_ms) and the Smart Turn ML model.

Knowing exactly **when the caller has finished speaking** is the hardest
part of a real-time voice agent. Answer too early - you talk over them.
Too late - the agent sounds sluggish. The platform combines **three
complementary layers** to get this right.

## 1. VAD - voice activity detection (Silero)

The transport layer runs every audio chunk through a local
[Silero VAD](https://github.com/snakers4/silero-vad) model (ONNX), which
answers a simple question for each frame:

> *"Is this speech, or silence?"*

Configured in the agent's `vad_configuration`:

| Parameter | Default | Meaning |
| - | - | - |
| `confidence` | `0.7` | Minimum model confidence to treat a frame as speech. |
| `start_seconds` | `0.2` | How much continuous speech means *SPEAKING*. |
| `stop_seconds` | `0.5` | How much continuous silence means *QUIET*. |
| `minimum_volume` | `0.6` | Minimum RMS volume treated as speech (cuts off noise). |

VAD is responsible for **interruption detection** (the caller speaking
while the bot speaks) and is the base signal for the turn analyzer.

<Note>
  A bare "yes" or "no" lasts about 0.2–0.4 s. With `start_seconds` at `0.4`
  (the former default, still stored on agents created earlier) VAD often misses
  such a reply and the agent reacts to it late. If your callers answer in one
  word, set `0.2`. A higher value only makes sense on noisy lines, where short
  clicks interrupt the agent.
</Note>

On the **Accurate** transcription tier (Soniox) it is VAD that closes an
utterance. When VAD misses it, the transcript is closed on its own after
0.5 s without new words, so a short reply does not wait for the caller's next
sentence.

## 2. Deepgram streaming parameters

Deepgram (STT over WebSocket) receives audio in parallel with VAD and
produces two kinds of transcripts:

* **Interim** - partial, updated many times per second.
* **Final (`is_final=true`)** - a committed fragment that won't change.

Two parameters decide when Deepgram "closes off":

### `endpointing` (ms)

The silence threshold that marks a **phrase boundary** - after what pause
Deepgram commits the current transcript (`is_final=true`). Lower values =
faster commits, but a single thought may split into several segments.

* Range: `10–1000` ms
* Default: `100` ms (fast)
* Higher values (300–500 ms) for callers who pause mid-sentence.

### `utterance_end_ms` (ms)

The silence threshold that marks the **end of the whole utterance** -
after what silence Deepgram emits the `UtteranceEnd` event, which triggers
the LLM.

* **Minimum: `1000` ms** - a *hard Deepgram requirement*. Values below it
  cause the WebSocket connection to be rejected with **HTTP 400**, and STT
  never connects. The backend automatically bumps older configurations up
  to 1000.
* Range: `1000–5000` ms
* Default: `1000` ms

## 3. Smart Turn V3 - ML end-of-turn detection

Silence timers alone aren't enough - people pause for many reasons
(thinking, breathing, reading out a number), and 1000 ms is an eternity in
conversation. **Smart Turn V3** is a local ONNX model that analyzes the
**prosody** of the utterance - intonation, rhythm, the final pitch
contour - and returns the probability that the turn is finished.

When enabled on the agent (`smart_turn_enabled: true`):

* after every VAD silence boundary the audio segment goes through the model,
* the model returns `{prediction: 0|1, probability: 0..1}`,
* `prediction == 1` → the LLM starts **immediately**, often hundreds of ms
  before the `utterance_end_ms` timeout,
* `prediction == 0` → the pipeline keeps listening; `utterance_end_ms`
  acts as a safety net.

### Does `utterance_end_ms` still matter with Smart Turn enabled?

**Yes.** It's a "belt and suspenders" setup:

| Situation | What fires first |
| - | - |
| Clear end of sentence, natural intonation | Smart Turn (\~200–400 ms after silence) |
| Ambiguous mid-sentence pause | Nothing - the pipeline keeps listening |
| End of utterance, but the model is unsure | `utterance_end_ms` fallback (1000 ms) |
| Smart Turn disabled | Only `utterance_end_ms` (1000 ms) |
| Smart Turn model file missing | Silent fallback to VAD + `utterance_end_ms` |

Lowering `utterance_end_ms` below 1000 ms would speed up the fallback
path, **but Deepgram doesn't allow it**. If you want lower latency -
enable Smart Turn.

## When the caller talks over the agent

The caller can interrupt the agent at any moment - the agent stops and
listens. So that an "mhm" or "uh-huh" dropped while the agent is talking does
not cut its sentence short, an interruption during the agent's speech counts
only from **two words** of the caller. "No, no" or "wait, wait" interrupts at
once; a single nod is ignored and the agent carries on. While the agent is
silent, every word of the caller counts as before.

The threshold is the **Interruption sensitivity** slider in the agent's
settings: at maximum any sound interrupts (an "mhm" too, as before), in the
middle two words, at minimum four. Lower sensitivity suits noisy lines where
the VAD catches fragments of words; higher, callers who interrupt with a
single word ("stop", "hello"). The API also takes an explicit
`workflow_configurations.interruption_min_words` (`0` = any sound).

A short answer under the threshold is not lost. When the caller drops a
"Yes" over the end of "shall I book you for Tuesday?", the agent does not cut
its sentence, but it keeps that "Yes", and if it finishes speaking within
**1.5 s** it treats it as the caller's ordinary reply. A nod in the middle of
a longer sentence (the agent goes on for more than 1.5 s) is still ignored.
If the caller goes on right after the short word and interrupts the agent,
both parts reach the model as one utterance. Change the window with
`workflow_configurations.short_reply_hold_seconds` (`0` turns it off,
maximum `5`); with the slider at maximum there is no threshold, so nothing
needs keeping.

## How long the caller really waits

In a call transcript, each agent reply carries a time with a stopwatch icon:
from the moment the caller went quiet to the agent's first audio. It is made
of the wait for the end of speech (`stop_seconds`), transcription, the
[LLM](/en/configurations/llm) and the start of speech synthesis. On the phone
the network adds its own delay, usually 0.2–0.4 s, which the platform does not
measure.

Calls from before this measurement show a brain icon and the model's thinking
time alone, which is only part of that pause.

## Recommended settings

* **Natural conversation (recommended):** Smart Turn ON,
  `utterance_end_ms = 1000`, `endpointing = 100`.
* **Dictating numbers / long pauses:** Smart Turn ON,
  `utterance_end_ms = 1500–2000`, `endpointing = 300`.
* **Minimum latency without Smart Turn:** `utterance_end_ms = 1000`,
  `endpointing = 100` - occasional talk-overs possible.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.