Every voice assistant answers one architectural question first: does one model listen and talk, or do three models hand the turn to each other? The answer decides your latency floor, your cost per minute, how much control you have over the words, and whether the assistant can read a PDF the user just attached.
Assistant Core ships both, under the names Standard and Realtime, and users can switch between them mid-conversation. This article is the comparison we wish we had when we started: what each mode is, what we measured, and a decision table.
The two architectures
Standard is a staged pipeline. A voice activity detector (VAD) decides when the user stopped talking. The audio goes to a speech-to-text model (ASR). The transcript goes to the chat model, which has the assistant's prompt, memory, knowledge base, and tools. The reply is streamed to a text-to-speech model (TTS) and played sentence by sentence. Each stage is a separate model that you choose independently: in production we run Soniox, xAI, ElevenLabs, and OpenAI for ASR; any chat model for the LLM; and xAI, Soniox, or ElevenLabs for TTS.
Realtime is one full-duplex speech-to-speech model. Audio streams both ways over one session; the model hears, reasons, and speaks without ever producing an intermediate transcript you control. It can be interrupted naturally, and it calls the same tools the assistant has. We run Gemini Live, OpenAI Realtime, and xAI Grok Voice.
Neither is "the modern one". They optimize for different things.
Latency: what we measured
The number that matters to a user is the silence between the end of their sentence and the first sound of the answer. From our benchmarks on Vietnamese and English clips:
| Step | Standard | Realtime |
|---|---|---|
| ASR finalizes after end of speech | ~0.27 s (Soniox) | n/a |
| End of speech → first audio | ~1.8 s | ~0.7 s |
Realtime wins by more than a second, and that second is the difference between "feels like a call" and "feels like a very fast IVR". The Standard pipeline spends its time in three network round trips and in the LLM's time to first token; a fast chat model and sentence-level TTS streaming keep it under two seconds, but it cannot get much lower without collapsing the stages.
If your use case is short back-and-forth questions on a device with no screen, that gap is the whole product. If the user is asking a support assistant a policy question, 1.8 seconds is fine, and the next three sections start to matter more.
Accuracy and control
A pipeline gives you a transcript, a text reply, and a choice of models at every stage. That means:
- The chat model follows your prompt and documents. The same model and the same retrieval that answer text chats answer voice turns, so accuracy on your own content is identical to chat.
- You pick the ASR for your language. Vietnamese speech recognition quality varies enormously between providers; being able to route Vietnamese to Soniox and keep everything else unchanged is why we built the pipeline around a provider registry.
- You pick the voice. Any TTS voice, speed, and spoken language, per assistant or per user.
- Files work. A user can attach an image or a PDF and ask about it by voice. Realtime sessions cannot take attachments; they wait for the next Standard turn.
A speech-to-speech model trades that control for naturalness. It uses its own voices, its own understanding of the audio, and its own judgment about when to speak. It is better at tone, prosody, and interruptions, and harder to steer closely to a document. For a greeting-and-FAQ assistant that must not improvise on prices, that is a real cost.
Cost per minute
The billing shapes are different, and so is the range:
- Standard pays per minute of audio for ASR, per 1,000 characters for TTS, and per token for the chat model. On an efficient chat model this adds up to a few credits per minute at today's rates.
- Realtime pays per session minute, and the range across models is wide: roughly 45 credits per minute on the cheapest realtime model to around 700 on the most expensive, at the time of writing. The current per-model rates are in the credits documentation.
For a personal assistant used a few minutes a day, Realtime is affordable and feels better. For a support line that runs for hours, Standard is the mode that keeps the bill flat, and the model choice at each stage is where you tune it. A realtime session also needs enough balance to cover at least one minute before it starts, and ends when the balance is used up, so budgets need headroom.
Decision table
| Choose Standard when | Choose Realtime when |
|---|---|
| Answers must follow your documents and instructions closely | The conversation should feel like a phone call |
| Users send images or documents | Users ask short, back-and-forth questions |
| You need a specific chat model, ASR, or TTS voice | Low latency matters more than cost |
| You want the spoken greeting | The device has no screen and voice is the whole interface |
| Cost per minute must stay low at volume | Interruptions and natural turn-taking matter |
Standard is the default on new assistants. If no realtime model is active, the app stays in Standard and says so.
One runtime, both modes
The practical reason to have both in one runtime is that the choice is rarely permanent. A product can start on Standard for accuracy and cost, offer Realtime to users who want it, and change the default per assistant later without touching the integration. The browser app, ESP32 devices, and the Developer API's voice WebSocket all speak to the same runtime, and Standard sessions over the API can also request emotion events and override the session language.
If you want to hear the difference rather than read about it, the homepage has a live call button that runs on this runtime. For the device side of the story, read how we moved ESP32 audio to MQTT + UDP and why acoustic echo cancellation decides whether barge-in works at all.