Voice & chat

Beta

Last updated Sep 23, 2026

Usher is voice-first, and chat is always one tap away. Both run on a single live session, so a user can start speaking, switch to typing, and switch back without the assistant losing track of the conversation.

One conversation

The orb talks. Aa opens the chat panel. Typed messages go to the same live session as speech, and the reply streams back as chat bubbles.
Opening chat pauses the microphone but keeps the session. Nothing is lost, and there’s no second model or second endpoint.
The panel header has two controls. – (Minimize chat) hides the panel and keeps the conversation going. End (End conversation) finishes it, the same as × on the orb.
The session survives route changes and React remounts. When the user moves to a new page, Usher tells the model where they are now, so it won’t offer the page they’re already on.
Voice and chat turn on with voqalKey alone: the browser microphone and speaker are wired for you. Without a key, the panel runs the local text-only mode, and tapping the orb opens and closes the panel.

Hands-free or push-to-talk

"auto" (default): the server detects speech and starts a turn when the user talks. Talking over the assistant interrupts it. It’s the simplest mode, but in a noisy room, or through laptop speakers without headphones, the assistant can pick up other voices.
"push-to-talk": the user presses and holds the orb to talk and lets go to send. Only held speech reaches the model, so nothing else can interrupt or trigger it. The orb reads “Hold to talk” at rest. Releasing the pointer, or dragging it off the orb, ends the turn.

src/usher.tsx

import { Usher, type UsherVoiceConfig } from "@voqal/usher-react";const voice: UsherVoiceConfig = {  interaction: "push-to-talk",  greeting: "", // don't speak first};<Usher voqalKey="pk_live_…" router={adapter} destinations={DESTINATIONS} voice={voice} />

What every control does

Move	What happens
Tap the orb (at rest)	Connects and starts listening. The assistant greets the user once per session.
Tap while connecting	Ignored. One tap makes one session.
Tap while listening	Pauses the microphone. Speech in progress is sent as finished, so the reply isn’t held up. The session stays open and the orb reads “Paused”.
Tap while paused	Turns the microphone back on in the same session, with the same context. Closes the chat panel if it's open.
Aa	Opens the chat panel and pauses the microphone (“Voice paused”). With no session yet, it starts a live session in text mode right away, which counts toward your key’s per-minute session limit and model usage.
– in the panel header (“Minimize chat”)	Closes the panel. The session keeps running, and the microphone goes back to what it was doing before Aa. A session that started in chat stays in text mode.
End in the panel header (“End conversation”)	Ends the session, exactly like × on the orb: closes the panel, clears the transcript, and stops voice. In local text-only mode it closes the panel and clears the transcript.
× on the orb	Ends the session, or cancels one that is still connecting, and clears the transcript. Shown only when there's something to end.
Connection drops	Usher reconnects on its own. With a hosted key it survives network drops and the server’s roughly 10-minute session cutoff by reconnecting with a fresh credential. The reconnected model starts without the earlier conversation, so it may not remember what was said before the drop; the transcript on screen stays.
Session can’t reconnect	The orb returns to rest and voice.onError gets UsherSessionLostError. The transcript stays visible until the next session starts.

The panel’s status line

Status	Meaning
Connecting…	The live session is starting.
Listening…	The session is live and the microphone is on.
Voice paused	The session is live and the microphone is off (chat is open, or the orb was tapped).
Not connected — send a message to reconnect	No live session: it failed to start or it dropped. Typing a message starts a new one.
Text mode	Local text-only mode: no voqalKey.

A message typed while the socket is down isn’t sent. It goes to voice.onError as LiveSessionNotConnectedError so you can tell the user to try again.

Orb states

Phase	Label under the orb	Meaning
`idle`	No label (“Hold to talk” in push-to-talk mode, “Paused” when a session is open with the mic off)	At rest, or between turns with the mic off.
`listening`	Listening	The mic is live. The glow ring follows the user's voice level.
`thinking`	Working	A turn is being processed. In chat, a “Thinking…” shimmer shows until the reply begins.
`speaking`	Speaking	The assistant is talking. The glow ring follows its voice level.

How Usher decides

Decision	Live session (voice or chat)	Local text mode
Navigate	Says one short line (“Sure, taking you to your cases”) and your router moves.	“Took you to My cases.”
Offer	Answers, then offers the page. A spoken “yes”, or the “Yes, take me to {page}” button under the reply, navigates.	The answer plus a “Yes, take me to {page}” button.
Answer	Replies in one or two sentences. Doesn't move.	The answer, with citations when the knowledge base was used.
Clarify	Asks one short question.	“Did you mean Billing or Billing settings?”
Refuse	Says plainly it can’t take them there, without saying whether the page is restricted or doesn’t exist, and suggests what it can do. Nothing moves.	A fixed line such as “You don't have access to that page.”

In a live session, the chat panel shows the assistant’s own words, streamed as it speaks. When it offers a page, a “Yes, take me to {page}” button appears under those words. Tapping it navigates through the same validation chain, and the panel then adds the fixed line “Took you to {page}.” The other fixed lines in the last column appear only in local text mode.

In a live session

The model decides between navigate, offer, clarify, declining a request it has no page for, and a plain reply. For a question your knowledge base answers, it gives the fact first and may offer the page after, never instead. It is told to navigate only on a clear request, to offer on “how do I” or “where is” questions, to ask when unsure, never to move the user to the page they’re on, and to always say something out loud. Whatever it picks, the validation chain runs before your router does.

In local text mode

A deterministic policy weighs each suggestion using the phrasing and the match score:

Answer when the user is already there, asks what the assistant can do, or is just saying hello or thanks.
Clarify when the top two pages both score 0.3 or more and are within 0.12 of each other.
Answer when the best score is under 0.3.
Navigate on act-now phrasing (“take me to”, “open”, “I need to change…”) with a score of 0.5 or more.
Otherwise offer. The policy can make a decision gentler, never bolder.

Voice options

The voice prop is optional. Pass one to set the interaction mode, the greeting, or error handling. It needs a voqalKey: a voice prop without a key throws.

Option	Type	Description
`interaction`	`"auto" \| "push-to-talk"`	Default "auto" (hands-free).
`greeting`	`string`	A hidden cue sent when a session connects, so the assistant speaks first. "" keeps it silent.
`onError`	`(error: unknown) => void`	Start failures (key, connection, microphone), UsherSessionLostError, and LiveSessionNotConnectedError. A cancelled connect is not reported. See Events & errors.
`onDiagnostic`	`(event: string, detail?: unknown) => void`	Lifecycle and level events for debugging.

Microphone and audio

The browser asks for microphone permission on the first tap. The page must be served over HTTPS (localhost is fine for development).
Audio is captured at 16 kHz mono in 100 ms blocks, with echo cancellation, noise suppression, and auto gain on. Replies play at 24 kHz.
Capture uses an AudioWorklet loaded from a blob: URL. A strict Content Security Policy must allow it; see Security.
The session language defaults to en-US. The model may answer in another language when the user speaks one, but the beta doesn’t configure or guarantee that.
isLiveAudioSupported() tells you whether the browser can capture audio at all. It returns false during server rendering.