Guides

Voice or chat: picking the mode for the task

Speaking and typing are two inputs to the same agent. The situation the user is in decides which one wins, and four of those situations are predictable.

VVoqal · Engineering9 min readPart of Agent basics

About this article

Teams argue about this as if it were a product decision, and it is a situation decision. The same user wants to speak on a bike and type on a train, and they are the same person with the same task and the same app.

That reframing is worth making early because it changes the build. If voice and chat are rival products, you pick one and defend it. If they are two inputs into one agent, you build the agent once and the mode becomes a setting the user changes without thinking about it.

Two different questions get argued as one#

Agent or chatbot is a question about capability. It asks whether the thing can finish the task or only describe how to finish it, and the structural difference between those two shows up in the backend rather than the interface.

Voice or chat is a question about input. It assumes the thing doing the work is the same underneath and asks only how the user gets a request into it. A voice interface bolted onto something that can only answer questions is a slower way to read a help article aloud.

The rest of this post assumes the agent can act, which is a recent enough capability that what changed in the model layer to allow it is worth knowing. Given that, the question here is only which mode the user should reach for.

The speed argument, with its actual numbers#

Speaking is faster than typing on a phone, and the gap is large. A controlled comparison of speech input against touchscreen keyboards measured 153 words per minute for English speech against 52 for the keyboard, a factor of 2.93, with the same pattern in Mandarin at 123 against 43 (Ruan et al., 25 August 2016). Error rates during entry went the other way from the folklore: 5.30 percent for speech against 11.22 percent for the keyboard.

Date that study when you use it. It was run in 2016, on transcription of given phrases rather than on composing your own sentence, and recognition has improved since. The direction has held up in the field data. A study of 37,370 volunteers found average mobile typing at 36.2 words per minute, and reported that with one or two fingers people type about 70 percent as fast on a phone as on a full keyboard (Palin et al., MobileHCI 2019).

Four of these are about the body and the room. The other four are about whether the request survives being said out loud.

The speed number only converts into a product win when the user is entering something. An agent that saves seven taps of navigation saves more than one that saves a sentence of typing, which is the argument for measuring completion rather than input rate.

Where typing beats speaking, plainly#

Privacy is the first one and the most underrated. If the answer names an amount, an address, a medication or a balance, the user is deciding whether the person next to them hears it. That decision happens before the microphone opens, and no amount of latency work changes it.

Noise is the second. Recognition degrades and each retry costs more than the typing would have. This is why an agent should keep the keyboard reachable from the voice dock rather than treating the switch as a settings change.

Exactness is the third. An IBAN, a coupon code, a two-factor digit or a serial number is a string with no redundancy, so a single substitution makes it wrong and the user cannot hear the error in the readback. Let people type those, and let the agent fill in everything around them.

Browsing is the fourth, and it is the one teams forget. Voice is good at “do this”. It is poor at “show me what there is”, because the user does not yet have the words for what they want and a spoken list is hard to hold in working memory. Nielsen Norman Group made the point about voice-only devices a decade ago: reciting a list of options “requires users to store the options in working memory while they make a selection” (Whitenton, 31 January 2016). A screen fixes that, which is the entire case for a spoken answer that draws widgets instead of reading them out.

The same task, both ways#

The right-hand path is shorter because the agent removed navigation, not because speech is magic. A typed sentence would take the same path.
A products widget headed fresh picks, listing grocery items with sizes, prices in Egyptian pounds and a round add button on each row.
The end of the right-hand path, drawn rather than read out. Eight of these rows read aloud would be unusable, and the user could not pick from them without a second turn. Sample items from the widget renderer.

Notice what did not change. The confirm card is in both paths, because the gate belongs to the action rather than to the input mode. Notice also that the right-hand column would look identical if the user had typed that sentence. The saving comes from the agent collapsing navigation, and the mode only decides how the sentence arrived.

A spoken turn costs more interface#

This is the part that surprises teams who have shipped a chat feature and expect voice to be the same thing with a microphone button.

A typed turn has two visible states and a scrollback. The question stays on the screen, so the user never has to remember what they asked, and a wrong word is edited before it is sent.

A spoken turn has five states in our SDK: idle, listening, transcribing, thinking and speaking. Each one needs to be visible, because a user who cannot tell listening from thinking will either repeat themselves or give up. Speech leaves nothing behind, so the transcript has to be drawn or the user loses the thread. And a wrong word only becomes visible after the system has already acted on its understanding of it.

Interruption is the row that costs real engineering. In text the user just stops reading. In voice something has to cancel playback and reclaim the listening state.

Interruption is the expensive row. In text, a user who has seen enough scrolls away. In voice, the user talks over the agent and something has to cancel the speech task, stop the player and take the listening state, in that order, before the late audio arrives. Platform vendors now treat this as table stakes rather than a nicety: Google’s Live API states that “users can interrupt the model at any time for responsive interactions” (Gemini Live API documentation). The general conversation-design work around this sits in the design rules for an interface with no visible affordances, and the implementation cost is real enough to plan for.

Deciding for your app#

Three questions get you most of the way.

Where is the user’s body when this task happens? If the honest answer includes driving, carrying, cooking, gloves or a warehouse floor, voice is the only input available, and the feature either exists in that mode or does not exist for those users.

What does the request look like when a person says it out loud? If it is one sentence that carries a condition, a quantity and a time, voice is doing work that a form would need four fields for. If it is “hmm, what do you have”, it is browsing, and browsing wants a screen.

Who cannot type this comfortably? The World Health Organization estimates that “1.3 billion people experience significant disability”, which is “16% of the world’s population, or 1 in 6 of us” (WHO, 7 March 2023). Add the temporary cases: a cast, a language whose keyboard is slower than its speech, and anyone holding a child. Voice as an alternative input path is the accessibility case rather than the convenience case, and it is a stronger argument than the speed one.

Then ship both and watch which one people use for which task. The instrumentation is cheap and the answer is specific to your product. Chasing the general question is how teams spend a quarter deciding something their own logs would have answered in a week, which is the argument for treating the first weeks of real traffic as the study.

Common questions#

Is voice or a chatbot better for a mobile app? Neither, as a general answer. They are two inputs into the same agent, so the useful question is which one suits the situation the user is in. Speaking wins when hands or eyes are busy or the keyboard is the slow part; typing wins when the exchange can be overheard, the room is loud, or the value has to be exact.

Is speaking really faster than typing on a phone? On measured text entry, yes, by roughly three times. A 2016 study recorded 153 words per minute for English speech against 52 for a touchscreen keyboard. The product benefit usually comes from the navigation the agent removes rather than from the raw input rate.

Do I have to build two interfaces? No, if the agent is the same underneath. The tool surface, the confirm gate and the widgets are shared. What differs is the input path and the state display, because a spoken turn has five phases to show and a typed one has two.

When is voice actively the wrong choice? When the answer is private and the user is in public, when the environment is loud, when the value entered has no redundancy, and when the user is browsing rather than requesting. Those four cover most of the complaints teams get after launching a voice feature.

Does supporting voice mean removing the keyboard? It should not. Keeping text entry one tap away from the voice dock is what makes the noisy-room and exact-value cases survivable, and it costs nothing to leave in.

Sources#

Filed underVoiceVoice UXAgentsAccessibility

Next

Plans, limits and what ships on each.

Compare what a deployment costs

The rest of Agent basics

Open the cluster

The structural difference between something that answers and something that finishes the task, and how to tell which one a vendor is selling you.

Elsewhere on the map