Why pre-LLM assistants failed, and what changed

Speech recognition was never the reason the old assistants annoyed you. By 2018 the transcript was usually right. The sentence arrived at the next layer intact and that layer threw it away, because the sentence was not one of the sentences someone had written down in advance.

The layer that broke was the one nobody demoed#

A classic assistant ran a pipeline. Speech to text produced a transcript, a classifier matched that transcript against a fixed list of intents, and a slot filler pulled out the variables the matched intent needed. Every capability was an intent someone had authored, with sample utterances and a slot schema attached.

That design has a failure mode with a name. Research on voice interface breakdowns calls it intent pattern match failure, where the pattern for an intent requires the user to supply slot values in a particular syntax and anything outside that syntax matches nothing at all (Analysis of user interaction failures in VUIs, 2020). The user hears “sorry, I don’t understand” and cannot tell whether the problem was the words, the accent, the feature or the phrasing.

People adapted, and the adaptations are the evidence that the design was wrong. A study of repair strategies with virtual assistants found users hyperarticulating, simplifying, restarting with a fresh utterance, settling for a result they did not want, or abandoning the attempt (Frontiers in Robotics and AI, 2024). Training your users to speak like a form is not a product.

Two structural limits killed the approach regardless of how much effort went into it. Coverage never caught up, because every new capability meant hand-authoring intents and sample phrasings while users kept inventing new ways to ask. And nothing composed. “Move fifty to savings, but only if rent already cleared” is a read, a comparison and a write. A grammar can hold one intent per utterance, so a sentence with a condition in it has nowhere to go. Worth being clear about what an intent grammar is good at, since this is not a story about a bad idea: for a closed domain with a handful of commands, it is fast, cheap, predictable and testable, which is why it survived as long as it did.

What actually replaced it#

Intent comes out of the sentence#

A model does not match against a list. It reads a goal out of arbitrary phrasing, including phrasing that carries conditions, corrections and context from earlier in the conversation. The input that killed the grammar era, someone saying something nobody scripted, is the ordinary case for a model. This is the change that makes an agent inside a product a different category of thing from the assistant that shipped with your phone in 2019.

The model emits a call, not a paragraph#

Understanding on its own produces a chatbot. The change that produced agents was structured output: give the model schemas describing your functions and it returns which function to call and what to pass it. That is the slot filler’s job, done by something that generalises to arguments and phrasings nobody enumerated.

It moves the work rather than removing it. A model can only call what you have described to it, and it calls the tool whose description best matches what it read. Ambiguous names, overlapping tools and vague argument descriptions produce wrong calls with total confidence, which is why the shape of the schemas you expose turns out to be most of the engineering.

The loop can check before it acts#

The third change is the one people skip. Interleaving reasoning with acting lets a model gather evidence, then decide, then act, and recover when a step returns something unexpected. ReAct measured this against imitation and reinforcement learning baselines and reported an absolute success-rate improvement of 34 percent on ALFWorld and 10 percent on WebShop, prompted with only one or two in-context examples (Yao et al., 6 October 2022). The same paper reports that interacting with a real information source reduces the hallucination and error propagation that pure chain-of-thought reasoning produces.

Anthropic’s working definition is worth keeping because it draws the line where the engineering actually differs: workflows run through predefined code paths, while agents “dynamically direct their own processes and tool usage, maintaining control over how they accomplish tasks” (Building effective agents, 19 December 2024). Most production features are still workflows and are better for it. The category is defined by who chooses the sequence.

Recognition improved too, in a way that matters less than people assume#

Whisper trained on 680,000 hours of multilingual and multitask supervision and generalised to standard benchmarks in a zero-shot setting, where the authors report that the models “approach” human accuracy and robustness (Radford et al., 6 December 2022). Note the verb. It is doing real work, and every vendor page that upgrades it to “match” or “exceed” is quoting a paper that does not say that.

Recognition getting better raised the floor everywhere, and it did not fix the layer that was broken. A perfect transcript into an intent grammar still fails on the condition clause. That is the whole argument for treating the model layer as the thing that changed, and it is why the sensible way to read the architecture of a voice feature today is as a stack where transcription is one replaceable component near the edge.

The new problems are real#

Nothing here says the hard part is over. It says the hard part moved, and the current one is measurable in a way the old one was not.

Reliability is now the constraint. τ-bench tested agents against a simulated user, domain-specific API tools and written policy rules, and reported that agents “are quite inconsistent (pass^8 <25% in retail)” for a model whose single-attempt success rate was already below 50 percent (Yao et al., 17 June 2024). That figure describes mid-2024 models and should be dated whenever it is quoted. The gap it names, between how often something usually works and how often it always works, is a property of tool-using systems rather than of one model, which is why running the same task eight times and counting the clean sweeps tells you more than an accuracy average.

Latency is the other one, and most of it is not yours. In our own production measurements a warm turn lands at roughly 2.5 to 3 seconds, and the dominant cost is the tenant’s own backend rather than the model or our code. A first turn after a cold start is worse, because a connection has to open and the prompt cache has to fill.

And correctness of action is a separate discipline from correctness of language. A model that reads intent well will read a hostile intent equally well, so anything that writes needs a gate in your server code rather than an instruction in a prompt.

What this means if you are building now#

The practical read is that the expensive part of an assistant is no longer language. It is the surface you expose to the model, the gate in front of the writes, and the way results get drawn.

Start from the actions rather than the conversation. Write down the six things a user should be able to finish, and check that each one maps to a call your backend can already serve. If it does not, that is backend work wearing an AI costume, and it is better to find that out in an afternoon than in a quarter.

Then decide what the answer looks like, and separately whether the user should speak it or type it, which is a question about the situation rather than about the model. A model that can call get_balance and reads out “your available balance is one thousand two hundred and forty Egyptian pounds” has taken a table and made it worse. The same call rendered as a balance card with the breakdown underneath is the reason to have built any of it.

Common questions#

Why did Siri and Alexa feel so limited before 2023? Because they resolved intent by matching a transcript against a fixed list of authored intents and then filling that intent’s slots. Any phrasing outside the authored patterns matched nothing, which is a documented failure category in voice interface research rather than a subjective impression.

Was the problem speech recognition? Rarely, by the late 2010s. Recognition improved a great deal, and Whisper’s authors describe zero-shot models that approach human accuracy and robustness. A correct transcript still failed at the intent layer, which is where the design limit was.

What specifically did LLMs change? Three things. Intent is read from arbitrary phrasing rather than matched against a list. The model emits a named call with typed arguments instead of prose. And it can interleave reasoning with acting, which is what lets it check a condition before performing a write.

Are LLM-based assistants reliable now? More capable, and still inconsistent enough to need measuring. τ-bench reported pass^8 below 25 percent in its retail domain for a mid-2024 model. Treat reliability as something you test per task rather than something the model vendor supplies.

What is left to build once you have a model that can call tools? The tool surface, the confirm gate in front of anything that writes, the rendering of results as product UI, and the latency work. In practice those are larger than the model integration.

Sources#

• Yao et al., ReAct: Synergizing Reasoning and Acting in Language Models, 6 October 2022. Accessed 12 September 2026.
• Radford et al., Robust Speech Recognition via Large-Scale Weak Supervision, 6 December 2022. Accessed 12 September 2026.
• Anthropic, Building effective agents, 19 December 2024. Accessed 12 September 2026.
• Yao et al., τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domains, 17 June 2024. Accessed 12 September 2026.
• Analysis of user interaction failures in voice user interfaces, arXiv preprint, 2020. Accessed 12 September 2026.
• Repair strategies in human interactions with virtual assistants, Frontiers in Robotics and AI, 2024. Accessed 12 September 2026.