When not to put an AI agent in your app

We sell the thing this post argues against, some of the time. That is worth saying at the top, because the four cases below are the ones where a customer would be worse off after buying from us, and we would rather you find them here than three months into a build.

The short answer: if the task is already two taps, leave it alone#

An agent earns its place when it collapses a long or vague task into a sentence. If the task is already two taps from the screen the user is on, speaking or typing a request is slower, less reliable, and more effortful than tapping. No amount of model quality fixes that, because the bottleneck is not the model.

That is the test in one line. The rest of this post is the cases where it is less obvious.

Four cases where an agent loses#

The task is faster by hand#

Reordering a weekly grocery shop is thirty taps and a good candidate. Paying a bill that has a Pay button on the home screen is one tap and a bad one. The failure mode here is not that the agent answers wrongly, it is that it answers correctly and the user still wishes they had tapped.

The quadrant that matters is how often the user does the task against how exact the answer has to be.

The user is browsing, not asking#

Conversation is a poor interface for finding out what exists. A user who does not yet know what they want needs a surface to scan, and a scannable surface is a grid or a list, not a dialogue. This is the oldest objection in interface design and it has not stopped being true.

The version of it that shows up in practice is subtler: users who would have browsed will ask, badly, because the input box is there. They will then get a precise answer to a question they did not mean, and conclude the feature does not work.

The answer must be exact and auditable#

Some numbers have to come from one place, be identical every time, and be defensible later. A tax figure, a legally required disclosure, a medication dose. An agent that paraphrases such a number has introduced a second source of truth into your product, and the fact that it is right 99 times is not the relevant statistic.

The usable middle ground is to let the agent navigate rather than answer: it can take the user to the authoritative screen, which is a real saving, without becoming a second publisher of the number.

You cannot expose the data the agent would need#

If answering the question requires data your API cannot return scoped to a single user, you do not have an agent problem, you have an API boundary problem, and building the agent first will make you solve it under deadline. This is the case that most often gets discovered late, because it looks like an integration detail until someone asks who exactly the tool call is authenticated as.

What the independent evidence says about self-service#

The closest thing to a prior on this comes from customer service research, and it is not encouraging for anyone assuming an assistant will absorb a support load.

Gartner surveyed 5,728 customers and found that “only 14 percent of their customer service and support issues were fully resolved by a company’s self-service channel”, as reported in August 2024. The same write-up notes that “while 73% of customers use self-service at some point in their customer service journey, it’s concerning to see that so few fully resolve there”, and that “even for issues described by customers as ‘very simple,’ only 36 percent were handled in self-service.”

A separate Gartner survey, this one of 265 customer service and support executives and reported in September 2025, found that “nearly 9 in 10 customer service journeys beginning in self-service are ultimately resolved through multiple channels, such as phone or email.”

Two caveats, because they change what these numbers mean for a product team. Both studies are about customer service rather than in-app tasks, and an agent that reorders groceries is not competing with a call centre. And they are different studies with different populations, so the 73% and the 14% and the 9-in-10 do not chain into one funnel, however much they look like they should. What they support is a prior, not a forecast: self-service has historically absorbed far less than its sponsors projected, and there is no reason to assume this generation is exempt without measuring it.

The deflection number is the wrong number to optimise for exactly this reason. A user who gave up is indistinguishable from a user who was helped, in every dashboard that counts deflections.

The objections your own users will raise#

Your users have already formed opinions about this, and they are more specific than you expect. The best public collection is a December 2025 Hacker News thread, “What Is Generative UI?”, where people who build software for a living say what worries them.

The commenter bccdee, on interfaces that adapt: “Personalized interfaces are bad. I don’t want to configure anything, and I don’t want anything automatically configured on my behalf.” And on the consequence for support, imagined by marcyb5st as a line from a future call: “Yeah, I cannot reproduce your issue because things on my end look different”.

Developer sentiment points the same way. Stack Overflow’s 2025 survey reached 49,009 people recruited through Stack Overflow’s own channels, and its AI questions were answered by smaller subsets. Among the 33,244 who answered the accuracy question, “more developers actively distrust the accuracy of AI tools (46%) than trust it (33%)”. Among the 31,476 who answered the frustrations question, 66% name “AI solutions that are almost right, but not quite” as their biggest.

Some of these objections have structural answers. An interface that varies per user is a real problem; a fixed catalogue of views the app already ships removes it, because the same question produces the same view every time. The distrust figure has no structural answer at all. It is a reason to ship something that completes a task rather than something that talks, and to let it be judged on that.

The test: can you write the three sentences?#

Before committing a quarter, write down three sentences a real user would actually say to start this task. Not paraphrases of your feature list. Sentences, with their hesitations and their shorthand, the way the user would say them out loud in a queue.

If you cannot produce three, the feature is a demo. If all three are the same sentence, you have a button, and you should ship the button. If the three sentences are genuinely different and all three would need the same handful of tool calls, you have found something worth building.

This is a cheap test and it is uncomfortable in a useful way, because it forces the question of whether anyone would phrase it that way unprompted.

Anthropic give the same advice from the engineering side, in Building effective agents: “when building applications with LLMs, we recommend finding the simplest solution possible, and only increasing complexity when needed. This might mean not building agentic systems at all.”

There is also a cost the test does not capture, which is the cost of asking. Nielsen Norman Group’s EAS framework (7 March 2025) puts it well for forms, and it transfers: “building a good form is like managing a bank account of trust. Every question is a withdrawal.” An agent that asks three clarifying questions before doing anything is making three withdrawals against a balance you may not have. That article is expert guidance rather than an experiment, and carries no quantitative evidence, so treat it as a framing rather than a finding.

Where it does earn its place#

The cases that survive all four questions look alike. The task is many taps done often, the user knows what they want before they start, the data is already exposed per user, and there is no legal requirement that the answer be canonical.

Grocery reordering is the clearest example. A basket is dozens of decisions the user has already made before, which is exactly the shape that collapses into a sentence.

Two details matter in that screenshot and neither is the agent. The result is a scannable list rather than a paragraph, so the user can fix a wrong item without another round of conversation. And nothing was bought. The distinction between an agent and a chatbot is that an agent can act, and the reason that is safe is that acting stops for a confirmation.

The honest version of the pitch is narrow. An agent is good at collapsing long, familiar, open-ended tasks, which is why rebooking a disrupted trip survives this test when most of a travel app does not. Everything else in your product is better served by the interface you already have, and the same question asked about voice specifically has a narrower answer still.

Common questions#

When should you not use AI in an app?#

When the task is already two taps, when the user is browsing rather than asking, when the answer must be exact and auditable, or when the data cannot be exposed scoped to one user. Three of those four are the common case for most screens in most apps.

Do users actually use in-app AI assistants?#

There is no good public number for in-app assistants specifically. The nearest evidence is customer service self-service, where Gartner found in 2024 that only 14% of issues were fully resolved in the self-service channel across 5,728 customers surveyed. That is a prior worth respecting and not a prediction about your app.

Is an in-app AI assistant worth building?#

It is worth building for tasks that are frequent, open-ended and many taps by hand. For anything precise or rare, the existing form or button wins. The cheapest way to find out is to write three sentences a user would say and see whether they are actually different from each other.

What makes an AI assistant bad UX?#

Answering where a button would have been faster, varying its interface between users so nobody can build a habit or reproduce a bug, asking clarifying questions the user did not budget for, and being almost right. Stack Overflow’s 2025 respondents named that last one, “almost right, but not quite”, as their single biggest frustration with AI tools, at 66%.

Sources#

• CX Today, Only 1 in 7 Customer Service Queries Resolved With Self-Service, Gartner Study Finds, 19 August 2024, reporting a Gartner survey of 5,728 customers.
• CX Dive, CX leaders say self-service, live chat overtake phone and email, 3 September 2025, reporting a Gartner survey of 265 customer service and support executives.
• Stack Overflow, 2025 Developer Survey, AI section, 49,009 responses, fielded 29 May to 23 June 2025, recruited through Stack Overflow’s own channels.
• Hacker News, What Is Generative UI?, submitted 3 December 2025.
• Nielsen Norman Group, The EAS Framework: Eliminate, Automate, Simplify, 7 March 2025. Expert guidance; the article presents no quantitative evidence.
• Anthropic, Building effective agents, 19 December 2024.