Agent or chatbot: telling the two apart before you buy

Both things sit in the corner of an app behind a small round button. Both open a text field. In a twenty-minute vendor call they look close enough that the question gets settled on price, and then six months later somebody works out that the thing shipped can describe a refund policy but cannot issue a refund.

The two are built differently, and five questions will pull them apart in any demo. If you have not yet settled what an in-app agent even is, start there; this post assumes you are already holding two quotes.

The difference in one sentence#

A chatbot maps a message to a reply. An agent reads the user’s current state, calls tools against your backend, stops for an approval before it writes anything, and hands back product UI rather than a paragraph. Anthropic’s engineering team draws the same line between agents, which are “systems where LLMs dynamically direct their own processes and tool usage, maintaining control over how they accomplish tasks”, and workflows, which run through predefined code paths (19 December 2024). A support bot is the far end of that second category: one path, one call, one string back.

Five questions that separate them in a demo#

Ask these in order. Each one is answerable inside the call, and each has a wrong answer that a prepared vendor cannot talk around.

Can it read the user’s current state?#

Ask the demo assistant something only the logged-in account knows. Yesterday’s order, this month’s balance, the delivery window on the thing in flight right now. A retrieval chatbot answers from documents, so it will either decline or produce something plausible and general.

This matters more than it sounds, because models are bad at checking state even when they can. The Berkeley Function Calling Leaderboard’s multi-turn work walks through this as one of three recurring failure scenarios: even the strongest models “sometimes fail to explore the current state before performing actions, which can be dangerous if the actions are non-reversible” (BFCL V3, 19 September 2024, updated 10 December 2024). A vendor who has thought about this will tell you which read happens on every turn. A vendor who has not will show you a cached answer.

Can it write?#

Ask it to do something, then check whether the thing happened. Not whether the assistant said it happened. Open the account in a second window and look.

The gap between narrating an action and performing one is where most demos live. It is also the gap that decides whether the feature earns its budget, because a system that can only read is a search box with better manners.

What does it show, besides text?#

Watch the screen while the answer arrives. If twelve transactions come back as a bulleted list inside a chat bubble, the vendor has a chatbot with a good prompt. If they come back as your list component, with your row layout and your tap targets, something is rendering product UI from a structured payload.

The difference is not cosmetic. A number in a card can be tapped, sorted, and acted on. The same number in a sentence is a dead end that the user has to re-enter somewhere else. The mechanism behind it, an agent that draws your own components at runtime rather than shipping markup, is a longer argument than this post has room for.

What happens before a write?#

Ask what stands between the model choosing to move money and the money moving. There are only two possible answers, and one of them is wrong.

The wrong answer is that the system prompt instructs the model to confirm first. That is a suggestion the model can be argued out of. The right answer is that the write path branches in server code before the call is dispatched, holds the arguments, and returns a card the user has to accept. Somebody outside this industry put it better than we can: “human-in-the-loop is a security boundary, and security boundaries belong in code, not in prompts” (pavelgj on dev.to). The MCP specification takes the same position, requiring that hosts “obtain explicit user consent before invoking any tool” (revision 2025-06-18). We take the confirmation apart properly elsewhere, including what the card has to show and what the agent must never say out loud.

What does it do when it fails?#

This is the question nobody prepares for, so ask it last and ask it rudely. Unplug something. Give it an order number that does not exist. Ask for a refund on an item that was never bought.

A chatbot apologises and offers to connect you to a human. A well-built agent names the thing that failed, says which step it got to, and offers the action that is still available. The failure message is where you find out whether the vendor has run this against real traffic, because real traffic produces failures that nobody designs for.

Why prompting a chatbot harder does not produce an agent#

The instinct, once the difference is clear, is to reach for the prompt. It does not work, and the reason is worth understanding before you spend a quarter finding out.

An agent needs four things a chatbot does not have: a live read of the user’s account, a set of tools bound to that user’s credentials, a branch in the write path, and a way to return UI instead of prose. None of those live in the prompt. They live in the runtime, the tool layer, and the client. You can write a very persuasive instruction telling a model to check the balance before answering, and it will still answer without checking, because there is nothing for it to call.

The measured reliability is worse than most people assume even when the plumbing is there. τ-bench put agents in exactly this shape, with a simulated user, real API tools and written domain policy rules, and found that “even state-of-the-art function calling agents (like gpt-4o) succeed on <50% of the tasks”, and that they “are quite inconsistent (pass^8 <25% in retail)” (Yao et al., 17 June 2024). Date that figure when you quote it: it describes GPT-4o in mid-2024 and the field has moved. The structural point has not moved. Reliability comes from the tool surface and the gates around it, which is why how you evaluate it belongs in the procurement conversation and not after launch.

The scorecard you can fill in during the call#

Five rows, one call. If three or more land in the left column, you are buying a chatbot, which is a perfectly reasonable thing to buy as long as you know that is what it is.

Where a chatbot is still the right answer#

Plenty of the time. The honest version of this argument has to include the cases where the simpler system wins, and Anthropic’s own guidance says the same: “for many applications, optimizing single LLM calls with retrieval and in-context examples is usually enough.”

A chatbot is the right call when the questions are about policy rather than state, when the corpus changes more often than the product does, and when there is no write worth gating. Documentation search, onboarding explainers, and “how do I do X in this app” all sit there comfortably.

It is worth keeping the base rate in view for both. Gartner research covering 5,728 customers found that only 14 percent of issues are fully resolved in self-service, and that even issues customers described as very simple resolved only 36 percent of the time (reported by CX Today, 19 August 2024). A follow-up survey of 265 customer service executives found that “nearly 9 in 10 customer service journeys beginning in self-service are ultimately resolved through multiple channels” (CX Dive, 3 September 2025). Neither number is an argument against building; both are an argument against believing a deflection projection from anyone, us included.

Your buyers are sceptical for good reason, too. The 2025 Stack Overflow Developer Survey found that 46 percent of developers actively distrust the accuracy of AI tools against 33 percent who trust it, and that 66 percent name “AI solutions that are almost right, but not quite” as their biggest frustration (Stack Overflow, 2025). A vendor who leads with a completion-rate promise is selling to the 33 percent.

The vendor comparisons that currently rank for this question are mostly listicles published by one of the vendors being compared, which is a genre rather than a source. The five questions above work better because you run them yourself, on the demo in front of you, and the answers are checkable in a second browser window.

Common questions#

What is the difference between an AI agent and a chatbot in a mobile app? A chatbot turns a message into a reply. An agent reads the signed-in user’s live state, calls tools against the app’s own backend, pauses for explicit approval before any write, and returns native UI rather than text. The agent needs a runtime, a tool layer and a confirm gate; the chatbot needs a model and a corpus.

Can I turn my existing chatbot into an agent by improving the prompt? No. The missing pieces are a live state read, tools bound to the user’s credentials, a server-side branch on the write path, and a UI contract. A prompt cannot call something that does not exist.

How do I test in a demo whether an agent really takes actions? Ask it to perform a write, then verify the change in a separate window rather than reading the transcript. Then break something, such as an order number that does not exist, and watch whether the failure message names the step that failed.

Is a chatbot ever the better choice? Yes, when the questions are about policy rather than account state, when the knowledge base changes faster than the product, and when there is no write worth gating. Documentation and onboarding questions are the clearest cases.

Sources#

• Anthropic, Building effective agents, 19 December 2024. Accessed 12 September 2026.
• Yao et al., τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domains, 17 June 2024. Accessed 12 September 2026.
• Berkeley Function Calling Leaderboard, BFCL V3: Multi-Turn and Multi-Step Function Calling, 19 September 2024, updated 10 December 2024. Accessed 12 September 2026.
• Model Context Protocol, Specification revision 2025-06-18. Accessed 12 September 2026.
• pavelgj, Human-in-the-loop AI: why “ask the LLM to confirm” isn’t enough, dev.to. Accessed 12 September 2026.
• CX Today, Only 1 in 7 customer service queries resolved with self-service, Gartner study finds, 19 August 2024. Accessed 12 September 2026.
• CX Dive, Self-service, live chat overtake phone and email, 3 September 2025. Accessed 12 September 2026.
• Stack Overflow, 2025 Developer Survey: AI. Accessed 12 September 2026.