AI agents work reliably on tasks that are bounded, verifiable, and reversible — and they fail, often expensively, on everything else. Code that has tests to run against. Data extraction that a schema can validate. Scheduled research where a wrong answer is merely a wasted paragraph. Those work today. The autonomous "AI employee" that runs your operations end-to-end while you sleep is, for now, demoware — not because the models are stupid, but because of arithmetic that no amount of model improvement erases quickly.
That arithmetic is the whole story, so let's do it first. Then the honest list of what works and what doesn't.
The compounding-error problem, in plain arithmetic
Take a hypothetical — and it is a hypothetical, not a measurement of any product. Suppose an agent takes a sequence of steps, and at each step it has some probability of getting things right. Say that probability is 95%: quite good, better than a distracted human on a boring task.
Chain five steps together and the odds that all five land correctly are 0.95 to the fifth — about 77%. Twenty steps, and you're at roughly 36%. Fifty steps, and you're at 8%. The failure rate doesn't add up; it compounds. A workflow that is nearly flawless at each individual step becomes a coin flip somewhere around step fifteen, and a lottery ticket by step fifty.
Now notice what that means. Every long-horizon agent demo you have watched — "it researched the market, built the list, wrote the emails, sent them" — is a chain of dozens of steps. The demo works because it was run until it worked, and you are being shown the run that worked.
Two things break the arithmetic, and they are the only two things that do:
- Verification at each step. If the agent can check its work — run the test, validate against a schema, confirm the record exists — a failed step is caught and retried rather than propagated. Verification turns a compounding failure into a bounded, recoverable one. This is why coding agents are the success story: the test suite is an oracle that says yes or no, cheaply, forever.
- Human checkpoints. Break a fifty-step chain into five ten-step segments with a human approving each boundary, and you've turned one lottery ticket into five manageable bets.
Any agent architecture that has neither is a machine for generating confident-sounding wrong answers, and it will do so silently — which is worse than failing loudly, because a loud failure is free information and a silent one is a landmine you find in six weeks.
What actually works today
Coding agents, with tests
This is the strongest category by a wide margin, and it's not close. Devin, the agent modes in Cursor, and GitHub Copilot's agentic features all benefit from the same structural gift: software has a built-in verifier. The tests pass or they don't. The code compiles or it doesn't. The agent gets a real signal, and can iterate against it without a human in the loop.
- Who should use it: engineers with a decent test suite, on well-scoped tasks — a bug with a reproduction, a refactor with clear boundaries, a migration with a defined shape.
- Who should NOT: anyone hoping to skip code review. The agent's output still needs reading, and on a codebase with weak tests you've simply automated the production of plausible-looking bugs.
- Honest trade-off: enormous leverage on bounded tasks; steadily worse as the task gets more ambiguous. Devin in particular launched into wild expectations and landed somewhere more modest and more real. That is the normal arc, and it's fine. See our best AI coding tools roundup for the wider field.
Structured data extraction, with schema validation
Pull fields out of invoices, contracts, resumes, or emails into a defined schema. Verifiable by construction: the output either conforms or it doesn't, and non-conforming output can be rejected and retried automatically.
- Who should use it: anyone with high-volume document intake and a human review queue for the low-confidence cases.
- Who should NOT: anyone planning to auto-approve extractions without sampling. The failures here are quiet — a wrong number in the right field looks exactly like a right one.
- Honest trade-off: genuinely production-ready, with the specific caveat that you must budget for the review queue you were hoping to eliminate.
Deterministic automation with a few LLM steps
The unglamorous winner. A workflow that is mostly conventional, deterministic automation — triggers, conditions, API calls — with a model dropped into the two or three steps that need judgement, like classifying an inbound message or summarising a thread.
Zapier AI, Make, and Lindy all live here. The reason they work is precisely that they are not very agentic: the control flow is fixed and inspectable, the model is confined to the steps where fuzziness is acceptable, and when it fails, it fails at a place you can see. Lindy pushes further toward autonomy than the other two, and that's the axis to evaluate it on — more autonomy, more surface for silent error.
- Who should use it: basically everyone. This is where the actual ROI in agentic AI has quietly accumulated.
- Who should NOT: teams hoping to describe a goal in English and get a working process. You still have to build the process.
- Honest trade-off: boring, reliable, unimpressive in a demo, and it works. Browse AI automation tools for the field.
Scheduled research and monitoring
Watch a set of sources, gather what's changed, summarise it, deliver it. Errors are cheap here because the output is advisory — a wrong summary wastes two minutes of reading, not a customer. Reversibility is the point. Perplexity and similar research tools handle this well, and the citation trail means you can check the load-bearing claims yourself.
- Who should use it: anyone doing competitive, market, or regulatory monitoring.
- Who should NOT: anyone who will forward the output without reading it. That's how a hallucinated fact becomes a company position.
- Honest trade-off: low risk, real time savings, but it does not replace the judgement of reading the sources on the things that matter.
Support triage with a confident escalation path
Intercom Fin, Zendesk AI, and Forethought resolve the repetitive tier-one tickets and — critically — hand off when unsure. The escalation path is the safety mechanism. An agent that knows what it doesn't know is a fundamentally different product from one that doesn't.
- Who should use it: support teams with a large volume of documented, repetitive questions.
- Who should NOT: anyone with a complex or high-stakes product who plans to make the bot hard to escape. Users are extremely good at detecting a wall, and they get angry at it.
- Honest trade-off: genuine deflection on FAQs; anger and churn if you tune the escalation threshold too high to save money.
What's still demoware
Fully autonomous general-purpose agents
The "give it a goal, walk away" category — the lineage that runs through AgentGPT and the wave of autonomous loop frameworks that captured everyone's imagination. These are genuinely fascinating, and worth playing with to build intuition about what agents can and cannot do.
But they are the purest expression of the compounding-error problem: long chains, no external verifier, no checkpoints. They will confidently pursue a subgoal that drifted three steps ago, and produce something that looks like a completed task. The honest use case is exploration and learning, not production.
The "AI employee"
Anything marketed as replacing a role rather than automating a task. The framing is a tell: real work is not a single long chain of steps, it is a hundred short chains punctuated by judgement calls, context that lives in someone's head, and the constant small act of noticing that something is off. Agents have no mechanism for noticing that something is off.
Long-horizon business operations without checkpoints
An agent that researches prospects, writes outreach, sends it, handles replies, and books meetings — end to end, unsupervised. Every step is irreversible (you cannot unsend an email to a prospect) and unverifiable (there is no test suite for "was that a good email"). This is the exact opposite of the conditions where agents work, which is why the category is full of impressive demos and quiet churn.
The three-question test
Before you deploy any agent, ask:
| Question | If yes | If no |
|---|---|---|
| Bounded? Is the task scoped, with a clear finish line? | Proceed | Break it into smaller tasks or don't automate it |
| Verifiable? Can something other than the agent check the work — cheaply, automatically? | Proceed; let it retry | Insert a human checkpoint, or don't automate it |
| Reversible? If it's wrong, can you undo it? | Proceed | Human approval before the irreversible step. Always. |
Swipe the table sideways to see every column →
Three yeses: automate it, and expect it to work. Two: automate it with a human at the boundary. One or zero: you are buying a demo, and the demo will keep working right up until it's load-bearing.
Notice that this test says nothing about which model or vendor you use. Model quality moves the reliability of each step. It does not change the structure of the chain, and the structure is what determines whether the thing survives contact with reality.
Where the money actually is
The unsexy conclusion: the highest-return agentic work in most companies right now is a handful of well-built automations with a model doing judgement in two or three steps, plus a coding agent for developers who already have tests. That's it. That's the list.
It doesn't demo well. Nobody raises a round on it. It works.
If you're evaluating the category, start with AI agents and AI automation tools side by side and notice how much overlap there is — the boundary between "agent" and "automation with an LLM step" is largely marketing, and the more honest products sit on the automation side of it. The glossary is worth a pass if the vocabulary is being used loosely around you, which it will be.
FAQ
Do AI agents actually work?
Yes, on tasks that are bounded, verifiable, and reversible. Coding agents with a test suite, structured data extraction validated against a schema, scheduled research, and conventional automations with a couple of LLM judgement steps all work in production today. Fully autonomous, long-horizon agents that operate without checkpoints do not reliably work, because errors compound across steps and nothing catches them.
Why do AI agents fail on long tasks?
Because per-step reliability compounds multiplicatively. Even a highly reliable step, repeated across dozens of steps with no verification between them, produces a low probability that the entire chain is correct. Worse, the agent has no way of knowing it went wrong, so it continues confidently from a bad state. Verification and human checkpoints are the only structural fixes.
Are AI agents worth it for a small business?
Almost certainly yes — but not the ones being marketed at you. The value is in tools like Zapier AI, Make, or Lindy handling well-defined workflows with a model in the judgement steps. Skip anything promising an autonomous AI employee. Start with the single most repetitive process you have, automate that, and see whether it holds up for a month.
What's the difference between an AI agent and an automation?
Less than the marketing implies. An automation has a fixed, inspectable control flow that you designed; an agent decides its own next steps. That autonomy is exactly what makes agents powerful in exploratory tasks and fragile in production, because a fixed control flow fails in ways you can see and predict. Most successful "agent" deployments are automations with model calls inside them.
Will AI agents replace jobs soon?
Not in the way the pitch decks suggest. Agents automate tasks, and a job is a bundle of tasks plus judgement, context, relationships, and the ability to notice when something is wrong — the last of which agents categorically lack. Expect roles to change shape as the bounded, verifiable parts get absorbed. Expect the "AI employee" to remain a marketing category for a while yet.
How do I evaluate an AI agent vendor?
Ask them what happens when it's wrong. A serious vendor has an immediate, specific answer about verification, retries, escalation, and audit logs. A vendor that redirects to model quality or accuracy claims is telling you they have not designed for failure — and failure is not an edge case, it is the thing you are buying insurance against.
Be the skeptic in the room
The agent category is in the phase where every ranking article was written by someone selling an agent. That should tell you what the incentives look like right now.
Run the three-question test. Automate the bounded, verifiable, reversible things — there are more of them than you think, and they pay. Refuse the rest until someone shows you what happens when it's wrong. When you're ready to compare, the agent and automation categories are indexed side by side, and the tools for developers collection covers the coding-agent end. Building something in this space that handles failure honestly? Submit it — we don't sell software, so we can afford to say when it doesn't work.
