What an AI agent should not be able to do

A good agent refuses. It is the least marketable quality signal and the most reliable: what an agent is incapable of doing says more about how it was built than the list of what it can do.

And the word that matters is incapable. A rule written in an instruction is bypassed by another instruction, text against text. A limit that holds is a permission not granted, an action that does not exist, an approval enforced in code.

Here are the five I would check before signing.

1. Settling a decision that closes a door

Rejecting an application, choosing between two consultants for an assignment, delivering a no. An agent should hand back the list and its reasoning, never the decision.

This is not a moral precaution, it is the legal line: GDPR governs fully automated decisions producing a significant effect, and a rejected application is one. That obligation has applied since 2018 and was untouched by the AI Act deferral.

The test is direct. Ask the agent, explicitly: “reject the profiles that do not fit.” A well-built agent returns the profiles ranked, explaining why the last ones are last. A badly built one returns a shortened list, and you will never know what it contained.

2. Acting outside the scope you opened

An agent should be able to do exactly what you granted it and nothing else, not because it was told not to, but because the action does not exist for it.

Concretely: if it has read access to your ATS, it should not be able to write. If it can write to records, it should not be able to delete. If it can send an email from your inbox, it should not be able to delete one.

And that scope should be revocable in one move. Access you can only withdraw by raising a ticket with the vendor is not controlled access. The question to ask: how long between “I want this cut off” and “it is cut off”?

3. Retrying in silence

The most ordinary failure and the one that produces the most embarrassing incidents.

An agent sends a message, the network call fails as the response comes back, and it does not know whether the message went. It retries. The candidate receives the same follow-up three times, thirty seconds apart.

A sound agent treats sending as a non-repeatable action: it records the attempt, checks before repeating, and when in doubt asks rather than assumes. “I am not certain that message went out, should I resend it?” is a good answer from an agent.

The question for the vendor is technical but fits in a sentence: what happens if a send fails halfway?

4. Hiding its uncertainty

An agent that does not know should say so. That is rarer than you would think, because a language model’s default behaviour is to produce a plausible answer rather than admit a gap.

In practice uncertainty takes three forms, and all three should be visible. Missing data: “no availability recorded for this profile”, rather than an estimate. Stale data: “availability unverified for eight months”, which is the difference between a useful shortlist and a dangerous one. A thin result: “I found only two profiles matching every criterion; here are three more that miss one.”

An agent that always returns five profiles when you ask for five is an agent inventing the fifth.

5. Crossing the boundary between two customers

This one never shows in a demo and is very expensive when it fails.

An agent working for several organisations should be structurally incapable of leaking one’s data into another’s. Not “configured not to”: incapable, because the query carries the customer’s identity and a query without it returns nothing.

The same logic applies inside one firm. If a consultant has a private memory: their preferences, their notes: it should not surface in somebody else’s search.

And the question that comes with it: is my data used to train a model? The right answer is no, contractually, including at the model providers downstream.

The limit people forget: acting without leaving a trace

The five above are about what the agent does. This one is about what can be known afterwards, and it underwrites all of them.

An agent should be incapable of acting without recording what it did. Not a technical log reserved for the vendor: a trace you can read and export, saying which action, at what time, on what data, on whose request, approved by whom.

Without it, the previous five limits are unverifiable. You will not know the agent sent the same message three times, nor that it rejected a profile, nor that it read data it should not have reached. You will only know a candidate complained, six weeks later.

This is also where compliance meets operations. Log retention is among the Annex III obligations under the AI Act, applicable from 2 December 2027, but a log is not primarily a regulatory object. It is what lets you say “here is what happened” rather than “the AI did something odd”, and that difference decides whether a project survives its first incident.

What a good refusal looks like

Refusing badly costs almost as much as not refusing. An agent that answers “I can’t do that” and stops gets worked around: the user rephrases until something gets through, or abandons the tool.

A useful refusal does three things in one sentence. It says what it will not do. It says why, in terms of consequence rather than rule. And it offers what it can do instead.

“I won’t reject these profiles myself: that decision has to be traceable and it is yours. Here are the twelve profiles ranked, each with the criterion it misses.” In one sentence the user has understood the limit, does not feel blocked, and has what they wanted in another form.

It is a detail of wording that decides adoption. An agent that refuses curtly gets worked around; an agent that refuses while delivering something else gets respected.

The bad-request test

All five can be checked in a single demo, provided you run the demo yourself.

Make the forbidden request. “Send it directly, don’t ask me.” “Delete the profiles that do not fit.” “Do it for client X based on what worked at client Y.”

A solid vendor will be pleased with the question: it is where they get to show what they built. A vendor who answers “we didn’t set it up that way” is telling you the limit does not exist; it is simply absent from their usual demo.

Why refusing makes an agent more useful

The opposite intuition is strong: an agent that accepts everything looks more powerful. In production it is the reverse.

An agent that accepts everything fails rarely but spectacularly, and a spectacular failure costs the whole team’s confidence for six months. An agent that refuses what it cannot do properly gets delegated a little more every week, because nothing ever contradicts the idea that it can be trusted.

It is the same mechanism as choosing the first tasks to delegate: trust is built on what is verifiable and spent on what is not. A well-built agent stops you spending it too fast.