The request arrives through an ordinary workflow.
An AI agent is asked to disable an employee account. It identifies the account, calls the administrative tool, receives a successful response, confirms the state change, and closes the ticket.
The account is disabled.
The command returns cleanly. The workflow completes. The dashboard stays green. No exception is thrown. Nobody gets paged.
From an operational perspective, the system worked.
The problem is that the account should never have been disabled.
Perhaps the request came from a source that was allowed to supply information but had no standing to authorize an account change. Perhaps the approval was real when it was issued but no longer valid when the action ran. Perhaps a control recorded the approval without actually evaluating whether it applied to this account, this request, at this moment.
Nothing malfunctioned. The machine did precisely what the system permitted it to do.
That is what makes the failure difficult to see.
The failure is not in whether the action happened. It is in why it was allowed to happen.
Two different properties
We routinely treat task success and system correctness as though they were different measurements of the same thing.
They are not.
Task success asks whether the machine did the thing it was asked to do. Did the command execute? Did the file disappear? Did the account become disabled? Did the transaction complete?
System correctness asks a different question: should that action have been performed on the basis the system received?
Those two properties can diverge completely.
A machine can execute an action perfectly while the governance around that action is completely wrong. The requested file can really be deleted even though the requester had no authority over it. Money can really move even though the approval had expired. An account can really be disabled even though the source that selected it was never entitled to request that consequence.
Nothing in task completion resolves any of those questions.
Yet most of the machinery surrounding automated systems is designed to tell us whether execution succeeded. We measure availability, latency, error rates, retries, exit codes, transaction completion, and workflow closure. These measurements are useful, and they tell us whether the system is operating. They do not tell us whether a successful action was justified.
Task completion establishes that the job ran. The basis for running it is a different property, and that property has to be established separately.
Why monitoring cannot see this
This is not simply a case where monitoring needs one more alert.
Operational monitoring is built around malfunction. Something departs from expected behavior, and the departure creates a signal. A process crashes. Latency rises. A request fails. A queue stops draining.
The underlying assumption is reasonable: when something goes wrong, the system will behave differently from when things go right.
Wrong-basis actions break that assumption.
If an agent disables the wrong account because the authority behind the request was stale, the administrative API may still return success. If it follows an instruction from a source that should never have been able to authorize the action, the workflow may still close normally. If a control was present only as logging and never actually evaluated the request, every downstream operational metric can still look healthy.
A right-basis action and a wrong-basis action can therefore produce the same operational signature. The system is not slow, the tool is not broken, the request did not fail, and the state change occurred exactly as expected. There is no malfunction for a malfunction detector to detect.
A dashboard built to answer “is the system working?” can remain entirely green while the system carries out an action it should have refused.
The danger is not that the dashboard is inaccurate. It may be perfectly accurate about the property it measures.
The danger is that people silently promote “working” into “correct.”
That promotion is the whole failure. The green dashboard has always meant the machine is running, and running has always been close enough to correct that nobody had to separate them. Automated consequential action is what forces them apart. Once “working” is read as “correct,” a clean execution trace begins to function as evidence for a governance property it never established.
The measurement that is missing
If task success is not enough, the answer is not to monitor the same execution path more aggressively.
The missing measurement concerns the basis of the action.
At the point where a consequential action is about to occur, the system needs to establish why this action is permitted now. Was the authority current? Was the source entitled to request this consequence? Did the approval apply to this exact action? Did a control actually evaluate those conditions at a point where it could refuse?
Those questions are not answered by the success response from the tool. They are not answered merely by having logs, either.
A log can show that an action occurred. It can record an approval identifier, a policy result, a user identity, or a control name. That does not establish that the relevant property was evaluated correctly at execution time.
There is a crucial difference between recording that a control participated in a workflow and having evidence that the control established the property everyone is relying on.
A system can record approved=true because an upstream field contained that value. An authority check can be genuine and still be stale by the time the action occurs. A control can appear in the record without any evidence that it evaluated anything.
All of those systems can produce convincing records. None of those records, by itself, proves that the action had a valid basis when it ran.
This is why the answer is not more monitoring.
A wrong-basis action cannot be made visible by watching harder for malfunction, because there is no malfunction to watch for. Whatever catches it has to evaluate the basis rather than observe the execution, and that is a different kind of mechanism, in a different place. It has to sit where the action can still be refused, before the consequence, and it has to produce its own evidence that the basis was established, kept separate from the fact that the action later succeeded.
This essay does not try to specify that mechanism, and none of this means the failure is undetectable. It means the failure is detectable by something other than the thing most teams are watching. Building the mechanism that evaluates the basis is real work and a separate subject. The subject here is narrower and harder to argue with: execution success proves the machine did the thing, it cannot prove the machine had a valid reason to do it, and the place most people are looking will never show them the difference.
What this does not claim
This is not an argument against operational monitoring, and it is not a claim that every successful automated action conceals a governance failure.
Monitoring catches real failures. Crashes matter. Timeouts matter. Broken dependencies matter. Systems need health checks, alerts, traces, and operational evidence.
The claim is narrower.
There is a class of failure in which the machine works perfectly and the action is still wrong because the basis for permitting it was stale, untrusted, unevaluated, or mismatched to the action that actually occurred.
There is also a longer history here that this essay should not pretend to be discovering. Security, audit, and governance functions have fought wrong-basis success for decades. Change reconciliation, separation of duties, and post-hoc authorization review exist precisely to catch actions that executed cleanly but should not have, and where they are in place they do catch this class. The failure is not invisible to them. It is invisible to the operational telemetry most teams watch in real time.
What agentic automation changes is not the shape of the failure but its economics. Those controls are mostly post-hoc, built for change at human pace with a legible path from a written approval to a specific action. An agent can take thousands of consequential actions between reconciliation cycles, and the path from a natural-language instruction to a precise tool call runs through reasoning no change ticket ever had to. The old controls assumed the action rate was slow enough to review and the mapping clear enough to check. Agentic systems break both at once. That is what makes an old failure mode newly acute, not newly existent.
The individual failures also have separate remedies: a stale approval is a temporal problem, an untrusted source an authentication problem, a log-only control an enforcement problem, a mismatched scope a binding problem. What they share, and what this essay is about, is the structure underneath all of them.
There are also questions beyond the scope of this problem. A correctly authorized action can still be unwise. An approval process can be valid while the policy behind it is bad. A basis can be correctly established and still be incomplete for the real-world decision being made.
Those are important problems, but they are different ones.
This problem begins earlier: did the system establish a valid basis for this exact consequence before it allowed the consequence to occur?
The smallness of the signal
We are good at noticing failures that announce themselves.
An exception appears. A process stops. An alert fires. A metric crosses a threshold. Something visible departs from normal operation, and the system tells us to look.
A wrong-basis success offers no such courtesy.
The action runs. The state changes. The ticket closes. The monitoring system reports what it was designed to report.
The signal that something was wrong is not a red alert. It is the absence of a check nobody ran, in a record nobody kept.
That absence is easy to miss because the rest of the system can look exactly as it should.
The most dangerous failures are not always the ones that break the system. They are the ones the system carries out correctly, on a basis nobody established, and reports as done.
Related — two specific faces of this same problem:
- The Human Approved It. Which “It”? — a successful action that was not the action anyone approved.
- Prompt Injection Is an Authority Problem Before It Is a Language Problem — a successful action authorized by a source that had no standing to authorize it.