# The human cannot be the bottleneck > Human oversight cannot scale by requiring a person to inspect every machine action. Human authority must scale by concentrating judgement where evidence, uncertainty and consequence require it. Paper: 01.2 Published: 2026-08-24 Canonical: https://harten.io/papers/01-2-human-cannot-be-the-bottleneck/ Original publication: https://www.linkedin.com/pulse/012-human-cannot-bottleneck-harten-technologies-zugoe Cover illustration: [View the title image](https://harten.io/media/papers/covers/v1/01-2-human-cannot-be-the-bottleneck.jpg?v=0485293af7fb). Illustrative cover artwork, not an application screenshot or a record of measured results. AI is making execution dramatically faster. That creates an uncomfortable consequence for how we think about control. If an AI system can analyse, plan, modify and verify software in minutes, but a human must inspect and approve every individual action, we have not created an autonomous system. We have created a faster queue. The obvious response to increasingly capable AI has been to put a human in the loop. That principle is directionally correct. Its simplest implementation is not: **AI does everything → human checks everything.** At small scale, this feels safe. At machine scale, it becomes a bottleneck. The question is therefore not whether humans remain in control. They must. The question is **where human judgement creates the most control**. That requires a different model: **Autonomy increases as evidence strengthens. Scrutiny increases as consequence grows.** ## Human attention is a control resource There is a tendency to treat human oversight as effectively unlimited. It is not. Every approval consumes attention. Every review requires context. Every interruption forces someone to reconstruct why a decision was made, what evidence supports it and what could happen if it is wrong. As the number of agents and the speed of execution increase, this problem compounds. Imagine an AI modernisation system processing thousands of files across a large software estate. It discovers dependencies, classifies components, proposes transformations, generates code, runs tests, evaluates results and produces further actions. Requiring a person to approve every one of those steps does not necessarily make the system safer. Eventually, it creates approval fatigue. The human becomes conditioned to accept. The control remains visible, but its effectiveness deteriorates. **A thousand approval buttons are not a governance model.** ## Not every action carries the same consequence Software engineering already understands this intuitively. Reading a source file is not equivalent to deleting production data. Generating a candidate transformation is not equivalent to deploying it. Running a test is not equivalent to changing an external interface. Reformatting generated code is not equivalent to changing authentication behaviour. Yet many approaches to agentic governance treat actions as though they belong to the same risk class. They do not. The more useful abstraction is a **consequence boundary**: a boundary that separates actions according to what happens if the system is wrong. Below the boundary, actions may be reversible, isolated or observational. Above it, actions begin to affect persistent state, external systems, security boundaries, contractual behaviour or production outcomes. Autonomy should therefore not be granted as one binary property of an agent. It should be granted to classes of action under explicit conditions. ## Evidence changes the autonomy equation Consequence alone is not enough. We also need to know how strongly the system can justify what it is doing. Consider two proposed transformations. In the first, the system identifies a dependency from a single ambiguous code reference. In the second, the same conclusion is supported by static analysis, repository structure, runtime evidence and an existing test suite. The proposed action might be identical. The evidential position is not. That difference should change how much autonomy the system receives. This produces a useful relationship: **Weak evidence → greater scrutiny.** **Strong evidence → greater bounded autonomy.** ## Bounded autonomy This is different from simply giving an agent permission to operate independently. Bounded autonomy means defining the conditions within which independent action is acceptable. An agent might be allowed to inspect repositories without approval, construct dependency models automatically, generate candidate transformations, execute isolated tests, retry failed analysis within defined limits, or accept low-consequence changes when evidence exceeds a defined threshold. The same system might require explicit human authority before changing an architectural boundary, modifying security-sensitive behaviour, accepting unresolved ambiguity, changing externally observable contracts, deploying into production or performing an irreversible operation. The objective is not maximum autonomy. It is **maximum safe autonomy within explicit boundaries**. That distinction becomes increasingly important as agents become more capable. ## From human-in-the-loop to human-on-the-loop Human-in-the-loop governance assumes the human participates directly in the execution path. For some decisions, that is exactly where the human belongs. For every decision, it does not scale. Human-on-the-loop governance changes the role. The human defines authority and the system operates within that authority. Evidence is continuously produced. Exceptions escalate. Consequence boundaries trigger intervention. The human retains the ability to inspect, constrain, override or stop the system. The human moves from being the approval mechanism to being the **authority over the mechanism**. That is a much stronger role. It also requires substantially better engineering. You cannot safely remove humans from routine execution unless the system can explain what happened without relying on the agent's own narrative. Actions need provenance, claims need evidence, decisions need traceability, authority needs explicit boundaries and failures need to be visible. The evidence itself must survive the execution that produced it. Without those properties, autonomy is simply trust. ## The agent should not mark its own homework A second control problem appears when the same system that performs an action is also responsible for deciding whether that action was correct. Generation and judgement are different responsibilities. A capable agent can produce a convincing implementation and a convincing explanation of why that implementation is correct. Neither establishes independent confidence. Important conclusions should therefore be challenged independently. A separate evaluator can test the evidence, look for contradictions, surface unresolved uncertainty and reject a result that does not meet the required threshold. This does not mean duplicating every action. It means separating execution from judgement where the consequence warrants it. The purpose of independent challenge is not to create another approval queue. It is to reduce the number of weakly supported conclusions that ever reach the human decision surface. ## Control should become more precise as AI becomes more capable There is a paradox emerging in AI engineering. The more capable our systems become, the more tempting it is either to remove controls entirely or surround every action with approval gates. Both approaches waste capability. Unbounded autonomy creates unacceptable exposure. Universal approval destroys the economics and speed that made autonomous execution valuable in the first place. The alternative is a control system capable of distinguishing between observation and intervention, confidence and evidence, reversible and irreversible actions, routine execution and consequential decisions, and the agent performing the work and the authority deciding whether that work is acceptable. That is the direction we are taking at Harten. Not humans removed from the system. Not humans trapped inside every execution loop. Humans governing systems capable of operating safely without requiring continuous human attention. Because if machines can execute at machine speed, governance eventually has to operate at machine scale. **The human cannot be the bottleneck.** But the human must remain the authority.