Assistance, Not Authority
Why automation must support — not replace — human judgment. The aviation principle that keeps the pilot in command.
The Core Argument
In aviation, the pilot is always in command. Automation is a tool, not a replacement. The pilot decides when to use it and when to override it.
This principle — assistance, not authority — is fundamental to aviation safety.
Automation can fly the plane, manage the engines, navigate the route, and even land the aircraft. But the pilot remains in command.
The pilot is the ultimate decision-maker. The pilot is accountable. The pilot is responsible.
This is the principle that must guide the Judgment Economy.
Why Assistance, Not Authority
Four reasons why the pilot must remain in command.
1. Automation has blind spots
Automation is designed for known scenarios. It cannot anticipate every situation. The pilot provides the judgment that automation lacks.
2. Automation cannot provide context
Automation sees data. The pilot sees the full situation — weather, terrain, passengers, mission, constraints. Context is essential to good decisions.
3. Automation cannot be accountable
Accountability is human. When something goes wrong, it is the pilot who is responsible. Automation cannot bear that responsibility.
4. Over-reliance creates risk
When pilots rely too much on automation, they lose skills. They become less capable of handling emergencies. The pilot must remain engaged and skilled.
The Acacia perspective:
In the Judgment Economy, AI must be assistance, not authority. The human remains in command — deciding which intelligence to trust, which recommendations to act on, and which risks to take.
How the Principle Works in Practice
The mechanisms that keep the pilot in command.
Clear role definition
The roles of pilot and automation are clearly defined. The pilot is the decision-maker. Automation is the tool. This clarity prevents confusion.
Override capability
The pilot can always override automation. This is not a design flaw — it is a safety feature. The pilot must be able to take control at any time.
Training and practice
Pilots are trained to handle automation failures. They practice manual flying. They rehearse emergencies. They maintain their skills.
Accountability systems
Decisions are recorded. Actions are documented. Accountability is built into the system. This ensures that the pilot's judgment can be reviewed and learned from.
Key insight:
The pilot-in-command principle is not about distrusting automation. It is about designing systems that preserve human judgment. It is about ensuring that the human is always capable, engaged, and accountable.
The Risk of Automation Authority
What happens when automation has too much authority.
Aviation has learned the hard way what happens when automation has too much authority.
Automation surprise
When automation does something unexpected, pilots can be confused or surprised. This can lead to errors or delayed responses.
Skill degradation
When automation handles routine tasks, pilots can lose manual flying skills. This can be dangerous when automation fails.
Automation bias
Pilots can trust automation too much. They may fail to question its recommendations or override it when necessary.
Accountability diffusion
When automation is involved, accountability can become unclear. Who is responsible for a failure? The pilot? The manufacturer? The software?
The Acacia perspective:
These risks are not unique to aviation. They apply equally to the Judgment Economy. When AI has too much authority, we risk automation surprise, skill degradation, automation bias, and accountability diffusion. We must design for these risks.
Applying the Principle to the Judgment Economy
What "assistance, not authority" means for AI and human judgment.
In the Judgment Economy, the same principle applies. AI must be assistance, not authority.
AI provides intelligence, not decisions
AI can generate analysis, recommendations, and options. But the human must decide which to trust and which to act on.
The human remains in command
The human is the ultimate decision-maker. They can override AI recommendations. They are accountable for the outcome.
Judgment is a skill that must be maintained
Just as pilots must practice manual flying, humans must practice judgment. They must stay engaged, critical, and capable.
Accountability is clear
In the Judgment Economy, accountability must be clear. Who decided what, based on what evidence, and why? This clarity is essential to trust.
The principle of "assistance, not authority" is not about limiting AI. It is about designing systems that preserve human judgment, accountability, and trust.
Reading path: