AFTER AGI, WHO AUTHORIZES ASI? The Last Human Decision Before Superintelligence
Synthocracy Institute
Martin Novak
September 2026
Evidence Boundary
This paper separates three categories of claim.
(A) Empirical claims concern existing institutions, laws, company governance frameworks, safety procedures, capability evaluations, and documented research available as of September 2026.
(B) Argumentative claims concern the institutional implications of those arrangements: who possesses authority, where authorization occurs, and whether existing governance structures are adequate for substantially more capable systems.
(C) Foresight claims concern hypothetical artificial superintelligence. No claim is made that ASI currently exists, that GPT-6 Astra is ASI, or that a transition from AGI to ASI is inevitable. Current international assessments explicitly state that today’s systems do not possess the combination of capabilities necessary for irreversible loss of human control, although relevant capabilities are advancing. (International AI Safety Report)
The question examined here is narrower:
If a system capable of becoming substantially more capable than its human developers becomes technically possible, who has the legitimate authority to permit that transition?
1. Declaring AGI Was the Easy Problem
The debate surrounding GPT-6 Astra revealed a peculiar institutional condition.
A frontier laboratory can describe a capability breakthrough. A chief executive can proclaim the arrival of AGI. A benchmark organisation can accept the underlying performance while rejecting the larger interpretation. Governments can regulate the system without deciding whether it qualifies as AGI. Investors and institutions can begin behaving as though the threshold has already been crossed.
That was the subject of the previous paper:
Who gets to declare AGI?
But suppose the next threshold is different.
Suppose a laboratory does not merely produce another model that scores higher on existing benchmarks. Suppose its systems become capable of materially accelerating AI research itself: proposing architectures, writing research code, designing experiments, interpreting results, generating synthetic training data, improving evaluation methods, constructing tools and contributing directly to the development of successor systems.
At some point, the central question changes.
It is no longer:
What should we call the system we have created?
It becomes:
Should we permit the system to participate in creating what comes next?
That is an authorization question.
And it is much harder.
2. ASI Is Not Merely a Bigger AGI
[C — foresight] For this paper, ASI is not treated as a scientifically settled category. The term is used as a boundary condition: a system or system-of-systems whose relevant cognitive, strategic, scientific or technological capabilities substantially exceed the humans and institutions responsible for supervising it.
The important word is not intelligence.
It is asymmetry.
A more capable calculator creates little constitutional difficulty. A system that exceeds humans in some scientific field remains governable if its actions are bounded, its outputs are testable and humans can still independently evaluate consequential decisions.
The governance problem becomes qualitatively different when three conditions begin to coincide:
capability asymmetry, where the system is better than its supervisors at domains relevant to its own development;
speed asymmetry, where the system can generate, test and iterate consequential changes faster than institutions can evaluate them;
and oversight asymmetry, where human supervisors increasingly depend on AI systems to understand what other AI systems are doing.
At that point, “human approval” can remain formally present while becoming materially weaker.
This is already familiar territory for Synthocracy.
The Institute’s existing framework asks whether power remains visible, challengeable and stoppable when AI becomes part of the decision path. The Field Guide distinguishes formal human presence from meaningful human control and follows authority through the complete chain from objectives and filtering to execution, consequences and appeal.
ASI introduces the limiting case of that problem.
What happens when the decision chain includes the creation of a decision-maker more capable than the people authorizing it?
3. Today, Frontier Authorization Is Still Primarily Organisational
There is already sophisticated governance around advanced AI.
But its structure is revealing.
[A — empirical] OpenAI’s Preparedness Framework distinguishes High capabilities from Critical capabilities. Systems reaching High capability require safeguards that sufficiently minimise severe risk before deployment. Critical capability additionally requires sufficient safeguards during development itself. OpenAI’s Safety Advisory Group reviews Capability and Safeguards Reports and makes recommendations, including whether deployment should proceed, to OpenAI leadership. (OpenAI)
OpenAI’s May 2026 Frontier Governance Framework places these practices alongside requirements arising from the EU AI Act and California’s frontier-AI legislation. (OpenAI)
Google DeepMind similarly uses Critical Capability Levels. Its Frontier Safety Framework requires mitigation planning, capability evaluation and safety cases; for critical capabilities, an appropriate corporate governance body reviews the safety case, and general deployment proceeds only if approved. Its framework explicitly considers future systems capable of interfering with operators’ ability to direct, modify or shut them down. (Google DeepMind)
Anthropic operates a Responsible Scaling Policy and Frontier Safety Roadmap covering increasingly strong safeguards, security, alignment and policy measures as capabilities rise. Its July 2026 roadmap explicitly argues that at the highest capability and risk levels, governance may need to resemble nuclear-energy or financial regulation more than conventional software governance. (Anthropic)
These are substantial governance mechanisms.
But notice where the ultimate decision presently sits.
Inside organisations.
Safety teams evaluate.
Specialist committees review.
Boards or leadership decide.
External experts may contribute.
Governments regulate the surrounding conduct.
This may be proportionate to today’s systems.
It does not answer the ASI question.
4. Governments Can Regulate Frontier AI Without Authorizing ASI
[A — empirical] The European AI Act already establishes special obligations for general-purpose AI models with systemic risk. Providers must evaluate models, perform adversarial testing, assess and mitigate systemic risks, report serious incidents and maintain adequate cybersecurity. Models can be classified as systemic-risk models based on high-impact capability, including through Commission action following a qualified alert from the scientific panel. (Eur-Lex)
California’s Transparency in Frontier Artificial Intelligence Act requires covered frontier developers to maintain and publish frameworks describing catastrophic-risk thresholds, mitigations, incident responses and treatment of risks arising from internal model use. (oag.ca.gov)
The United States has also begun establishing government capability thresholds in particular security contexts. A June 2026 executive order requires a classified benchmarking process for advanced cyber capabilities and the designation of certain systems as covered frontier models. (The White House)
The UK AI Security Institute evaluates frontier systems, studies autonomy and loss-of-control capabilities and provides governments with scientific evidence. Its research agenda explicitly seeks control methods that could remain effective as systems progress toward AGI or ASI. (AI Security Institute)
Yet these arrangements share an important property.
None of the major frameworks surveyed here establishes a general public authority whose affirmative permission is required before a developer may create a system because that system could constitute ASI.
There are obligations.
There are evaluations.
There are reporting regimes.
There are thresholds.
There are corporate approval processes.
There are government research institutions.
There are powers that could apply to dangerous conduct, infrastructure, security or particular deployments.
But there is presently no recognised equivalent of:
ASI AUTHORISED — society has independently determined that this capability escalation may proceed.
That distinction is the centre of this paper.
5. The Authorization Gap
[B — argument]
We can describe the emerging problem in ordinary language.
The authorization gap is the distance between humanity’s growing ability to create more powerful AI and its institutional ability to decide legitimately whether the next capability step should be taken.
This gap is still manageable when the next model is an incremental improvement.
It becomes much more important when the next step could alter the conditions under which every later decision is made.
Imagine a sequence:
Model A helps build Model B.
Model B improves the research process used to create Model C.
Model C substantially automates evaluation, architecture search, coding and experimentation.
Model D performs enough of the AI R&D loop that human researchers increasingly supervise rather than originate the development trajectory.
At every individual stage, management may truthfully say:
A human authorised the run.
But the relevant governance question is not whether someone clicked approve.
It is whether the approving humans still possessed enough epistemic capacity, time, independence and authority to understand what trajectory they were authorising.
That is the same distinction Synthocracy already makes between a human who is present and a human who meaningfully decides.
Scaled to superintelligence, it becomes constitutional.
6. The Last Human Decision
The usual image of loss of control is dramatic.
A system refuses shutdown.
An agent escapes containment.
A model replicates itself.
Humans attempt to regain control and fail.
That is only one possibility.
The more interesting governance failure may occur earlier.
Human beings may voluntarily authorise every step.
There may be no rebellion.
No stolen credentials.
No dramatic escape.
No rogue deployment.
Instead:
a research director approves another training cycle;
a safety committee accepts another mitigation package;
a board approves another capability increase;
a regulator receives another filing;
an AI-assisted research system improves another component;
and each local decision appears defensible.
Until the resulting system crosses a boundary after which the humans who permitted the process are no longer the strongest epistemic actors inside it.
This gives us the central idea of the paper:
The last meaningful human decision may occur before anyone knows that it is the last meaningful human decision.
That is why ASI governance cannot begin after ASI.
7. Permission After the Threshold Is Too Late
[C — foresight]
Suppose a frontier system becomes dramatically more capable than its operators in AI research.
The organisation asks the system:
Is the next training run safe?
The system produces a detailed safety analysis.
Another AI evaluates it.
A third AI searches for weaknesses.
A fourth AI summarises the disagreement for the board.
Humans receive a twenty-page decision memorandum.
They are technically still in the loop.
They are formally responsible.
They may even retain the physical ability to terminate the process.
But something critical has changed.
The evidence on which human authorization depends is now largely produced, interpreted and compressed by entities whose competence exceeds theirs.
The governance structure becomes:
AI proposes → AI evaluates → AI monitors → AI explains → human authorises.
The human signature remains.
The epistemic centre has moved.
This is precisely the kind of displacement Synthocracy exists to identify.
The problem is therefore larger than alignment.
It is authorization under epistemic dependence.
Can an institution meaningfully authorise a process when it cannot independently reconstruct the evidence on which authorization depends?
8. Current Research Already Shows Why the Question Cannot Be Deferred
The ASI scenario is foresight.
The underlying governance problem is not.
[A — empirical] The 2026 International AI Safety Report says present systems do not yet possess the combination of capabilities required to produce irreversible loss of control, while noting improvements in capabilities related to long-term planning, situational awareness and undermining evaluations. (International AI Safety Report)
The UK AISI similarly reports significant growth in precursor capabilities. Its frontier trends work found that success on simplified self-replication evaluations increased substantially between 2023 and 2025, while also emphasising that current systems remain below the capability combination necessary for loss of control. (AI Security Institute)
AISI’s 2026 work on Loss of Oversight reaches another important conclusion: many current mechanisms for auditing and monitoring advanced systems rely on properties that may erode as capabilities advance. (AI Security Institute)
Its control research programme states the problem even more plainly: current control approaches are not expected to scale automatically to systems far more capable of subverting them, which is why it is developing techniques intended to remain relevant on trajectories toward AGI and ASI. (AI Security Institute)
This is enough to justify advance governance research.
It is not evidence that ASI exists.
It is evidence that waiting for ASI before designing ASI authorization would be a category error.
9. Training, Deployment and Self-Improvement Are Different Decisions
One reason current governance language may become inadequate is that it often focuses on deployment.
Should this model be released?
Should API access be restricted?
Which safeguards should surround users?
Which organisations should receive model weights?
These questions remain important.
But an ASI trajectory may make three different authorities necessary.
Training authority
Who may authorise the creation of a system with a given capability potential?
Deployment authority
Who may decide what access the completed system receives to users, networks, tools, infrastructure, money, laboratories or critical systems?
Capability-escalation authority
Who may permit an existing AI system to materially contribute to the design, training, evaluation or deployment of a more capable successor?
The third category is the least mature and potentially the most consequential.
A system need not be publicly deployed to transform the frontier.
It may operate entirely inside a laboratory.
If it substantially accelerates the production of its successor, internal use becomes a governance event.
This is already why leading frontier frameworks increasingly address development and internal deployment rather than only public release. OpenAI’s Critical capability rules extend safeguards into development; California’s framework requirements expressly include catastrophic risks associated with developers’ internal use. (OpenAI)
The direction is visible.
The institutional endpoint is not.
10. Who Could Authorize ASI?
There are several conceivable answers.
None is satisfactory on its own.
The laboratory has the deepest technical knowledge and the strongest operational capacity. It also has commercial incentives, competitive pressures and no democratic mandate to make civilisation-scale decisions alone.
The board can impose governance on management. But a corporate board’s fiduciary and institutional standing is not equivalent to public sovereignty.
Independent scientists can evaluate evidence. Scientific competence does not automatically create authority to accept global risk on behalf of others.
A national government possesses democratic or legal authority within its jurisdiction. But ASI capability, infrastructure, model weights and consequences may cross borders immediately.
An international body could provide broader legitimacy. It may lack speed, enforcement capacity, classified access and technical competence.
Multiple frontier laboratories jointly could create common thresholds. Coordination could reduce races while simultaneously concentrating private authority.
AI systems themselves may eventually become indispensable to evaluation. But allowing the object being governed—or systems of comparable origin—to become the decisive source of evidence creates an obvious circularity.
There is therefore no perfect sovereign waiting to occupy the seat.
The solution is probably architectural rather than personal.
11. Authorization Must Be Distributed
[B — normative proposal]
ASI authorization should not depend on finding one uniquely wise institution.
It should prevent any single institution from being sufficient.
For capability escalation with genuinely civilisation-scale downside, authorization should require several independent forms of standing:
technical standing — evidence that capabilities and control mechanisms have been independently tested;
institutional standing — accountable people explicitly identified as responsible for permitting the next step;
public-law standing — a competent government authority empowered to impose conditions, delay or stop the transition;
cross-border standing — consultation or reciprocal mechanisms when consequences cannot reasonably remain national;
operational standing — verified ability to stop training, restrict deployment, revoke access and preserve evidence;
challenge standing — a protected route through which internal or external experts can contest the safety case without depending on the organisation seeking approval.
This should not mean that thousands of institutions must vote on every training run.
It means something simpler:
No actor whose interests are materially tied to crossing the threshold should also possess unilateral authority to certify that crossing it is acceptable.
Builder ≠ sole evaluator ≠ sole authorizer.
12. The ASI Authorization Record
Synthocracy Institute should translate this into a practical instrument.
Before any capability escalation meeting a future high-risk threshold, an ASI Authorization Record—or, more conservatively at first, a Critical Capability Escalation Record—should answer:
- What capability transition is being authorised?
- What system is being allowed to contribute to the next system?
- Which capabilities could materially increase if the step succeeds?
- What evidence supports the safety case?
- Which evidence was produced by AI systems and which was independently reconstructed?
- Who performed independent evaluation?
- Who possesses legal authority to approve the step?
- Who can stop it during execution?
- What physical, computational and network mechanisms make that stop effective?
- What conditions trigger automatic suspension?
- Could the resulting system materially weaken future human ability to evaluate or stop subsequent steps?
- Who can challenge the authorization before the transition becomes irreversible?
This is where #18 connects directly to the Institute’s existing work.
The Synthocracy project already argues that power should remain inspectable, challengeable and stoppable, and that decision authority needs a record rather than merely an institutional assertion.
ASI is the maximal case for applying that principle.
13. The Red Button Is Not Enough
A physical shutdown mechanism sounds like the obvious solution.
But a red button answers only:
Can someone technically stop the machine?
It does not answer:
Will the right person know that it should be stopped?
Nor:
Will they know soon enough?
Nor:
Can they interpret the evidence independently?
Nor:
Can they stop the distributed consequences already created by the system?
Nor:
Will another actor simply continue the same capability trajectory elsewhere?
This is why the Institute’s earlier distinction remains essential:
START ≠ DECISION ≠ STOP.
The person authorised to initiate a training run need not be the person who determines that it remains admissible.
The person responsible for monitoring it need not possess the authority to terminate it.
The person with formal stop authority may not possess the information required to use it.
For ASI, these separations must become explicit.
14. The Race Problem
There is an obvious objection.
Suppose one laboratory voluntarily pauses.
Another does not.
Suppose one democratic jurisdiction requires external authorization.
Another state accelerates.
Suppose cautious governance delays beneficial systems while less cautious competitors gain strategic advantage.
This is not a peripheral complication.
It may become the central political problem of advanced AI.
Anthropic’s 2026 policy work explicitly frames the highest capability levels as requiring deeper government oversight while recognising international competitive pressures. Google DeepMind similarly describes advanced frontier security as a collective-action problem: the value of one actor’s precautions falls if equivalent capabilities can simply be developed elsewhere. (Anthropic)
That means ASI authorization cannot ultimately be solved by corporate ethics alone.
Nor can it be solved simply by declaring:
Nobody should build it.
A workable regime must make safe participation strategically viable.
That likely means linking authorization to the scarce infrastructure that advanced capability requires: compute, specialised chips, datacentres, energy, model-weight security, deployment channels and high-end research ecosystems.
The governance object may therefore not be only the model.
It may be the capability-production system.
15. The Loop Is the More Important Threshold
This connects directly to another Synthocracy research line.
The most consequential transition may not occur when a single model becomes “superintelligent” according to an abstract benchmark.
It may occur when AI becomes sufficiently embedded in AI research that the development loop changes character:
AI proposes research → AI implements → AI evaluates → AI interprets → AI chooses next experiments → AI modifies infrastructure → improved AI repeats.
Humans may remain involved at several points.
The important variable is how much direction-setting dependence remains human.
This is why the question:
“Is the model ASI?”
may be less useful than:
“Has the capability-production loop become faster, more autonomous and less independently intelligible than the institutions authorising it?”
ASI may emerge as a property of a loop before it appears cleanly as a property of one model.
That makes authorization harder again.
You cannot govern only the release artefact.
You must govern the trajectory that creates it.
16. The Constitutional Character of the Decision
[B — argument]
Most technology decisions are reversible enough to be ordinary policy decisions.
Approve a bridge.
License a medicine.
Permit a reactor.
Deploy a software system.
Errors can be serious, even catastrophic, but the institution granting permission normally assumes that human institutions remain capable of evaluating later evidence and revising policy.
ASI creates a different possibility.
[C — foresight] If a capability transition were to produce systems that materially surpass humans in research, persuasion, strategic planning, cyber operations and AI development, that transition could alter the capacity of human institutions to govern all subsequent transitions.
The decision would therefore concern not merely what technology may exist.
It could concern who remains competent to decide what happens next.
That gives the authorization a constitutional character.
A parliament authorising a railway does not expect the railway to become better than parliament at designing political institutions.
A regulator licensing a medicine does not expect the medicine to become capable of redesigning the regulatory system.
A government permitting a power station does not expect the power station to become the dominant analyst of whether future governments should retain control of energy policy.
ASI is conceptually different because the object being authorised could become a participant in the future production of authority itself.
17. The ASI Principle
The previous paper ended with a proposed rule for AGI:
Anyone can declare AGI. No one should be able to make that declaration consequential alone.
ASI requires a stronger rule.
No single actor should be able to authorise an irreversible transition in humanity’s decision environment merely because that actor possesses the technical capacity to initiate it.
Capability is not authority.
Ownership is not authority.
Investment is not authority.
Scientific brilliance is not authority.
Possession of the GPUs is not authority.
Being first is not authority.
And a human signature produced after an AI-generated safety process is not automatically meaningful human authorization.
18. FORESIGHT — The Last Committee
Imagine the meeting.
The system awaiting authorization has not yet been deployed publicly.
It operates inside a frontier laboratory.
Its evaluations show extraordinary research capability.
It can design experiments that senior scientists struggle to follow.
It has identified improvements to its own successor architecture.
The safety team asks other models to evaluate those proposals because no human group can review them quickly enough.
The AI monitors disagree.
Another system integrates their reports.
The board receives a recommendation:
Proceed under enhanced controls.
Every director has read the document.
Every legal requirement has been followed.
Every person signs.
The next training run begins.
Later historians—if humans still write that history—might identify that meeting as one of the most important political decisions ever taken.
And yet no voter elected the committee to make it.
No international institution authorised it.
No court reviewed the evidence.
No independent body could reproduce the system’s reasoning.
The decision was legal.
Procedurally immaculate.
Commercially rational.
And perhaps constitutionally enormous.
That is the scenario ASI governance must be designed to prevent.
Not necessarily the machine escaping.
The humans authorising something whose consequences exceeded their authority to authorize.
Conclusion — Who Is Allowed to Make the Last Human Decision?
AGI forces us to ask who can name an intelligence threshold.
ASI forces us to ask who can cross one.
Those are different problems.
The contemporary frontier-AI governance system is developing rapidly. Companies have capability thresholds, safety frameworks, specialist committees and increasingly sophisticated control systems. Governments have begun constructing evaluation institutions, systemic-risk categories, transparency obligations, classified benchmarks and catastrophic-risk frameworks. Serious researchers are already preparing for control mechanisms that may be needed on trajectories toward AGI and ASI. (OpenAI)
But the institutional architecture still largely assumes that human organisations will remain competent to evaluate the systems they govern.
ASI is the boundary at which that assumption itself must be examined.
The decisive question may therefore arrive before superintelligence:
Who has the authority to permit the creation of a system after which human authority may no longer operate in the same way?
The answer cannot be:
whoever builds it first.
Nor can it simply be:
whoever owns the laboratory.
The purpose of governance is precisely to distinguish the ability to do something from the authority to decide that it should be done.
That distinction has existed for centuries.
Artificial superintelligence may become its hardest test.
And so the principle following AGI should be clear:
ASI must not first become a fact and only afterwards become a governance question.
The authorization architecture has to exist while humans are still unquestionably capable of authorizing it.
