An AI Judge Said "Yes," But Was It Allowed to Decide? Putting Clear Limits on the Systems That Approve What AI Agents Do

What this is about, in plain words

AI systems increasingly check other AI systems. A “judge” model reviews an agent’s plan, a policy engine scores a risk, a human reviewer signs off, or an independent service confirms that a job was really completed. This disclosure asks a question most designs skip: who made that evaluator an evaluator, what is it allowed to look at and decide, how long does that authority last, and what does its verdict actually permit?

Its central rule is that an evaluator’s approval is never, by itself, permission to act. A positive, signed, quorum-backed or “trusted” verdict cannot by itself move money, release data, change production systems, use credentials or actuate equipment. Any real-world effect still has to pass its own authorisation and an exact check at the point where it happens.

Around that rule, the disclosure covers the evaluator’s whole lifecycle:

• What an evaluator can be: a rule engine, AI model, simulation, human reviewer, panel, firmware or a hybrid. It can run anywhere: same device, separate host, cloud, edge, TEE or third party.
• Three deployment forms:
  • ephemeral: created for one task and then destroyed;
  • persistent: a long-lived service with separate per-project mandates;
  • hybrid: a persistent host that spins up short-lived evaluation contexts.
• Protected appointment: an evaluator becomes one only when the real authority admits it. It cannot appoint itself, claim neutrality, or be chosen by the agent it is judging.
• Bounded powers: what evidence it may see, which outputs it may give (approve-for-consideration, deny, hold, request evidence, dispute, unknown), and what it may never do. For example, it may not treat a timeout as proof that something didn’t happen, count two accounts from one group as independent reviewers, or turn a simulation into outcome truth.
• Independence and neutrality: defined by actual control, keys, conflict-of-interest rules and isolation, not by physical distance or separate company names.
• Lifetime vs. validity: a decision can go stale before the appointment expires. Destroying a runtime doesn’t make its old verdicts current, and losing connectivity never extends authority.

The 27 embodiments, grouped

1. Multi-agent negotiation and freeze (E01–E03): protected joint-plan commitment between agents; cryptographically enforced communication freeze and distributed quiescence before evaluation; negotiation-state integrity and containment of messages still in flight.
2. Consequence evaluation (E04–E06): an independent consequence evaluator with protected evidence admission; isolated “what-if” reconstruction in a non-effective sandbox; risk, benefit–loss and tail-exposure scoring with per-domain thresholds.
3. Evaluator-directed control (E07–E10): choosing the next actor and giving controlled next-turn instructions; corrective instructions and renegotiation; reconfiguring agent roles and bounded delegation; resuming communication for one turn, then automatically re-freezing.
4. Gating the real effect (E11–E12): a single-use, actor-bound “next-move passport”; finality-sink enforcement before communication, privileged execution, external dispatch or physical actuation.
5. Substitution and continuity (E13–E16): risk-triggered neutral substitute agents; escrow and migration of pending work; phased execution with protected completion receipts; verified actual-outcome feedback before continuing.
6. Integrity and resilience (E17–E19, E24): collusion-resistant review and source provenance; network partitions, unknown outcomes and durable fencing; session closure with obligation reconciliation and irreversible retirement of authority; self-expiring supervisor and evaluator authority with generation-fenced replacement.
7. Cross-organization governance (E20–E23, E25–E27):
   • mandatory approvers and authenticated hosting;
   • cross-domain evaluator appointment and dispute containment;
   • trust formation and exact policy compatibility;
   • leaderless coordination between agents from different companies;
   • dynamic membership with threshold governance;
   • minimal-disclosure outcome evidence (prove the needed fact without exposing the whole record);
   • multi-domain execution finality with partial-outcome reconciliation and recovery.

Many embodiments include mathematical models, pseudocode, decision flowcharts, edge-case tables and runnable Python reference implementations. The document is careful to call these illustrative and not proof of production security.

Industry relevance

AI-generated section. This industry relevance assessment was generated with AI assistance and is not part of the patent filing. Readers are strongly advised to cross-check it with the original filings and with each organisation’s own current public documentation.

• AI labs and agent platforms: “LLM-as-judge”, critic and guardrail models, and multi-agent orchestration frameworks all use evaluators. This work defines when such a judge is legitimately appointed, what it may see, and why its verdict still isn’t execution authority. That matters for any vendor building agent platforms where one model approves another’s actions.
• Cross-company agent ecosystems: agent-to-agent protocols, agent marketplaces, and procurement or supply-chain agents negotiating across firms. Embodiments 20–27 address who governs evaluation when no single company is in charge.
• Payments and commerce: procurement approvals, invoice and delivery verification, and dispute handling. An approved order still needs buyer-side payment authorisation.
• Cloud and IT operations: change-approval boards, automated SRE remediation and incident review. A reviewer’s “approve” can’t silently become a production write.
• Telecom and industrial: maintenance completion, actuator proposals with device-local safety evaluators, and partitioned networks where outcomes are unknown.
• Regulation and audit: human-oversight and independent-review duties under AI laws such as the EU AI Act, plus conformity and third-party assessment. The disclosure gives a technical meaning to “independent”, “appointed” and “current” reviewers.