There is a newer version of the record available.

Published October 31, 2026 | Version 2.0

Logical AI OS, Validation Layer: Effect of a Zero-Inference External Mechanical Layer on Constraint Following — Bounded Results from Synthetic Rounds and Two Blind Public Benchmarks

Authors/Creators

  • 1. Independent Researcher

Description

A zero-inference external mechanical layer reads shape constraints (length, item count, format, paragraph
structure, required/forbidden terms, person, language, character set, relative length) from a user request,
checks the answer against them, and repairs violations both after generation and during generation; the
implementation of both actions is withheld. Under frozen rules, unseen items and rule-blind gold, layer v1.8
raises per-prompt full compliance of a 3B model by +14.5 points (39.3→53.8%) and of a 7B model by +17.1
points (51.3→68.4%) on a synthetic constraint-dense set (S3). Layer v1.11, shot blind on the rule-only subset of
the public ComplexBench benchmark and scored with its rule checker on whole responses, gives +3.2/+5.1
points at instruction level and +5.7/+7.1 at prompt level (mappable subset: +7.4/+10.1); these subset scores
are not comparable with the benchmark’s official scores. Paired tests on per-item outcomes (McNemar;
bootstrap 95% CI) make the 7B gains significant on both sets (S3 p<0.001; ComplexBench prompt-level
p=0.008) and the 3B gain significant on S3 (p=0.001) but only borderline on ComplexBench (prompt-level
p=0.08; instruction-level p=0.33). On real Chinese traffic (WildChat) the effect is near zero because explicitly
constrained requests are rare there; the LMSYS-Chat-1M Chinese subset yielded 2 usable items and was not
built into a round. The first public shot (Multi-IF zh) was slightly negative (−1.8/−1.3; not distinguishable from
noise, p=0.10/0.21) and is diagnosed as reading coverage: the extractor matched 34 of 362 mappable
constraints (type and parameter), fixed in-sample to 320/362 and out-of-sample to 145/252 on ComplexBench.
Claims are bounded: feasibility is established; effectiveness is established within the shape layer; breadth is
not. Control-layer artifacts (all action implementations, rule tables and thresholds) are withheld by design; this
report publishes effects, protocol, coverage, a failure taxonomy and boundaries.

Control-layer artifacts are withheld by design (see Sections 2 and 12).

Abstract (Chinese)

一個零推論的外部機械層,能把使用者請求中對答案的形狀要求(長度、條數、格式、段落結構、必含/禁含詞、人稱、語言、字元集、相對長度)讀出來、核出來,並在生成後與生成時補回去(兩種動作的實作皆不公開)。在規則凍結、題目未見、答案本盲於規則的條件下,架構 v1.8 使 3B 模型的逐篇全守率 +14.5 點(39.3→53.8%)、7B +17.1 點(51.3→68.4%,合成卷 S3);架構 v1.11 在公開題庫 ComplexBench 純規則子集上盲打、以其規則檢查器評整篇回覆,3B 條過率 +3.2、7B +5.1,題全過率 +5.7/+7.1(可映射子集 +7.4/+10.1);子集分數不可與該題庫官方分數比較。逐題配對檢定(McNemar;bootstrap 95% 區間):7B 在兩個題集上皆顯著(S3 p<0.001;ComplexBench 題全過 p=0.008),3B 在 S3 顯著(p=0.001)、在 ComplexBench 僅邊緣(題全過 p=0.08;條過 p=0.33)。真實中文流量(WildChat)上效應近零,因為限制明寫的請求極稀;LMSYS-Chat-1M 中文子集篩後僅 2 篇,未建卷。第一發公開題庫(Multi-IF zh)小負(−1.8/−1.3,與噪音無法區分,p=0.10/0.21),診斷為讀題覆蓋:抽取器只對上 362 條可映射限制中的 34 條(型與參數),修後樣本內 320/362、樣本外(ComplexBench)145/252。主張有界:可行性證實;有效性在形狀層有界證實;廣度未證。控制層實作(全部動作實作、規則表與門檻)依設計不公開;本報告公開效應、協定、覆蓋率、失敗型錄與邊界。

控制層實作依設計不公開(見第 2 節與第 12 節)。

Files

report_v2_1_part2_validation_layer.pdf

Files (581.0 kB)

Name Size
md5:cda7f6ab0bde989e40d86a3942f2ee10
581.0 kB Preview Download

Additional details

Related works

Continues
Technical note: 10.5281/zenodo.23226740 (DOI)
Is supplemented by
Software: https://github.com/blackwing04/Logical-AI-OS/releases/tag/v2.0-part2 (URL)
References
Preprint: arXiv:2410.06458 (arXiv)