On-Chain LLM Inference Under Instruction Budgets: An Instruction-Budget Cost Model, Ternary Floor Evidence, and Session Costs

LLM inference on Internet Computer Protocol (ICP) canisters runs under a hard per-call budget of B = 40×10⁹ instruction-cost units, not wall-clock time — the instruction-bounded regime. The same Qwen 2.5 0.5B Q8_0 model, on the same ICP mainnet canister, yields 10 tok/call on the community baseline and 29 tok/call on our fork: the cost coefficient is software-determined.

(R1) Decode follows tok/call ≈ B/(α_eff·2P), where α_eff is the effective cost coefficient at the published-parameter count — a deployment-stack property computable from a model card. Across 11 modern decoder models from 8 families, α_eff = 1.53 (LOAO MAPE 7.7%). Under the executed-FLOPs convention the physical coefficient α_exec clusters at 1.54 (CV 9.3%) for 9 of 11 models. (R2) Matmul accounts for ≈98.7% of decode cost; per-element unstructured sparsity is provably counter-productive on the measured integer paths. (R3) A purpose-built TQ2_0 ternary WASM SIMD path on purpose-trained models decodes at the empirical floor of the measured decode path (α_eff = 1.03; α_exec = 1.17), with byte-identical outputs across a local replica, 13-replica Swiss Subnet consensus, and a standard ICP mainnet subnet — across two build generations and two independent weight-conversion pipelines.

Secondary results: uniform Q8_0 stays Pareto-non-dominated under instruction metering; multi-call sessions pay a measured two-term IO tax (load ≈ 43.0 M + 0.177 M·n_kv on the current build); an on-chain encoder case study unifies with the decode cluster under the executed-FLOPs convention.

The companion artifact (this record's ZIP; also at github.com/Simlowker/instruction-bounded-inference-artifact) contains the full measurement registry, raw on-chain logs, analysis scripts, and the end-to-end reproduction procedure for the headline result (~30 min, no cycles required). Every load-bearing numeric claim resolves to a named CSV.