Published June 4, 2026 | Version v1.2.0
Software Open

terrizoaguimor/tinymars: v1.2.0 — Native (from-scratch) result + paper

Authors/Creators

  • 1. Celiums Solutions LLC

Description

The from-scratch experiment: a 110M decoder born with proprioceptive channels from layer 1.

  • Perpendicular force replicates from scratch — the channel decides the preferred output on 88.8% of 455 held-out counterfactual pairs (chance 25%).
  • Relief valve (new) — with channels zeroed, the native predicts held-out targets better than two channel-less baselines that bracket its parameter count (4.825 vs 5.252 / 5.253 nats), attributing the gain to the channel pathway, not size.
  • Paper: docs/paper/proprioceptive-channels.{md,tex,pdf} — two experiments (adapter + native), 15 verified citations.
  • New: training/native/, eval/native/native_eval.py.

Honest scope: toy scale (110M / 1B tokens), logprob on in-distribution held-out data, one iteration. Built on nanochat (A. Karpathy, MIT).

Files

terrizoaguimor/tinymars-v1.2.0.zip

Files (2.5 MB)

Name Size Download all
md5:7507c68fd9f485c298419e3c1d7a41ea
2.5 MB Preview Download

Additional details

Related works