AXIOM: Adaptive eXpressive Intelligence with Optimized Message-passing — A New Architecture for Efficient, Semantically-Aware Language Modeling at Linear Complexity
Description
We present THX-AXIOM (AXIOM: Adaptive eXpressive Intelligence with Optimized Message-passing), a novel neural network architecture for language modeling that addresses the fundamental limitations of transformer-based models. Our architecture introduces five key innovations: (1) a triple embedding system combining concept, semantic type, and position embeddings; (2) sparse graph neural network layers achieving O(|E|) complexity instead of O(n²); (3) concept-based tokenization where tokens carry inherent semantic meaning; (4) self-organizing topology that adapts computation to input complexity; and (5) built-in uncertainty quantification for confidence estimation. With only 30.5M parameters, THX-AXIOM reduces computational complexity by up to 839× for long sequences. A 327M-parameter prototype sharing the same GNN backbone and energy-based objective achieved consistent training convergence (94.3% cross-entropy loss reduction over 200,000 steps), validating the architectural approach on consumer hardware.
Files
Adaptive eXpressive Intelligence with Optimized Message-passing.pdf
Files
(290.3 kB)
| Name | Size | Download all |
|---|---|---|
|
md5:e13ce5f0c9204ae25217e6b9e9e1ec4d
|
290.3 kB | Preview Download |