Published August 7, 2025
| Version v1.0 (Preprint)
Preprint
Open
GD‑Attention: A Nonlinear Selection Mechanism with Unique Coherence\ Guarantees — Grounded in the Structure of Meaning
Creators
Description
This paper introduces GD-Attention, a nonlinear selection mechanism derived from Ghost Drift theory. Unlike traditional Softmax attention that blends values probabilistically, GD-Attention deterministically selects a single coherent key via energy minimization. We mathematically prove the uniqueness of this selection based on a semantic energy landscape with strong convexity guarantees. This framework offers a new paradigm for attention mechanisms, emphasizing semantic integrity, non-additivity, and interpretability.
Files
Simulated_AI_Reviews.pdf
Files
(1.4 MB)
| Name | Size | Download all |
|---|---|---|
|
md5:c16c9d62125c8a88d6844a7d3ab116f8
|
698.4 kB | Preview Download |
|
md5:ac9b0799490bde9849e106d942dbde93
|
722.6 kB | Preview Download |
Additional details
Related works
- Is supplement to
- Preprint: 10.5281/zenodo.16751929 (DOI)
References
- A.~Vaswani, N.~Shazeer, N.~Parmar, J.~Uszkoreit, L.~Jones, A.~N. Gomez, L.~Kaiser and I.~Polosukhin. ``Attention is All You Need.'' \emph{Advances in Neural Information Processing Systems}, 30 (2017).
- T.~Mikolov, K.~Chen, G.~Corrado and J.~Dean. ``Efficient Estimation of Word Representations in Vector Space.'' \emph{ICLR Workshop Track Proceedings} (2013).
- Y.~LeCun, S.~Chopra, R.~Hadsell, M.~Ranzato and F.~J.~Huang. ``A Tutorial on Energy‑Based Learning.'' In \emph{Predicting Structured Outputs}, 2006.