Published June 6, 2026 | Version 1.0
Preprint Open

Semantic Attention(SMAT): Attention Gated Through Learned Similarity and Hidden State Centrality

Authors/Creators

  • 1. independent Researcher

Description

SMAT adds a learned semantic similarity matrix and a centrality-derived value gate to transformer attention. The gate is load-bearing, removing it collapses performance. Semantic information acts through gating, not direct attention biasing.

Files

SMAT RESEARCH PAPER.pdf

Files (561.6 kB)

Name Size Download all
md5:60097ae2ced0b057b52083355e96b079
561.6 kB Preview Download

Additional details

Software

Repository URL
https://github.com/OutrageouslyBad200/SMATest
Programming language
Python
Development Status
Active