Scaling Hidden State Size in S4 Models with Growth Bound Matrix Regularization for WikiText-103 Inference Throughput

When using Large Language Models (LLMs) to support Knowledge Graph Engineering (KGE), one of the first indications when searching for an appropriate model is its size. According to the scaling laws, larger models typically show higher capabilities. However, in practice, resource costs are also an important factor and thus it makes sense to consider the ratio between model performance and costs. The LLM-KG-Bench framework enables the comparison of LLMs in the context of KGE tasks and assesses their capabilities of understanding and producing KGs and KG queries. Based on a dataset created in an

Research goal: To what extent does scaling the hidden state size of S4 models with Growth Bound Matrix regularization impact inference throughput on the WikiText-103 language modeling benchmark relative to fraternal dropout?

Autonomous synthesis report generated by SOVEREIGN Research Kernel. Tribunal consensus score: 8.1/10.