TISH: A Biologically Inspired Learnable Neural Network Activation

This paper introduces TISH (Tensormatics Inspired Stabilized Hybrid) — a biologically inspired, learnable activation function for deep neural networks. Unlike conventional fixed-form activations such as ReLU, GELU, and Mish, TISH incorporates three interpretable learnable parameters governing responsiveness (s), gradient floor (b), and curvature (a), enabling automatic and controllable gradient adaptation during training. The paper proposes a mode-based learnability framework (Super, Ultra, and Extreme) that enables systematic exploration of the stability–expressivity trade-off, and introduces the Controlled Learnability Principle — the observation that partial parameter learnability can outperform full learnability in terms of generalization. Through extensive experiments on CIFAR-10 with ResNet architectures evaluated across multiple random seeds, TISH achieves a mean test accuracy of 89.57%, significantly outperforming ReLU (88.01%), GELU (88.70%), and Mish (88.66%) with very large effect sizes (Cohen's d > 1.9). The paper further demonstrates emergent layer-wise gradient hierarchy and provides rigorous mathematical analysis including gradient bounds, Lipschitz continuity, and asymptotic behavior characterization.