On The Ontology of Individuated Neurons: Symmetry-Reformulations Enable Dynamic Topologies

This paper primarily leverages a new symmetry-principled class of primitives to demonstrate architectures that can grow and shrink in real-time with task demand. This is made possible by structural changes, connected by symmetry, which leave the computation invariant under neurogenesis and nearly identical under neurodegenesis. Using such `isotropic' primitives, the notion of an individuated neuron is lost, allowing freedom in how layers are represented and interpreted. Permitted from this a formal procedure for layer-wise diagonalisation, where typically interconnected layers, such as dense layers, convolutional kernels and more, can be reexpressed such that neurons have a one-to-one ordered connectivity. This indicates which one-to-one neuron communications are strongly impactful on functionality and those which are not. Inconsequential neurons can be removed, and new `scaffold' neurons added, whilst remaining analytically functionally invariant. A new tunable model parameter, `intrinsic length', is also introduced to aid in this analytical invariance. This approach makes connectivity pruning equivalent to neurodegenesis. This diagonalisation also offers new possibilities for interpretability into such networks, and it is demonstrated that isotropic dense networks can asymptotically reach a sparsity factor of $50\%$ whilst retaining exact network functionality. The work includes investigations into integration and separation within networks that respond to both differing and joint tasks. Finally, the construction is generalised, demonstrating a nested functional class for this form of isotropic primitive architectures.