EvoTransformer: A Whitepaper on Evolving Neural Architectures for Open-Ended AI

EvoTransformer is a transformer architecture that incorporates evolutionary principles—mutation, selection, and shape-safe weight inheritance—directly into the model’s training and adaptation process. Unlike conventional static architectures, EvoTransformer can evolve its structure both during training and after deployment, enabling continual optimization within fixed compute budgets.

This whitepaper details the conceptual foundations, genome representation, mutation strategies, and speciation mechanisms used in EvoTransformer. We present small-compute experiments on PIQA, HellaSwag, and SST-2 benchmarks, showing competitive quality-per-parameter results compared to established baselines trained under identical compute constraints.

Designed for sovereign, domain-specific, and resource-constrained deployments, EvoTransformer offers a reproducible and adaptable framework for efficient AI. Limitations, including the absence of large-scale pretraining, are openly discussed. Reproducibility resources—such as configuration files, seeds, and logs—are provided to encourage independent verification and further research.