IntrinsicML: Transparent Hand-Vectorized SIMD Microkernels for Machine Learning Workloads

We present IntrinsicML, an open-source C++17 library of clean, well-documented hand-vectorized microkernels for core machine learning primitives targeting x86 processors with AVX2/FMA and optional AVX-512 support. The library implements two primary kernels: (1) a cache-blocked, explicitly packed single-precision general matrix multiplication (SGEMM) microkernel following the classic five-loop Goto/BLIS structure with panel packing, register blocking (MR×NR), software prefetching, and runtime ISA dispatch; and (2) a fully branch-free AVX2 vectorized approximation of the Gaussian Error Linear Unit (GeLU) activation function using a rational polynomial evaluation of tanh. Python bindings exposed via pybind11 provide direct NumPy-compatible interfaces. A distinguishing feature of the project is its dual commitment to pedagogical transparency and production-grade engineering. Every design decision is documented in source comments and accompanying design documents. The build system uses modern CMake with runtime AVX-512 detection, 64-byte aligned memory allocation, comprehensive unit and numerical precision tests, and a mature GitHub Actions CI/CD pipeline comprising continuous integration, dedicated build-and-test, weekly benchmarking, and zero-compromise security workflows.
We provide a detailed walkthrough of the packed GEMM implementation, honest performance characterization relative to naive triple-loop code, and a frank discussion of the remaining gap to highly-tuned production BLAS libraries such as OpenBLAS and Intel oneMKL. IntrinsicML is intended as a transparent reference that helps practitioners and students understand the mapping from high-level algorithmic structure (tiling, packing, register blocking) to concrete x86 machine instructions, while remaining fully buildable, testable, and usable in real ML pipelines.