SIMD Acceleration of Matrix-Vector Operations on RISC-V for Variable Precision Neural Networks

The rapid proliferation of Edge AI applications demands efficient, low-power computing architectures tailored to specific workloads. The RISC-V ecosystem is a promising solution, and has led to a fast growth of implementations based on custom instructions extensions, but with varying degrees of functionality and support which may hinder easy adoption.In this paper, we extensively review existing RISC-V extensions targeting primarily the AI domain and respective compilation flows, highlighting challenges in deployment, usability, and compatibility. We further implement and provide usable containerized environments for two of these works. To address the identified challenges, we then propose an approach for lightweight early validation of custom instructions via source-to-source transformations, without need of compiler modifications. We target our own Single Instruction Multiple Data (SIMD) accelerator, which we integrate into a CORE-V cv32e40px baseline core through custom instructions, and versus which we achieve up to 11.9× speedup for matrix-vector operations.