Efficient TinyML Inference on a Fault-Tolerant RISC-V SoC with Vector Extension

Tiny machine learning (TinyML) enables low-power microcontrollers to run neural network inference locally, allowing real-time classification near the sensor with reduced latency and no reliance on external infrastructure. This is especially relevant in constrained environments such as autonomous systems, satellites, and safety-critical embedded applications, where energy efficiency and reliability are essential. This work evaluates the inference performance of a quantized convolutional neural network on HARV-SoC, a fault-tolerant RISC-V system-on-chip with vector extension support. The SoC includes our port of TensorFlow Lite for Microcontrollers with accelerated vectorized convolution functions. Inference was tested using RGB and grayscale inputs to analyze how input structure impacts vectorization. Results show that, compared to the original convolution function, the RGB and grayscale models achieved 1.80 and 2.05 times speedup, respectively. The highest gains were observed when the input structure enabled better use of the available parallelism.