Inference engine for custom neural networks with oneAPI

A recent effort to explore a neural network inference in FPGAs focusing on low-latency applications in triggering subsystems of the LHC, which would enable searches for new dark sector particles and novel measurements of the Higgs boson, resulted in a firmware implementation of machine learning algorithms using High-Level Synthesis language (HLS) for FPGAs, called hls4ml. Deep Learning algorithms using the hls4ml framework have quite impressive performance on FPGAs, but do not work well on contemporary architectures, like CPUs. To enable the possibility of using hls4ml models in High Level Trigger for CPUs, we explore usage of Intel oneAPI Toolkits in the hls4ml framework. We design, implement and integrate the inference engine with oneAPI into the hls4ml, and show that it can accelerate over hundreds of times the inference time for CPUs, if the data parallelism is exploited.