A Novel Posit-based Fast Approximation of ELU Activation Function for Deep Neural Networks

Nowadays,  real-time  applications  are  exploiting DNNs  more  and  more  for  computer  vision  and  image  recognition  tasks.  Such kind of applications are posing strict constraints in terms of both fast and efficient information representation and processing. New formats for representing real numbers have been proposed and among them the Posit format appears to be very promising, providing means  to  implement  fast  approximated  version  of widely  used activation functions in DNNs. Moreover, information processing performance  are  continuously  improved  thanks  to  advanced vectorized  SIMD  (single-instruction  multiple-data)  processor architectures  and  instructions  like  ARM  SVE (Scalable Vector Extension). This  paper explores both  approaches (Posit-based implementation of activation functions and vectorized SIMD processor architectures) to  obtain  faster  DNNs.  The  two  proposed  techniques  are able  to  speed  up  both  DNN training  and  inference steps.