Robustness assessment of a C++ implementation of the LeNet-5 convolutional neural network.

The architecture of the LeNet-5 convolutional neural network (CNN) was defined by LeCun in its paper "Gradient-based learning applied to document recognition" (https://ieeexplore.ieee.org/document/726791) to classify images of hand written digits (MNIST dataset).

This architecture has been customized to use Rectified Linear Unit (ReLU) as activation functions instead of Sigmoid.

It consists of the following layers:

• conv1: Convolution 2D, 1 input channel (28x28), 3 output channels (28x28), kernel size 5, stride 1, padding 2.
• relu1: Rectified Linear Unit (3@28x28).
• max1: Subsampling buy max pooling (3@14x14).
• conv2: Convolution 2D, 3 input channels (14x14), 6 output channels (14x14), kernel size 5, stride 1, padding 2.
• relu2: Rectified Linear Unit (6@14x14).
• max2: Subsampling buy max pooling (6@7x7).
• fc1: Fully connected (294, 147)
• fc2: Fully connected (147, 10)

The fault hypotheses for this work include the occurrence of:

• BF: single bit-flip faults
• S0: single, double-adjacent and triple-adjacent stuck-at-0 faults
• S1: single, double-adjacent and triple-adjacent stuck-at-1 faults

In the memory cells containing all the parameters of the CNN:  

• w: weights (float32)
• b: biases (float32)

Images 200 to 249 from the MNIST dataset have been used as workload.

This dataset contains the raw data obtained from running exhaustive fault injection campaigns for all considered fault models, targeting all considered locations and for all the images in the workload.

Files information

• golden_run.csv: Prediction obtained for all the images considered in the workload in the absence of faults (Golden Run). This is intended to act as oracle to determine the impact of injected faults.
• single_faults/bit_flip folder: Prediction obtained for all the images considered in the workload in presence of single bit-flip faults. There is one file for each parameter of each layer.
• single_faults/stuck_at_0 folder: Prediction obtained for all the images considered in the workload in presence of single stuck-at-0 faults. There is one file for each parameter of each layer.
• single_faults/stuck_at_1 folder: Prediction obtained for all the images considered in the workload in presence of single stuck-at-1 faults. There is one file for each parameter of each layer.
• double_adjacent_faults/stuck_at_0 folder: Prediction obtained for all the images considered in the workload in presence of double adjacent stuck-at-0 faults. There is one file for each parameter of each layer.
• double_adjacent_faults/stuck_at_1 folder: Prediction obtained for all the images considered in the workload in presence of double adjacent stuck-at-1 faults. There is one file for each parameter of each layer.
• triple_adjacent_faults/stuck_at_0 folder: Prediction obtained for all the images considered in the workload in presence of triple adjacent stuck-at-0 faults. There is one file for each parameter of each layer.
• triple_adjacent_faults/stuck_at_1 folder: Prediction obtained for all the images considered in the workload in presence of triple adjacent stuck-at-1 faults. There is one file for each parameter of each layer.

Methodology information

First, the CNN was used to classify all the images of the workload in the absence of faults to get a reference to determine the impact of faults. This is golden_run.csv file.

After that, one fault injection experiment was executed for each of the 16 most significant bits bit of each element of each parameter of the CNN, as previous fault injection experiments showed that the occurrence of the considered faults in the 16 least significant bits does not impact the behaviour of the network.

Each experiment consisted in:

• Affecting the bits (inverting it in case of bit-flip faults, setting it to 0 or 1 in case of stuck-at-0 or atuck-at-1 faults) identified by the mask.
• Classifying all the images of the workload in the presence of this fault. The obtained output was stored in a given .csv file.
• Removing the fault from the CNN by restoring the affected bits to its previous value.

List of variables (Name : Description (Possible values))

• IMGID: Integer number identifying the considered image (200-249).
• TENSORID: Integer number identiying the parameter affected by the fault (0 - No fault, 1 - conv1.w, 2 - conv1.b, 3 - conv2.w, 4 - conv2.b, 5 - fc1.w, 6 - fc1.b, 7 - fc2.w, 8 - fc2.b).
• ELEMID: Integer number identiying the element of the parameter affected by the fault (-1 - No fault, [0-2] - conv1.b, [0-74] - conv1.w, [0-5] - conv2.b, [0-149] - conv2.w, [0-146] - fc1.b, [0-43217] - fc1.w, [0-9] - fc2.b, [0-1469] - fc2.w).
• MASK: 8-digit hexadecimal number identifying those bits affected by the fault ([00000000 - No fault, FFFFFFFF - all 32 bits faulty]).
• FAULT: String identiying the type of fault (NF - No fault, BF - bit-flip, S0 - Stuck-at-0, S1 - Stuck-at-1).
• SOFTMAX: 10 decimal numbers obtained after applying the softmax function to the provided output. They represent the probability of the image of belonging to the corresponding category for classification.
• PRED: Integer number representing the category predicted for the processed image.
• LABEL: integer number representing the actual category for the processed image.