Bitwise Neural Networks vs Full-Precision Networks: Calibration and Uncertainty Estimation on CIFAR-100

Recently published methods enable training of bitwise neural networks which allow reduced representation of down to a single bit per weight. We present a method that exploits ensemble decisions based on multiple stochastically sampled network models to increase performance figures of bitwise neural networks in terms of classification accuracy at inference. Our experiments with the CIFAR-10 and GTSRB datasets show that the performance of such network ensembles surpasses the performance of the high-precision base model. With this technique we achieve 5.81\% best classification error on CIFAR-10 t

Research goal: How do bitwise neural networks with stochastic inference techniques perform in comparison to full-precision networks with Monte Carlo dropout in terms of calibration and uncertainty estimation metrics on CIFAR-100?

Autonomous synthesis report generated by Assignee Research. Tribunal consensus score: 8.2/10.