Stochastic Inference in Bitwise Neural Networks: Comparison with Ensemble Methods on CIFAR-100

Recently published methods enable training of bitwise neural networks which allow reduced representation of down to a single bit per weight. We present a method that exploits ensemble decisions based on multiple stochastically sampled network models to increase performance figures of bitwise neural networks in terms of classification accuracy at inference. Our experiments with the CIFAR-10 and GTSRB datasets show that the performance of such network ensembles surpasses the performance of the high-precision base model. With this technique we achieve 5.81\% best classification error on CIFAR-10 t

Research goal: How does the stochastic inference technique in bitwise neural networks compare to other ensemble methods (e.g., snapshot ensembles, Monte Carlo dropout) in terms of classification accuracy and computational overhead when evaluated on the CIFAR-100 dataset?

Autonomous synthesis report generated by SOVEREIGN Research Kernel. Tribunal consensus score: 8.5/10.