A comprehensive guide to Bayesian CNN with variational inference

Artificial Neural Networks are connectionist systems that perform a given task by learning on examples without having prior knowledge about the task. This is done by finding an optimal point estimate for the weights in every node. Generally, the network using point estimates as weights perform well with large datasets, but they fail to express uncertainty in regions with little or no data, leading to overconfident decisions. In this work, Bayesian Convolutional Neural Network (BayesCNN) using Variational Inference is proposed, that introduces probability distribution over the weights. Furthermore, the proposed BayesCNN architecture is applied to tasks like Image Classification, Image Super-Resolution and Generative Adversarial Networks. BayesCNN is based on Bayes by Backprop which derives a variational approximation to the true posterior. We, therefore, introduce the idea of applying two convolutional operations, one for the mean and one for the variance. We predict how certain the model prediction is based on the epistemic and aleatoric uncertainties and show how the uncertainty can decrease, allowing the decisions made by the network to become more deterministic over time.

Autorentext

Kumar Shridhar holds a Masters degree in Computer Science from the University of Kaiserslautern, Germany with a minor in Psychology. His masters' work was in the area of Bayesian Deep Learning, including proposed ways to apply Bayesian inference to Convolution Neural Networks and to estimate uncertainties over predictions.