Online and Interactive Bayesian Inference Debugging

Overview

This is the Online Appendix to: Online and Interactive Bayesian Inference Debugging. It contains:

• raw and cleaned data from the study,
• code used for the data analysis in Jupyter Notebooks
• additional graphs and statistical tests and checks inside Jupyter Notebooks
• Dockerfiles and instructions on how to reproduce the study environment
• Source code for InferLog Holmes and instructions on how to build it.
• Source code for the tool used to log the participants task attempts.

Paper Abstract

Probabilistic programming is a rapidly developing programming paradigm which enables the formulation of Bayesian models as programs and the automation of posterior inference. It facilitates the development of models and conducting Bayesian inference, which makes these techniques available to practitioners from multiple fields. Nevertheless, probabilistic programming is notoriously difficult as identifying and repairing issues with inference requires a lot of time and deep knowledge. Through this work, we introduce a novel approach to debugging Bayesian inference that reduces time and required knowledge significantly. We discuss several requirements a Bayesian inference debugging framework has to fulfill, and propose a new tool that meets these key requirements directly within the development environment. We evaluate our results in a study with 18 experienced participants and show that our approach to online and interactive debugging of Bayesian inference significantly reduces time and difficulty on inference debugging tasks.