Published October 7, 2022 | Version v1

Reproducible Notebook Containers using Application Virtualization

  • 1. DePaul University
  • 2. Argonne National Lab

Description

Notebooks have gained wide popularity in scientific computing. A notebook is both a web-based interactive frontend to program workflows and a lightweight container for sharing code and its output. Reproducing notebooks in different target environments, however, is a challenge. Notebooks do not share the computational environment in which they are executed. Consequently, despite being shareable they are often not reproducible. The application virtualization (AV) method enables shareability and reproducibility of applications in heterogeneous environments. AV-based tools, however, encapsulate noninteractive, batch applications. In this paper, we present FLINC, a user-space method and tool for creating reproducible notebook containers. FLINC virtualizes the notebook process that enables interactive computation and creates notebook containers, which include the environment and all data dependencies accessed by the notebook file. It relies on provenance collected during virtualization to ensure the correct behavior of a notebook when run repeatedly in different environments. We demonstrate how FLINC exports notebook containers seamlessly to non-notebook environments. Our experiments show that FLINC creates lighter weight containers as compared to equivalent non-interactive, batch containers, and preserves the same interactive workflow for the user as in current notebook platforms.

Files

Presentation-FLINC-eScience-22 - Split.pdf

Files (5.0 MB)

Name Size
md5:1e3655c8ed3c6558c7d132dcbd3b2984
5.0 MB Preview Download