Data archive for paper "Machine Learning Emulation of 3D Cloud Radiative Effects"

Overview

This archive contains models, data, and the Singularity image to optionally rerun experiments described in "Machine Learning Emulation of 3D Cloud Radiative Effects".

For the Python tool to generate synthetic data, please refer to the Synthia repository.

Prerequisites

• Linux or macOS with Bash shell.
• Singularity (tested with version 3.6.3-1.el8)*.
• Portable Batch System (PBS) job scheduler**.

*Please note that all steps require Singularity to be installed on your system. If you are looking for information on how to install or use Singularity, please refer to the Singularity documentation.

**Although PBS in not a strict requirement, it is required to run all helper scripts as included in this repository. Please note that depending on your specific system settings and resource availability, you may need to modify PBS parameters at the top of submit scripts stored in the hpc directory (e.g. #PBS -lwalltime=24:00:00).

Initialization

Deflate the data archive with:

./init.sh

Compile ecRad with Singularity:

./tools/singularity/compile_ecrad.sh

Usage

To reproduce the results as described in the paper, run the following commands from the hpc folder:

qsub -v JOB_NAME=mlp_default ./submit_grid_search_default.sh
qsub -v JOB_NAME=mlp_synthia ./submit_grid_search_synthia.sh
qsub submit_benchmark.sh

then, to plot stats and identify notebooks run:

qsub submit_stats.sh

License

Paper code released under the MIT license. Data released under CC BY 4.0. ecRad released under the Apache 2.0 license.