Unsupervised Discovery of Extreme Weather Events Using Universal Representations of Emergent Organization

A record of code and data used to produce results for manuscript "Unsupervised Discovery of Extreme Weather Events Using Universal Representations of Emergent Organization" by Adam Rupe, Karthik Kashinath, Nalini Kumar, and James P. Crutchfield.

Includes DisCo source code, original run scripts used on the Cori supercomputer at NERSC, LBNL, as well as conda environments with required dependencies. The local causal state segmentation fields computed on Cori are included so that figures and analyses can be reproduced without HPC resources. 

Contents

Data

• netcdf_data.zip
  Contains CAM5.1 netcdf files with core climate variables.
• IVT_netcdfs.zip
  Contains netcdf files with Integrated Vapor Transport fields computed from the fields in netcdf_data.zip
• twodimturb.zip
  A netcdf file with the vorticity field from two-dimensional free-decay turbulence.
• reduced_npy_data.zip
  Numpy arrays of 4x reduced-resolution climate data (integrated vapor field and mid-column velocity components)
• jupyter_trim.npy
  Numpy ndarray of integer grayscale of interpolated RGB data from the NASA Cassini spacecraft
• IVT_alt-result-16.zip
  Local causal state segmentation results used as ".../IVT_alt/result-16/fields/" in climate notebooks. 
• IVT-result-8.zip
  Local causal state segmentation results used as ".../IVT/result-8/fields/" in climate notebooks.
• TC_seg_field.npy
  Numpy ndarray of TECA TC segmentation output. 
• TECA_BARD.zip
  Netcdf files of TECA BARD AR segmentation output. 
• LCS_1deg_reduced-full-3.zip
  Numpy arrays of local causal state segmentation output for the low-resolution climate data.
• turb-result-15.zip
  Numpy arrays of local causal state segmentation output for the two-dimensional turbulence data.
• vortex_counts-15.npy
  Output of union-find algorithm counting individual vortices from turb-results-15.zip turbulence segmentation output. 

Notebooks

• climate_figs.ipynb
  Reproduces majority of climate related figures in the manuscript.
• climate_old.ipynb
  Reproduces some older figures, some of which are used in Supplementary Information.
• climate-low-res.ipynb
  Creates the low-resolution numpy arrays from the CAM5.1 netcdf files, and also creates related figures.
• extreme_precipitation.ipynb
  Performs extreme precipitation analysis.
• jupyter_figs.ipynb
  Creates figures for Jupiter atmosphere segmentation. 
• TECA-compare.ipynb
  Creates figures from TECA segmentations. 
• turbulence_figs.ipynb
  Creates turbulence figures and vortex decay analyses. 

Src

• pdisco.py
  Python source code with core DisCo algorithms for distributed reconstruction of local causal states. 
• visuals.py
  Python source code for visualizing spacetime fields using matplotlib.

Test-single-node

• turb_small.npy
  Small sample of turbulence data used to test the DisCo local causal state reconstruction pipeline on a single machine. 
• single-node-turb.ipynb
  A full pipeline test of local causal state reconstruction that can run on single machine (e.g. a laptop). 
• single-node-turb.py
  A python script version of single-node-turb.ipynb. 

Scripts

Collection of python and SLURM run scripts used for experiments run on the Cori supercomputer. 

Param-logs

Logs of parameters and metadata used for various experiments run on the Cori supercomputer. The run numbers used for particular figures or analyses are indicated in the relevant notebook by the data being loaded.

Conda environment files

• disco-deps.yml
  Simple list of basic dependencies needed to run DisCo local causal state reconstruction code. 
• disco-env.yml
  Full conda environment details for running DisCo code on a single machine at time of this record's publication. 
• disco-cori-env.yml
  Record of conda environment used on the Cori supercomputer for producing the segmentation results shown in the manuscript.