Datasets for Machine Learning Shortwave Radiative Transfer

Author - Henry Schneiderman, henry@pittdata.com
Please contact me for any questions or feedback

Input reanalysis data downloaded from ECMWF's Copernicus Atmospheric Monitoring Service. Each atmospheric column contains the following input variables:

mu - Cosine of solar zenith angle
albedo - Surface albedo
is_valid_zenith_angle - Indicates if daylight is present
Vertical profiles (60 layers): Temperature Pressure, Change in Pressure, H2O (vapor, liquid, solid), O3, CO2, O2, N2O, CH4

The ecRad emulator (Hogan and Bozzo, 2018) generated the following output profiles at the layer interfaces for input each atmospheric column:

flux_down_direct, flux_down_diffuse, 
flux_down_direct_clear_sky, flux_down_diffuse_clear_sky,
flux_up_diffuse, flux_up_clear_sky

All data is sampled at 5,120 global locations

The training dataset uses input from 2008 sampled at three-hour intervals within every fourth day

The validation dataset uses input from 2008 sampled at three-hour intervals within every 28th day offset two days from the training set to avoid duplication

Testing datasets use input from 2009, 2015, and 2020. Each of these samples data at three-hour intervals within every 28th day.

For more information see:
Henry Schneiderman. "An Open Box Physics-Based Neural Network for Shortwave Radiative Transfer." Submitted to Artificial Intelligence for the Earth Systems.