Published December 22, 2020 | Version v1
Journal article Open

Compressed code and data

Authors/Creators

Description

Lossy compression of earth system model data based on hierarchical tensor with Adaptive-HGFDR 

INTRODUCTION
------------
"Lossy compression of earth system model data based on hierarchical tensor with Adaptive-HGFDR" involves four compression algorithms: Adaptive-HGFDR, Blocked_HGFDR, ZFP, SZ. Therefore, the file includes the codes of these four algorithms and the experimental codes and data involved in the paper.
There are four experiments involved in the thesis: 1.Optimal block count selection, 2.Comparison with SZ, 3.Comparison with ZFP and Blocked-HGFDR, 4.Evaluation with multiple variables. Each algorithm has a corresponding readme to help users run it.
The user only needs to open the corresponding experiment code and run the code according to the prompts to restore the experiment in the paper.
Tip: Most of the experimental codes only get the data in the picture, not the picture directly. Most of the pictures in the paper are made by ORIGIN(origin).


COMPONENT
------------
The file is divided into two parts, one part is Compression algorithm, including the code and operating instructions of the four compression algorithms;
the other part is Main experiment, including the four experiments in the paper, each experiment is independent of each other, and the user can run it separately.
Compression algorithm:  1.Adaptive-HGFDR, 2.Blocked_HGFDR, 3.ZFP, 4.SZ. 
Main experiment: 1.Optimal block count selection, 2.Comparison with SZ, 3.Comparison with ZFP and Blocked-HGFDR, 4.Evaluation with multiple variables. 
If users want to implement all the experiments in the paper, they can run each experiment in the main experiment in turn.

1.Optimal block count selection————》The code and data in Figure 2 and Figure 3 in the paper can be obtained.(Figure 2: The relationship between the block count and the compression ratio)(Figure 3: Original data and compressed data with different block counts.)
2.Comparison with SZ————》The code and data in Figure 4 in the paper can be obtained.(Figure 4. The compression error distribution along different dimensions.)
3.Comparison with ZFP and Blocked-HGFDR————》The code and data in Figure 5, Figure 6 and Figure 7 in the paper can be obtained.
4.Evaluation with multiple variables. ————》The code and data in Figure 8 in the paper can be obtained.

DATA
------------
The experimental data used in the paper includes T temperature data (1024 * 512 * 26) and 22 other variable data (1024 * 512 * 221).
The experimental data are Large-scale Data Analysis and Visualization Symposium Data obtained from (OSDC) Open Science Data Cloud. This data set consists of files from a series of global climate dynamics simulations run on the Titan supercomputer at Oak Ridge National Laboratory in 2013 by postdoctoral researcher Abigail Gaddis, Ph.D. The simulations were performed at approximately 1/3-degree spatial resolution, or a mesh size of 1024x512 for 2D. We downloaded this simulation data in the common NetCDF (network Common Data Form) format in 2016 from https://www.opensciencedatacloud.org/.
The experimental data can also be downloaded for free from http://doi.org/10.5281/zenodo.3997216.

ENVIRONMENT
------------
Adaptive-HGFDR Research experiments were performed by the MATLAB R2017a environment on a Windows 10 Workstation (HP Compaq Elite 8380 MT) with Intel Corei7-3770 (3.4 GHz) processors and 8 GB of RAM.
Before running the experimental code, add the corresponding library (function) to the path.

Files

Compressed code and data.zip

Files (1.5 GB)

Name Size Download all
md5:053307e665045b7277686cd5d603aff2
1.5 GB Preview Download