Thermal Anomaly Segmentation Dataset - Thermal UAS-based Images from Germany with Annotations for Semantic Segmentation Model Training

The Thermal Anomaly Segmentation (TASeg) dataset is provided in accompaniment of the paper "Leak detection using thermal imagery: Deep learning versus traditional computer vision state-of-the-art" and can be utilised for the multi-stage training of spectral deep learning models for binary semantic segmentation. Specifically, this model is utilised in the context of leak detection in district heating networks to segment thermal anomalies from the background in urban cityscapes.

The provided data consists of thermal imagery recorded in Germany, close to Munich and Karlsruhe, in December 2019 and January / March 2021 using FLIR and DJI's Zenmuse XT2 and a Matrice 600 / Matrice 300 unmanned aircraft system (UAS). Seven datasets (KA1, KA2, MU1, MU2, MU6, MU15, and MU16) form the basis of the dataset. These are provided as part of a previous Zenodo dataset publication "Detecting District Heating Leaks in Thermal Imagery: Comparison of Anomaly Detection Methods - Source Code and Datasets".

For the training of deep learning semantic segmentation models, a multi-stage procedure is utilised with here provided two datasets: a generated set for stages 1 and 2 and a manual set for stage 3. These two datasets are provided here:

1. The "generated_set" contains segmented annotation masks generated via heuristic algorithm, specifically adaptive triangle-histogram-thresholding.
2. The "manual_set" consists of segmented annotation masks created by hand, by means of a custom labelling GUI tool.

These two datasets are split as follows for training:

1. Generated: 3,171 images -> Train: 2,142, Validation: 404, Test: 625
2. Manual: 269 images -> Train: 172, Validation: 52, Test: 45

In addition, the conda environment for training is provided here, compressed into the "dl_env.tar.gz".

We supply the software via the GitHub repository TASeg to showcase how these datasets and environment can be utilised to train transformers (such as the SegFormer) and convolutional neural networks (such as DeepLabV3+).

Usage

Dataset files

The two compressed dataset zip files can be decompressed in a terminal by running e.g.

unzip generated_set.zip
unzip manual_set.zip

These will be decompressed into the file structure, where "XY##" represents one of the seven UAV flights ("XY" = city abbrevation, "##" = flight number):

├── generated_set/
│   ├── dataset_info.json
│   ├── image/
│   │   ├── train/
│   │   │   ├── XY##_DJI_..._R.npy.lz4
│   │   │   └── ...
│   │   ├── val/...
│   │   └── test/...
│   ├── label/
│   │   ├── train/
│   │   │   ├── XY##_DJI_..._R.png
│   │   │   └── ...
│   │   ├── val/...
│   │   └── test/...
│   └── preview/
│       ├── train/...
│       │   ├── XY##_DJI_..._R.png
│       │   └── ...
│       ├── val/...
│       └── test/...
│
└── manual_set/
│   └── ...

Both sets contain subfolders of data, specifically:

• "image" (".npy.lz4" format): 3-channel temperature arrays as training images
  
  • channel 1 & 2: masked arrays with the district heating network pipeline
  • channel 3: unmasked, full array
    
    
• "label" (".png" format): 1-channel annotation images
  
  • channel 1: array with each pixel equal to either 0 (background) or 1 (thermal anomaly)
    
    
• "preview" (".png" format): side by side preview of images and their annotations

Environment file

Use the following command to unpack the environment:

tar -xzf dl_env.tar.gz -C /path/to/dst/dir

After that, the environment can be activated and utilised for model training in combination with f.e. Python 3.8 via:

source /path/to/dst/dir/dl_env/source/activate