Thermal Bridges on Building Rooftops - Hyperspectral (RGB + Thermal + Height) drone images of Karlsruhe, Germany, with thermal bridge annotations

Overview:

The dataset of Thermal Bridges on Building Rooftops (TBBR dataset) consists of annotated combined RGB and thermal drone images with a height map. All images were converted to a uniform format of 3000x4000 pixels, aligned, and cropped to 2680x3370 to remove empty borders. See the "Usage" section below for details about the stored formats made available here.

The raw images for our dataset were recorded with a normal (RGB) and a FLIR-XT2 (thermal) camera on a DJI M600 drone. They show six large building blocks of around 20 buildings per block recorded in the city centre of the German city Karlsruhe east of the market square. Because of a high overlap rate of the images, the same buildings are on average recorded from different angles in different images about 20 times.

All images were recorded during a drone flight on March 19, 2019 from 7 a.m. to 8 a.m. At this time, temperatures were between 3.78 ° C and 4.97 ° C, humidity between 80% and 98%. There was no rain on the day of the flight, but there was 2.3mm/m² 48 hours beforehand. For recording the thermographic images an emissivity of 1.0 was set. The global radiation during this period was between 38.59 W / m² and 120.86 W / m². No direct sunlight can be seen visually on any of the recordings.

The dataset contains 926 images with a total of 6,927 annotations of thermal bridges on rooftops, split into train and test subsets with 723 (5,614) and 203 (1,313) images (annotations), respectively. The annotations only include thermal bridges that are visually identifiable with the human eye. Because of the aforementioned image overlap, each thermal bridge is annotated multiple times from different angles.

For the annotation of the thermal images the image processing program VGG Image Annotator from the Visual Geometry Group, version 2.0.10, was used. The thermal bridge annotations are outlined with polygon shapes. These polygon lines were placed as close as possible but outside the area of significant temperature increase. If a detected thermal bridge was partially covered by another building component located in the foreground, the thermal bridge was also marked across the covering in case of minor coverings. Adjacent thermal bridges, which affect different rooftop components, were annotated separately. For example, a window with poor insulation of the window reveal located in the area of a poorly insulated roof is annotated individually. There is no overlap between annotated areas. While each image contains annotations, they also include thermal bridges present that are not annotated.

Usage:

Each compressed archive file represents one of the six building blocks. For the related publication the final block (Flug1_105Media) was used as a hold-out test sample. The archives contain Numpy files (one per image) of shape (2680, 3370, 5), where the final dimension is the colour channel in the format [B, G, R, Thermal, Height].

Archives were compressed using ZStandard compression. They can be decompressed in a terminal by running e.g.

tar -I zstd -xvf Flug1_105Media.tar.zst

these will be decompressed into the file structure:

images/
└── Flug1_105Media/
    └── DJI_0004_R.npy
    └── DJI_0006_R.npy
    └── ...

Corresponding annotations are provided in the COCO JSON format. There is one file for training (Flug1_100Media - Flug1_104Media blocks) and one for test (Flug1_105Media block). They contain a single class (thermal bridge) and expect the folder structure shown below.

Note: The annotation files contain relative paths to numpy files, in case of problems please convert to absolute paths (i.e. insert the containing directory before each file path in the JSON annotation files).

We recommend the following folder structure for reproduction of our work with Detectron2:

├── train/
│   ├── Flug1_100-104Media_coco.json
│   └── images/
│       ├── Flug1_100Media/
│       │   ├── DJI_XXXX_R.npy
│       │   └── ...
│       ├── ...
│       └── Flug1_104Media/
│           ├── DJI_XXXX_R.npy
│           └── ...
└── test/
    ├── Flug1_105Media_coco.json
    └── images/
        └── Flug1_105Media/
            ├── DJI_XXXX_R.npy
            └── ...