Thermal Bridges on Building Rooftops - Hyperspectral (RGB + Thermal + Height) drone images of Karlsruhe, Germany, with thermal bridge annotations

Overview:

The dataset of Thermal Bridges on Building Rooftops (TBBR dataset) consists of annotated combined RGB and thermal drone images with a height map. All images were converted to a uniform format of 3000x4000 pixels, aligned, and cropped to 2680x3370 to remove empty borders. See the "Usage" section below for details about the stored formats made available here.

The raw images for our dataset were recorded with a normal (RGB) and a FLIR-XT2 (thermal) camera on a DJI M600 drone. They show six large building blocks of around 20 buildings per block recorded in the city centre of the German city Karlsruhe east of the market square. Because of a high overlap rate of the images, the same buildings are on average recorded from different angles in different images about 20 times.

All images were recorded during a drone flight on March 19, 2019 from 7 a.m. to 8 a.m. At this time, temperatures were between 3.78 ° C and 4.97 ° C, humidity between 80% and 98%. There was no rain on the day of the flight, but there was 2.3mm/m² 48 hours beforehand. For recording the thermographic images an emissivity of 1.0 was set. The global radiation during this period was between 38.59 W / m² and 120.86 W / m². No direct sunlight can be seen visually on any of the recordings.

The dataset contains 926 images with a total of 6,927 annotations of thermal bridges on rooftops, split into train and test subsets with 723 (5,614) and 203 (1,313) images (annotations), respectively. The annotations only include thermal bridges that are visually identifiable with the human eye. Because of the aforementioned image overlap, each thermal bridge is annotated multiple times from different angles.

For the annotation of the thermal images the image processing program VGG Image Annotator from the Visual Geometry Group, version 2.0.10, was used. The thermal bridge annotations are outlined with polygon shapes. These polygon lines were placed as close as possible but outside the area of significant temperature increase. If a detected thermal bridge was partially covered by another building component located in the foreground, the thermal bridge was also marked across the covering in case of minor coverings. Adjacent thermal bridges, which affect different rooftop components, were annotated separately. For example, a window with poor insulation of the window reveal located in the area of a poorly insulated roof is annotated individually. There is no overlap between annotated areas. While each image contains annotations, they also include thermal bridges present that are not annotated.

Usage:

Each compressed archive file represents one of the six flight paths. For the related publication the final path (Flug1_105Media) was used as a hold-out test sample. The archives contain Numpy files (one per image) of shape (2680, 3370, 5), where the final dimension is the colour channel in the format [B, G, R, Thermal, Height].

Archives were compressed using ZStandard compression. They can be decompressed in a terminal by running e.g.

tar -I zstd -xvf Flug1_105Media.tar.zst

these will be decompressed into the file structure:

images/
└── Flug1_105Media/
    └── DJI_0004_R.npy
    └── DJI_0006_R.npy
    └── ...

Corresponding annotations are provided in the COCO JSON format. There is one file for training (Flug1_100Media - Flug1_104Media blocks) and one for test (Flug1_105Media block). They contain a single class (thermal bridge) and expect the folder structure shown below.

Note: The annotation files contain relative paths to numpy files, in case of problems please convert to absolute paths (i.e. insert the containing directory before each file path in the JSON annotation files).

We provide the TBBRDet software which includes a dataloader and dataset inspection tools which make use of the Detectron2 and MMDetection libraries.

We recommend the following folder structure for use:

├── train/
│   ├── Flug1_100-104Media_coco.json
│   └── images/
│       ├── Flug1_100Media/
│       │   ├── DJI_XXXX_R.npy
│       │   └── ...
│       ├── ...
│       └── Flug1_104Media/
│           ├── DJI_XXXX_R.npy
│           └── ...
└── test/
    ├── Flug1_105Media_coco.json
    └── images/
        └── Flug1_105Media/
            ├── DJI_XXXX_R.npy
            └── ...

Metadata:

The experimental metadata was structured with the Spatio Temporal Asset Catalog (STAC) specification family. This specification provides a standardized way for describing geospatial assets. It defines related JSON object types of Item, Catalog, and Catalog, extending on Collection as the basis.

One STAC Collection JSON object provides information about the recorded images and the environmental conditions during recordings. It also contains information about the overall bounding box of the entire area in which images were recorded.

This object links to related STAC Item JSON objects containing information about the recorded city blocks and the cameras. The objects for the city blocks contain the GeoJSON geometry of the respective block and the
corresponding bounding box. The objects containing the camera information are based on an existing STAC extension for camera related metadata.

Metadata of the archived NumPy files for each image was structured using the Data Package schema from the Frictionless Standards. This standard describes a collection of data files. Therefore, metadata about all containerized NumPy files of the six flight paths (Flug1_100Media - Flug1_104Media blocks and Flug1_105Media block) is provided within a JSON-based file.

Note that camera1 corresponds to the RGB camera and camera2 the thermal.

FAIR Digital Objects:

All files are represented in a standardized way as FAIR Digital Objects
(FAIR DOs) to enable machine actionable decisions on the data in spirit of
the FAIR principles.

Persistent Identifier (PID):

Persistent Identifiers (PIDs) are resolvable with the Handle.Net Registry (HNR).