Dataset for "Enhancing Cloud Detection in Sentinel-2 Imagery: A Spatial-Temporal Approach and Dataset"

This dataset is built for time-series Sentinel-2 cloud detection and stored in Tensorflow TFRecord (refer to https://www.tensorflow.org/tutorials/load_data/tfrecord).

Each file is compressed in 7z format and can be decompressed using Bandzip or 7-zip software.

Dataset Structure:

Each filename can be split into three parts using underscores. The first part indicates whether it is designated for training or validation ('train' or 'val'); the second part indicates the Sentinel-2 tile name, and the last part indicates the number of samples in this file.

For each sample, it includes:

1. Sample ID;
2. Array of time series 4 band image patches in 10m resolution, shaped as (n_timestamps, 4, 42, 42);
3. Label list indicating cloud cover status for the center \(6\times6\) pixels of each timestamp;
4. Ordinal list for each timestamp;
5. Sample weight list (reserved);

Here is a demonstration function for parsing the TFRecord file:

import tensorflow as tf

# init Tensorflow Dataset from file name
def parseRecordDirect(fname):
    sep = '/'
    parts = tf.strings.split(fname,sep)
    tn = tf.strings.split(parts[-1],sep='_')[-2]
    nn = tf.strings.to_number(tf.strings.split(parts[-1],sep='_')[-1],tf.dtypes.int64)
    t = tf.data.Dataset.from_tensors(tn).repeat().take(nn)
    t1 = tf.data.TFRecordDataset(fname)
    ds = tf.data.Dataset.zip((t, t1))
    return ds

keys_to_features_direct = {
    'localid': tf.io.FixedLenFeature([], tf.int64, -1),
    'image_raw_ldseries': tf.io.FixedLenFeature((), tf.string, ''),
    'labels': tf.io.FixedLenFeature((), tf.string, ''),
    'dates': tf.io.FixedLenFeature((), tf.string, ''),
    'weights': tf.io.FixedLenFeature((), tf.string, '')
        }

# The Decoder (Optional)
class SeriesClassificationDirectDecorder(decoder.Decoder):
  """A tf.Example decoder for tfds classification datasets."""
  def __init__(self) -> None:
    super().__init__()

  def decode(self, tid, ds):
    parsed = tf.io.parse_single_example(ds, keys_to_features_direct)
    encoded = parsed['image_raw_ldseries']
    labels_encoded = parsed['labels']
    decoded = tf.io.decode_raw(encoded, tf.uint16)
    label = tf.io.decode_raw(labels_encoded, tf.int8)
    dates = tf.io.decode_raw(parsed['dates'], tf.int64)
    weight = tf.io.decode_raw(parsed['weights'], tf.float32)
    decoded = tf.reshape(decoded,[-1,4,42,42])
    sample_dict = {
      'tid': tid, # tile ID
      'dates': dates, # Date list
      'localid': parsed['localid'], # sample ID
      'imgs': decoded, # image array
      'labels': label, # label list
      'weights': weight
    }
    return sample_dict

# simple function 
def preprocessDirect(tid, record):
    parsed = tf.io.parse_single_example(record, keys_to_features_direct)
    encoded = parsed['image_raw_ldseries']
    labels_encoded = parsed['labels']
    decoded = tf.io.decode_raw(encoded, tf.uint16)
    label = tf.io.decode_raw(labels_encoded, tf.int8)
    dates = tf.io.decode_raw(parsed['dates'], tf.int64)
    weight = tf.io.decode_raw(parsed['weights'], tf.float32)
    decoded = tf.reshape(decoded,[-1,4,42,42])
    return tid, dates, parsed['localid'], decoded, label, weight

t1 = parseRecordDirect('filename here')
dataset = t1.map(preprocessDirect, num_parallel_calls=tf.data.experimental.AUTOTUNE)

#

Class Definition:

• 0: clear
• 1: opaque cloud
• 2: thin cloud
• 3: haze
• 4: cloud shadow
• 5: snow

Dataset Construction:

First, we randomly generate 500 points for each tile, and all these points are aligned to the pixel grid center of the subdatasets in 60m resolution (eg. B10) for consistence when comparing with other products. 
It is because that other cloud detection method may use the cirrus band as features, which is in 60m resolution. 

Then, the time series image patches of two shapes are cropped with each point as the center.
The patches of shape \(42 \times 42\) are cropped from the bands in 10m resolution (B2, B3, B4, B8) and are used to construct this dataset.
And the patches of shape \(348 \times 348\) are cropped from the True Colour Image (TCI, details see sentinel-2 user guide) file and are used to interpreting class labels.

The samples with a large number of timestamps could be time-consuming in the IO stage, thus the time series patches are divided into different groups with timestamps not exceeding 100 for every group.