OpenEarthMap Land Cover Mapping Few-Shot Learning Challenge

***The challenge is over, please use this verion for post-challenge research. This version contains all the data and files, 
except the labels of the query-set of the testset that are withheld and researchers can submit the predictions at the 
challenge leaderboard for evaluation***

Overview

This challenge is co-organized with the L3D-IVU 2024 CVPR workshop. The challenge is an extension of the OpenEarthMap benchmark dataset for a generalized few-shot semantic segmentation (GFSS) task. The challenge aims to evaluate and benchmark learning methods for few-shot semantic segmentation on the OpenEarthMap dataset to promote research on geoinformatics for social good. The motivation is to enable researchers to develop few-shot learning algorithms for high-resolution RS image semantic segmentation.

Page

https://cliffbb.github.io/OEM-Fewshot-Challenge/

Baseline

The baseline model for the challenge is available here.

Leaderboard

The challenge leaderboard is opened for post-challenge research to enable researchers to evaluate the predictions on the query-set of the testset.

Description

The dataset has been designed for remote sensing few-shot learning, particularly, for GFSS tasks in land cover mapping. The dataset consists of only 408 samples from the original OpenEarthMap dataset for RS image semantic segmentation. It extends the original 8 semantic classes of the OpenEarthmap benchmark to 15 classes, which is split into 7:4:4 for train_base_class, val_novel_class, and test_novel_class disjointed sets, respectively (i.e., train_base_class ∩ val_novel_class ∩ test_novel_class = ∅). The 408 samples are also split into 258 as `trainset`, 50 as `valset`, and 100 as `testset`. The `trainset` is for pre-training a backbone network. It contains only the images and labels of the train_base_class split. Both the `valset` and the `testset` consist of a support set and a query set for a 5-shot  with 4 novel classes  and 7 base classes  GFSS task. The `valset` and the `testset` contain the images and labels of the val_novel_class and the test_novel_class splits, respectively.

The challenge is in two phases: development phase and evaluation phase. The `valset` is for the development phase and the `testsets` is for the evaluation phase. Both `valset` and `testset`  have 20 image-label pair examples, 5-set examples for each of the 4 novel classes in the support set. The `valset` and the `testset` contain an additional 30 images and 80 images, respectively, in the query set, which are to be predicted using the 20 labelled images in their support set. The labels for each image in the support sets do not contain any of the train_base_class split. Also, in each 5-set examples, the labels contain only one novel class (i.e., one novel class per 5-set examples). However, in the `valset`, the labels for the images in the query set contain both train_base_class and val_novel_class; and in the `testset, the labels for the images in the query set contain both  train_base_class and test_novel_class `. Note that both the support set and query set in the `valset` are different from the ones in the `testset`.

File Structure and Content (All files are in `.tif` format):
-----------------------------------------------------------
1. **trainset.zip**:
    - Contains `images` and `labels` folders
    - `images` folder: 258 images of size 1024x1024 with a GSD (Ground Sampling Distance) of 0.6-1m.
    - `labels` folder: 258 segmentation masks of the images in the `images` folder.

2. **valset.zip**:
    - Contains `images` and `labels` folders
    - `images` folder: 50 images of size 1024x1024 with a GSD (Ground Sampling Distance) of 0.6-1m.
    - `labels` folder: 20 labels of the ``support set`` images in the `images` folder. The labels for
                       the 30 ``query set`` images in the `images` folder are withheld.
3. **testset.zip**:
    - Contains `images` and `labels` folders
    - `images` folder: 100 images of size 1024x1024 with a GSD (Ground Sampling Distance) of 0.6-1m.
    - `labels` folder: 20 labels of the ``support set`` images in the `images` folder. The labels for
                       the 80 ``query set`` images in the `images` folder are withheld.
 4. **train.txt**:
    - Contains a list of file names in the `trainset.zip`.
 
3. **val.json** and **test.json**:
    - Contains a list of file names the in the `valset.zip` and `testset.zip`, respectively. Below is
      the structure of the `val.json` and `test.json` files.
    - fnames = {
                 {"support_set": {8: ["filename_1.tif", "filename_2.tif", ...., "filename_5.tif"],
                                  9: ["filename_1.tif", "filename_2.tif", ...., "filename_5.tif"],
                                 10: ["filename_1.tif", "filename_2.tif", ...., "filename_5.tif"],
                                 11: ["filename_1.tif", "filename_2.tif", ...., "filename_5.tif"]},
                 {"query_set": ["filename_1.tif", "filename_2.tif", "filename_3.tif", ...
                                ...., 
                               "filename_n.tif"]}                   
               }


Land Cover Mapping Classes Strucure:
------------------------------------
1. **The `trainset`:
     classId2className = {
                          # ***Base classes***
                          1: 'tree',
                          2: 'rangeland',
                          3: 'bareland',
                          4: 'agric land type 1',
                          5: 'road type 1',
                          6: 'sea, lake, & pond',
                          7: 'building type 1'
                        }

2. **The `valset` and `testset`:
     classId2className = {
                          # ***Base classes***
                          1: 'tree',
                          2: 'rangeland',
                          3: 'bareland',
                          4: 'agric land type 1',
                          5: 'road type 1',
                          6: 'sea, lake, & pond',
                          7: 'building type 1'
                          # ***Novel classes***
                          8: '',
                          9: '',
                          10: '',
                          11: ''
                        }

      - The class names for the ***Novel classes*** depends on the data set.

        For the `valset`, the class names can be updated as:
                        {
                          8: 'road type 2',
                          9: 'river',
                          10: 'boat & ship',
                          11: 'agric land type 2'
                        }

        For the `testset`, the class names can be updated as:
                        {
                          8: 'vehicle & cargo-trailer',
                          9: 'parking space',
                          10: 'sports field',
                          11: 'building type 2'
                        }

License

See OpenEarthMap