Welcome to Pyeo’s documentation!

Introduction

Python For Earth Observation is a collection of functions for downloading, manipulating, combining and classifying geospatial raster and vector data. It is intended to require a minimum of dependencies - most functions only require the basic GDAL/OGR/OSR stack and Numpy.

Installation

With Git and Miniconda or Anaconda installed, cd to an install location then run the following lines

git clone https://github.com/clcr/pyeo.git
cd pyeo
conda env create --file environment.yml --name pyeo_env
conda activate pyeo_env
python -m pip install . -vv

In a Python prompt, try import pyeo - you should see no errors.

Quick start

Before you start, you will need:

  • A linux (or maybe mac, untested) machine

  • A raster of your window area

  • A shapefile of polygons over your training areas with a field containing class labels

  • A raster to classify. This can be the same as your original raster.

    • All rasters and shapefiles should be in the same projection; ideally in the local projection of your satellite data.

  • Git

  • Anaconda/Miniconda

Use

To create a model

conda activate pyeo_env
cd (whereever your data is)
extract_signatures your_raster.tif your_shapefile.shp output.csv
# You can now explore your signatures as a .csv file, and perform any interim processing
create_model_from_signatures output.csv model.pkl
classify_image your_raster model.pkl output_image.tif

Assumptions and design decisions

Rasters

When working with raster data (geotiff, .jp2, ect) using Pyeo, the folllowing assumtions have been made:

  • Any function that reads a raster can read from any gdal-readable format

  • All interim rasters are stored internally as a geotiff

  • All internal rasters have a .tif extension in the filename

  • Unless otherwise stated, all rasters are assumed to be in a projected coordinate system - i.e. in meters. Functions may fail if passed a raster in lat-long projection

Masks

Some Pyeo functions include options for applying masks.

  • A raster may have an associated mask

  • A mask is a geotif with an identical name as the raster it’s masking with a .msk extension

    • For example, the mask for my_sat_image.tif is my_sat_image.msk

  • A mask is

    • a single band raster

    • of identical height, width and resolution of the related image

    • contains values 0 or 1

  • A mask is applied by multiplying it with each band of its raster

    • So any pixel with a 0 in its mask will be removed, and a 1 will be kept

Timestamps

  • Pyeo uses the same timestamp convention as ESA: yyyymmddThhmmss

    • For example, 1PM on 27th December 2020 would be 20201227T130000

  • All timestamps are in UTC

Models

  • All models are serialised and deserialised using joblib.dump or joblib.load

A small test suite is located in pyeo/tests/pyeo_tests.py; this is designed for use with py.test. Some example applications and demos are in pyeo/apps; for an illustration of the use of the library, pyeo/apps/change_detection/simple_s2_change_detection.py is recommended.

Applications

image_comparison

An application for applying a pickled scikit-learn model to two contiguous raster images.

Example call:

$image_comparison image_1.tif image_2.tif model.pkl class_map.tif

rolling_composite_s2_change_detection

An app for providing continuous change detection. Runs the following algorithm

Step -1: Create initial composite with last date (stored in filename?)

Step 0: Load composite

Step 1: Download images from last date in composite until present last date

Step 2: Preprocess each image

Step 3: Generate cloud mask for each image

For each preprocessed image:

Step 4: Build stack with composite

Step 5: Classify stack

Step 6: Update composite with last cloud-free pixel based on cloud mask

Step 7: Update last_date of composite

simple_s2_change_detection

A change detection script that downloads, stacks and classifies a set of 10m sentinel 2 images.

When run, this script will download every S2 image in the aoi (defined by the geojson at aoi_path) between the two dates that meets the specified cloud cover range. It will use the sen2cor distribution specified in the .ini file to atmospherically correct the data to L2A, merge each set of 10m bands into a single geotiff, stack the images into pairs based on the algorithm in create_new_stacks and classify those images using a scikit-learn model

To use this script, fill out the [sent_2], [forest_sentinel] and [sen2cor] sections of the configuration file change_detection.ini, then run

$ python pyeo/apps/change_detection/simple_s2_change_detection.py --conf /path/to/change_detection.ini

Produces two directories of un-mosaiced imagery; one of classified images and one of class probabilites

filter_by_class_map

Filters every pixel in input_image that is not one of filter_classes.

Usage:

$ filter_by_class_map my_image.tif my_class_map.tif my_output_image.tif useful_class_1 useful_class_2

This will create an image, my_output_image.tif, that contains only the pixels from my_image.tif that are labelled as useful_class_1 and useful_class_2 in my_class_map.tif