Welcome to Pyeo’s documentation!¶
Contents:
Introduction¶
Python For Earth Observation is a collection of functions for downloading, manipulating, combining and classifying geospatial raster and vector data. It is intended to require a minimum of dependencies - most functions only require the basic GDAL/OGR/OSR stack and Numpy.
Installation¶
With Git and Miniconda or Anaconda installed, cd
to an install location then run the following lines
git clone https://github.com/clcr/pyeo.git
cd pyeo
conda env create --file environment.yml --name pyeo_env
conda activate pyeo_env
python -m pip install . -vv
In a Python prompt, try import pyeo
- you should see no errors.
Quick start¶
Before you start, you will need:
A linux (or maybe mac, untested) machine
A raster of your window area
A shapefile of polygons over your training areas with a field containing class labels
A raster to classify. This can be the same as your original raster.
All rasters and shapefiles should be in the same projection; ideally in the local projection of your satellite data.
Git
Anaconda/Miniconda
Use¶
To create a model
conda activate pyeo_env
cd (whereever your data is)
extract_signatures your_raster.tif your_shapefile.shp output.csv
# You can now explore your signatures as a .csv file, and perform any interim processing
create_model_from_signatures output.csv model.pkl
classify_image your_raster model.pkl output_image.tif
Assumptions and design decisions¶
Rasters¶
When working with raster data (geotiff, .jp2, ect) using Pyeo, the folllowing assumtions have been made:
Any function that reads a raster can read from any gdal-readable format
All interim rasters are stored internally as a geotiff
All internal rasters have a .tif extension in the filename
Unless otherwise stated, all rasters are assumed to be in a projected coordinate system - i.e. in meters. Functions may fail if passed a raster in lat-long projection
Masks¶
Some Pyeo functions include options for applying masks.
A raster may have an associated mask
A mask is a geotif with an identical name as the raster it’s masking with a .msk extension
For example, the mask for my_sat_image.tif is my_sat_image.msk
A mask is
a single band raster
of identical height, width and resolution of the related image
contains values 0 or 1
A mask is applied by multiplying it with each band of its raster
So any pixel with a 0 in its mask will be removed, and a 1 will be kept
Timestamps¶
Pyeo uses the same timestamp convention as ESA: yyyymmddThhmmss
For example, 1PM on 27th December 2020 would be 20201227T130000
All timestamps are in UTC
Models¶
All models are serialised and deserialised using joblib.dump or joblib.load
A small test suite is located in pyeo/tests/pyeo_tests.py; this is designed for use with py.test. Some example applications and demos are in pyeo/apps; for an illustration of the use of the library, pyeo/apps/change_detection/simple_s2_change_detection.py is recommended.
Applications¶
image_comparison¶
An application for applying a pickled scikit-learn model to two contiguous raster images.
Example call:
$image_comparison image_1.tif image_2.tif model.pkl class_map.tif
rolling_composite_s2_change_detection¶
- An app for providing continuous change detection. Runs the following algorithm
Step -1: Create initial composite with last date (stored in filename?)
Step 0: Load composite
Step 1: Download images from last date in composite until present last date
Step 2: Preprocess each image
Step 3: Generate cloud mask for each image
For each preprocessed image:
Step 4: Build stack with composite
Step 5: Classify stack
Step 6: Update composite with last cloud-free pixel based on cloud mask
Step 7: Update last_date of composite
simple_s2_change_detection¶
A change detection script that downloads, stacks and classifies a set of 10m sentinel 2 images.
When run, this script will download every S2 image in the aoi (defined by the geojson at aoi_path) between the two dates that meets the specified cloud cover range. It will use the sen2cor distribution specified in the .ini file to atmospherically correct the data to L2A, merge each set of 10m bands into a single geotiff, stack the images into pairs based on the algorithm in create_new_stacks and classify those images using a scikit-learn model
To use this script, fill out the [sent_2], [forest_sentinel] and [sen2cor] sections of the configuration file change_detection.ini, then run
$ python pyeo/apps/change_detection/simple_s2_change_detection.py --conf /path/to/change_detection.ini
Produces two directories of un-mosaiced imagery; one of classified images and one of class probabilites
filter_by_class_map¶
Filters every pixel in input_image that is not one of filter_classes.
Usage:
$ filter_by_class_map my_image.tif my_class_map.tif my_output_image.tif useful_class_1 useful_class_2
This will create an image, my_output_image.tif, that contains only the pixels from my_image.tif that are labelled as useful_class_1 and useful_class_2 in my_class_map.tif