pyeo.raster_manipulation¶
Functions for working with raster data.
NOTE: supported datatypes gdal.GDT_Unknown gdal.GDT_Byte gdal.GDT_UInt16 gdal.GDT_Int16 gdal.GDT_UInt32 gdal.GDT_Int32 gdal.GDT_Float32 gdal.GDT_Float64 gdal.GDT_CInt16 gdal.GDT_CInt32 gdal.GDT_CFloat32 gdal.GDT_CFloat64
NOTE: Geotransforms
NOTE: Projections
NOTE: Masks
-
pyeo.raster_manipulation.
apply_array_image_mask
(array, mask, fill_value=0)¶ Applies a mask of (y,x) to an image array of (bands, y, x). Replaces any masked pixels with fill_value Mask is an a 2 dimensional array of 1 ( unmasked) and 0 (masked)
-
pyeo.raster_manipulation.
apply_band_function
(in_path, function, bands, out_path, out_datatype=5)¶ Applys an arbitrary band mathemtics function to an image at in_path and saves the result at out_map. Function should be a function ofblect of the form f(band_input_A, band_input_B, …)
-
pyeo.raster_manipulation.
apply_fmask
(in_safe_dir, out_file, fmask_command='fmask_sentinel2Stacked.py')¶ Calls fmask to create a new mask for L1 data
-
pyeo.raster_manipulation.
apply_sen2cor
(image_path, sen2cor_path, delete_unprocessed_image=False)¶ Applies sen2cor to the SAFE file at image_path. Returns the path to the new product.
-
pyeo.raster_manipulation.
atmospheric_correction
(in_directory, out_directory, sen2cor_path, delete_unprocessed_image=False)¶ Applies Sen2cor cloud correction to level 1C images
-
pyeo.raster_manipulation.
average_images
(raster_paths, out_raster_path, geometry_mode='intersect', format='GTiff', datatype=5)¶ When provided with a list of rasters, will stack them into a single raster. The nunmber of bands in the output is equal to the total number of bands in the input. Geotransform and projection are taken from the first raster in the list; there may be unexpected behavior if multiple differing proejctions are provided.
- Parameters
raster_paths – A list of paths to the rasters to be stacked, in order.
out_raster_path – The path to the saved output raster.
geometry_mode – Can be either ‘instersect’ or ‘union’. - If ‘intersect’, then the output raster will only contain the pixels of the input rasters that overlap. - If ‘union’, then the output raster will contain every pixel in the outputs. Layers without data will have their pixel values set to 0.
format – The GDAL image format for the output.
datatype – The datatype of the gdal array
-
pyeo.raster_manipulation.
buffer_mask_in_place
(mask_path, buffer_size)¶ Expands a mask in-place, overwriting the previous mask
-
pyeo.raster_manipulation.
build_sen2cor_output_path
(image_path, timestamp, version)¶ Creates a sen2cor output path dependent on the version of sen2cor :param image_path: Path to the image :param timestamp: Timestamp that processing was started (required for v >= 2.8) :param version: String of the version of sen2cor (ex “2.05.05”)
- Returns
- Return type
The path of the finished sen2cor file
-
pyeo.raster_manipulation.
clip_raster
(raster_path, aoi_path, out_path)¶ Clips a raster at raster_path to a shapefile given by aoi_path. Assumes a shapefile only has one polygon. Will np.floor() when converting from geo to pixel units and np.absolute() y resolution form geotransform. Will also reproject the shapefile to the same projection as the raster if needed.
- Parameters
raster_path – Path to the raster to be clipped.
aoi_path – Path to a shapefile containing a single polygon
out_path – Path to a location to save the final output raster
-
pyeo.raster_manipulation.
combine_masks
(mask_paths, out_path, combination_func='and', geometry_func='intersect')¶ ORs or ANDs several masks. Gets metadata from top mask. Assumes that masks are a Python true or false. Also assumes that all masks are the same projection for now.
-
pyeo.raster_manipulation.
composite_directory
(image_dir, composite_out_dir, format='GTiff', generate_date_images=False)¶ - Using composite_images_with_mask, creates a composite containing every image in image_dir. This will
place a file named composite_[last image date].tif inside composite_out_dir
- Parameters
image_dir – The directory containing the rasters and associated .msk files to be composited.
composite_out_dir – The directory that will contain the final composite
format – The raster format of the output image.
generate_date_images – If true, generates a corresponding date image for the composite. See docs for composite_images_with_mask.
-
pyeo.raster_manipulation.
composite_images_with_mask
(in_raster_path_list, composite_out_path, format='GTiff', generate_date_image=False)¶ Works down in_raster_path_list, updating pixels in composite_out_path if not masked.
- Parameters
in_raster_path_list – A list of paths to rasters.
composite_out_path – The path of the output image
format – The gdal format of the image.
generate_date_image – If true, generates a single-layer raster containing the dates of each image detected.
- Returns
- Return type
The path to the composite.
Notes
Masks are assumed to be a multiplicative .msk file with the same path as their corresponding image; see REFERENCE. All images must have the same number of layers and resolution, but do not have to be perfectly on top of each other. If it does not exist, composite_out_path will be created. Takes projection, resolution, ect from first band of first raster in list. Will reproject images and masks if they do not match initial raster.
-
pyeo.raster_manipulation.
create_mask_from_class_map
(class_map_path, out_path, classes_of_interest, buffer_size=0, out_resolution=None)¶ Creates a mask from a classification mask: 1 for each pixel containing one of classes_of_interest, otherwise 0
-
pyeo.raster_manipulation.
create_mask_from_confidence_layer
(l2_safe_path, out_path, cloud_conf_threshold=0, buffer_size=3)¶ Creates a multiplicative binary mask where cloudy pixels are 0 and non-cloudy pixels are 1. If cloud_conf_threshold = 0, use scl mask else use confidence image
-
pyeo.raster_manipulation.
create_mask_from_model
(image_path, model_path, model_clear=0, num_chunks=10, buffer_size=0)¶ Returns a multiplicative mask (0 for cloud, shadow or haze, 1 for clear) built from the model at model_path.
-
pyeo.raster_manipulation.
create_matching_dataset
(in_dataset, out_path, format='GTiff', bands=1, datatype=None)¶ Creates an empty gdal dataset with the same dimensions, projection and geotransform as in_dataset. Defaults to 1 band. Datatype is set from the first layer of in_dataset if unspecified
- Parameters
in_dataset – A gdal.Dataset object
out_path – The path to save the copied dataset to
format – The Ggal image format. Defaults to geotiff (“GTiff”); for a full list, see https://gdal.org/drivers/raster/index.html
bands – The number of bands in the dataset. Defaults to one.
datatype – The datatype of the returned dataset. See the introduction for this module.
- Returns
- Return type
An gdal.Dataset of the new, empty dataset that is ready for writing.
-
pyeo.raster_manipulation.
create_new_image_from_polygon
(polygon, out_path, x_res, y_res, bands, projection, format='GTiff', datatype=5, nodata=-9999)¶ Returns an empty image that covers the extent of the imput polygon.
- Parameters
polygon – An OGR.Geometry object of a single polygon
out_path – The path to save the new image to
x_res – Pixel width in the new image
y_res – Pixel height in the new image
bands – Number of bands in the new image.
projection – The projection, in wkt, of the output image.
format – The gdal raster format of the output image
datatype – The gdal datatype of the output image
nodata – The nodata value of the output image
- Returns
- Return type
A gdal.Image object
-
pyeo.raster_manipulation.
create_new_stacks
(image_dir, stack_dir)¶ For each granule present in image_dir Saves the result in stacked_dir. Assumes that each image in image_dir is saved with a Sentinel-2 identifiter name - see merge_raster.
- Parameters
image_dir – A path to the directory containing the images to be stacked, all named as Sentinel 2 identifiers
stack_dir – A path to a directory to save the stacked images to.
- Returns
- Return type
A list of paths to the new stacks, each names as
Notes
The pairing algorithm is as follows: Step 1: Group directory b Step 2: For each tile number, sort by time Step 3: For each image in the sorted list, stack wach image with it’s next oldest image.
-
pyeo.raster_manipulation.
filter_by_class_map
(image_path, class_map_path, out_map_path, classes_of_interest, out_resolution=10)¶ Filters a raster with a set of classes for corresponding for pixels in filter_map_path containing only classes_of_interest. Assumes that filter_map_path and class_map_path are same resolution and projection.
- Parameters
image_path – Path to the imaage to be filtered
class_map_path –
out_map_path –
classes_of_interest –
out_resolution –
-
pyeo.raster_manipulation.
flatten_probability_image
(prob_image, out_path)¶ Takes a probability output from classify_image and flattens it into a single layer containing only the maximum value from each pixel.
- Parameters
prob_image – The path to a probability image.
out_path – The place to save the flattened image.
-
pyeo.raster_manipulation.
get_image_resolution
(image_path)¶ Returns the resolution of the image in its native projection. Assumes square pixels.
-
pyeo.raster_manipulation.
get_masked_array
(raster, mask_path)¶ Returns a numpy.mask masked array for the raster. Masked pixels are FALSE in the mask image (multiplicateive map), but TRUE in the masked_array (nodata pixels). If the raster is multi-band and the mask is single-band, the mask will be applied to every raster. :param raster: A gdal.Image object :param mask_path: The path to the mask to use
- Returns
- Return type
A numpy.masked array of the raster.
-
pyeo.raster_manipulation.
get_sen2cor_version
(sen2cor_path)¶ Gets the version number of sen2cor from the help string. :param sen2cor_path: Path the the sen2cor executable
- Returns
- Return type
A string of the version of sen2cor at sen2cor_path
-
pyeo.raster_manipulation.
get_sen_2_band_path
(l2_safe_dir, band, resolution=None)¶ Returns the path to the raster of the specified band in the specified safe_dir.
-
pyeo.raster_manipulation.
mosaic_images
(raster_paths, out_raster_file, format='GTiff', datatype=5, nodata=0)¶ Mosaics multiple images with the same number of layers into one single image. Overwrites overlapping pixels with the value furthest down raster_paths. Takes projection from the first raster.
- Parameters
raster_paths – A list of paths of raster to be mosaiced
out_raster_file – The path to the output file
format – The image format of the output raster.
datatype – The datatype of the output raster
nodata – The input nodata value; any pixels in raster_paths with this value will be ignored.
-
pyeo.raster_manipulation.
open_dataset_from_safe
(safe_file_path, band, resolution='10m')¶ Opens a dataset given a safe file. Give band as a string.
-
pyeo.raster_manipulation.
preprocess_landsat_images
(image_dir, out_image_path, new_projection=None)¶ Stacks a set of Landsat images into a single raster and reorders the bands into [bands, y, x] - by default, Landsat uses [x,y] and bands are in seperate rasters.
-
pyeo.raster_manipulation.
preprocess_sen2_images
(l2_dir, out_dir, l1_dir, cloud_threshold=60, buffer_size=0, epsg=None, bands=('B08', 'B04', 'B03', 'B02'), out_resolution=10)¶ For every .SAFE folder in in_dir, stacks band 2,3,4 and 8 bands into a single geotif, creates a cloudmask from the combined fmask and sen2cor cloudmasks and reprojects to a given EPSG if provided
-
pyeo.raster_manipulation.
raster_sum
(inRstList, outFn, outFmt='GTiff')¶ Creates a raster stack from a list of rasters. Adapted from Chris Gerard’s book ‘Geoprocessing with Python’. The out put data type is the same as the input data type.
- Parameters
inRstList – List of rasters to stack.
outFmt – String specifying the input data format e.g. ‘GTiff’ or ‘VRT’.
outFn – Filename output as str including directory else image will be written to current working directory.
-
pyeo.raster_manipulation.
raster_to_array
(rst_pth)¶ Reads in a raster file and returns a N-dimensional array.
- Parameters
rst_pth – Path to input raster.
- Returns
- Return type
As N-dimensional array.
-
pyeo.raster_manipulation.
reproject_directory
(in_dir, out_dir, new_projection, extension='.tif')¶ Reprojects every file ending with extension to new_projection and saves in out_dir
- Parameters
in_dir – A directory containing the rasters to be reprojected/
out_dir – The directory to save the output files to. Output files will be saved in out_dir, with the same filenames.
new_projection – The new projection in wkt.
extension – The file extension to reproject
-
pyeo.raster_manipulation.
reproject_image
(in_raster, out_raster_path, new_projection, driver='GTiff', memory=2000.0, do_post_resample=True)¶ Creates a new, reprojected image from in_raster using the gdal.ReprojectImage function.
- Parameters
in_raster – Either a gdal.Raster object or a path to a raster
out_raster_path – The path to the new output raster.
new_projection – The new projection in .wkt
driver – The format of the output raster.
memory – The amount of memory to give to the reprojection.
do_post_resample – If set to false, do not resample the image back to the original projection.
Notes
The GDAL reprojection routine changes the size of the pixels by a very small amount; for example, a 10m pixel image can become a 10.002m pixel resolution image. To stop alignment issues, by deafult this function resamples the images back to their original resolution
-
pyeo.raster_manipulation.
resample_image_in_place
(image_path, new_res)¶ Resamples an image in-place using gdalwarp to new_res in metres. WARNING: This will make a permanent change to an image! Use with care.
- Parameters
image_path – Path to the image to be resampled
new_res – Pixel edge size in meters
-
pyeo.raster_manipulation.
save_array_as_image
(array, path, geotransform, projection, format='GTiff')¶ Saves a given array as a geospatial image to disk in the format ‘format’. The datatype will be of one corresponding to Array must be gdal format: [bands, y, x].
- Parameters
array – A Numpy array containing the values to be saved to a raster
path – The path to the location to save the output raster to
geotransform – The geotransform of the image to be saved. See note.
projection – The projection, as wkt, of the image to be saved. See note.
format –
-
pyeo.raster_manipulation.
stack_and_trim_images
(old_image_path, new_image_path, aoi_path, out_image)¶ Stacks an old and new S2 image and trims to within an aoi. :param old_image_path: Path to the image that will be the first set of bands in the output image :param new_image_path: Path to the image that will be the second set of bands in the output image :param aoi_path: Path to a shapefile containing the AOI :param out_image: Path to the location of the clipped and stacked image.
-
pyeo.raster_manipulation.
stack_image_with_composite
(image_path, composite_path, out_dir, create_combined_mask=True, skip_if_exists=True, invert_stack=False)¶ Creates a single 8-band geotif image with a cloud-free composite, and saves the result in out_dir. Images are named “composite_tile_timestamp-of-composite_timestamp-of-image”. Bands 123 and 4 are the BGR and I bands of the composite, and bands 567 and 8 are the BGR and I bands of the image.
- Parameters
image_path – Path to the image to be stacked .
composite_path – Path to the composite to stack the image with
out_dir – The directory to save the resulting composite to
create_combined_mask – If true, combines the cloud mask files associated with the images into a single mask. The mask will mask out clouds that exist in either image.
skip_if_exists – If true, skip stacking if a file with the same name is found.
invert_stack – If true, changes the ordering of the bands to image BGRI - composite BGRI. Included to permit compatability with older models - you can ususally leave this alone.
- Returns
- Return type
The path to the new composite.
-
pyeo.raster_manipulation.
stack_images
(raster_paths, out_raster_path, geometry_mode='intersect', format='GTiff', datatype=5)¶ When provided with a list of rasters, will stack them into a single raster. The nunmber of bands in the output is equal to the total number of bands in the input. Geotransform and projection are taken from the first raster in the list; there may be unexpected behavior if multiple differing proejctions are provided.
- Parameters
raster_paths – A list of paths to the rasters to be stacked, in order.
out_raster_path – The path to the saved output raster.
geometry_mode – Can be either ‘instersect’ or ‘union’. - If ‘intersect’, then the output raster will only contain the pixels of the input rasters that overlap. - If ‘union’, then the output raster will contain every pixel in the outputs. Layers without data will have their pixel values set to 0.
format – The GDAL image format for the output.
datatype – The datatype of the gdal array
-
pyeo.raster_manipulation.
stack_old_and_new_images
(old_image_path, new_image_path, out_dir, create_combined_mask=True)¶ Stacks two images with the same tile Names the result with the two timestamps. First, decompose the granule ID into its components: e.g. S2A, MSIL2A, 20180301, T162211, N0206, R040, T15PXT, 20180301, T194348 are the mission ID(S2A/S2B), product level(L2A), datatake sensing start date (YYYYMMDD) and time(THHMMSS), the Processing Baseline number (N0206), Relative Orbit number (RO4O), Tile Number field (T15PXT), followed by processing run date and then time
-
pyeo.raster_manipulation.
stack_sentinel_2_bands
(safe_dir, out_image_path, bands=('B02', 'B03', 'B04', 'B08'), out_resolution=10)¶ Stacks the specified bands of a .SAFE granule directory into a single geotiff
-
pyeo.raster_manipulation.
trim_image
(in_raster_path, out_raster_path, polygon, format='GTiff')¶ Trims a raster to a polygon.
- Parameters
in_raster_path – Path to the imput raster
out_raster_path – Path of the output raster
polygon – A ogr.Geometry containing a single polygon
format – Image format of the output raster. Defaults to geotiff.