pyeo.filesystem_utilities

Contains functions for sorting, creating and comparing images as part of the filesystem. Includes any function that works on a generic pyeo timestamp and sentinel 2 functions.

Key functions

init_log() Sets up logging to both console and file

create_file_structure() Creates a recommended file structure for automated work

sort_by_timestamp() Sorts a set of files by timestamp

Function reference

pyeo.filesystem_utilities.check_for_invalid_l1_data(l1_SAFE_file)

Checks the existance of the specified resolution of imagery. Returns True with a warning if passed an invalid SAFE directory; this will prevent disconnected files from being deleted.

Parameters

l1_SAFE_file (str) – Path to the L1 file to check

Returns

result – 1 if imagery is valid, 0 if not and 2 if not a safe-file

Return type

int

pyeo.filesystem_utilities.check_for_invalid_l2_data(l2_SAFE_file, resolution='10m')

Checks the existence of the specified resolution of imagery. Returns a True-value with a warning if passed an invalid SAFE directory; this will prevent disconnected files from being deleted.

Parameters
  • l2_SAFE_file (str) – Path to the L2A file to check

  • resolution ({"10m", "20m", "60m"}) – The resolution of imagery to check. Defaults to 10m.

Returns

result – 1 if imagery is valid, 0 if not and 2 if an invalid .SAFE file

Return type

int

pyeo.filesystem_utilities.clean_aoi(aoi_dir, images_to_keep=4, warning=True)

Removes all but the last images_to_keep newest images in the L1, L2, merged, stacked and composite directories. Will not affect the output folder. Use with caution.

Parameters
  • aoi_dir (str) – A directory made by create_file_structure()

  • images_to_keep (int, optional) – The number of images to keep

pyeo.filesystem_utilities.clean_l2_data(l2_SAFE_file, resolution='10m', warning=True)

Removes a safe file if it doesn’t have bands 2, 3, 4 or 8 in the specified resolution folder. If warning=True, prompts before removal.

Parameters

l2_SAFE_file (str) – Path to the L2 .SAFE file

pyeo.filesystem_utilities.clean_l2_dir(l2_dir, resolution='10m', warning=True)

Calls clean_l2_data on every SAFE file in l2_dir

Parameters
  • l2_dir (str) – The L2 directory

  • resolution ({"10m", "20m", "60m"}, optional) – Resolution to check. Defaults to 10m

  • warning (bool, optional) – If True, prompts user before deleting files.

pyeo.filesystem_utilities.create_file_structure(root)
Creates the folder structure used in rolling_s2_composite and some other functions: ::

root -images –L1 –L2 –merged –stacked -composite –L1 –L2 –merged -output –categories –probabilities

Parameters

root (str) – The root folder for the file strucutre

pyeo.filesystem_utilities.get_change_detection_dates(image_name)

Extracts the before_date and after_date dates from a change detection image name.

Parameters

image_name (str) – A Pyeo-produced image name (example: class_composite_T36MZE_20190509T073621_20190519T073621.tif)

Returns

before_date, after_date – The dates associated with the image

Return type

DateTime

pyeo.filesystem_utilities.get_image_acquisition_time(image_name)

Gets the datetime object from a .safe filename of a Sentinel image. No test. Returns None if no timestamp present :param image_name: The .SAFE filename :type image_name: str

Returns

acquisition_time – A DateTime object providing the acquisition time

Return type

DateTime

pyeo.filesystem_utilities.get_l1_safe_file(image_name, l1_dir)

Returns the path to the L1 .SAFE file of a L2 image. Gets from granule and timestamp. image_name can be a path or a filename. Returns None if not found.

Parameters
  • image_name (str) – The name of the L2 image.

  • l1_dir (str) – The path to the folder containing L1 images.

Returns

l1_image – The path to the L1 image, or None if not found.

Return type

str or None

pyeo.filesystem_utilities.get_l2_safe_file(image_name, l2_dir)

Returns the path to the L2 .SAFE file of a L1 image. Gets from granule and timestamp. image_name can be a path or a filename. Returns None if not found.

Parameters
  • image_name (str) – The name of the L1 image.

  • l2_dir (str) – The path to the folder containing L2 images.

Returns

l2_image – The path to the L2 image, or None if not found.

Return type

str or None

pyeo.filesystem_utilities.get_mask_path(image_path)

Returns the path to the mask file of the image. Does not verify that the mask exists. :param image_path: Path to the image :type image_path: str

Returns

mask_path – Path to the corresponding mask.

Return type

str

pyeo.filesystem_utilities.get_preceding_image_path(target_image_name, search_dir)

Finds the image that directly precedes the target image in time.

Parameters
  • target_image_name (str) – A Pyeo or Sentinel generated image name

  • search_dir (str) – The directory to search for the preceding image.

Returns

preceding_path – The path to the preceding image

Return type

str

Raises

FileNotFoundError – If there is no image older than the target image

pyeo.filesystem_utilities.get_pyeo_timestamp(image_name)

Returns a list of all timestamps in a Pyeo image (yyyymmddhhmmss)

Parameters

image_name (str) – The Pyeo image name

Returns

timestamp_list – A list of the timestamps in the name

Return type

list of str

Finds every image related to the target image (L1, L2, merged, stacked and classified). Based on timestamp, and assumes a project structured by create_file_structure() :param target_image_name: The target image :type target_image_name: str :param project_dir: The root of the project directory :type project_dir: str

Returns

related_images – A list of all the images related to target_image

Return type

list of str

pyeo.filesystem_utilities.get_safe_product_type(image_name)

Returns the product string (MSIL1C or MSIL2A) from a .safe file identifier :param image_name: The name of the image :type image_name: str

Returns

product_string – The product string

Return type

{“MSIL1C” or “MSIL2A”}

pyeo.filesystem_utilities.get_sen_2_baseline(image_name)

“Returns the baseline orbit identifier of a Sentinel 2 image :param image_name: The Sentinel image name or path :type image_name: str

Returns

baseline – The baseline orbit identifier (eg: ‘N0214’)

Return type

str

pyeo.filesystem_utilities.get_sen_2_granule_id(safe_dir)

Returns the unique ID of a Sentinel 2 granule from a SAFE directory path

Parameters

safe_dir (str) – Path to a .SAFE file

Returns

id – The full ID of that S2 product (eg: ‘S2B_MSIL2A_20180103T172709_N0206_R012_T13QFB_20180103T192359’)

Return type

str

pyeo.filesystem_utilities.get_sen_2_image_orbit(image_name)

Returns the relative orbit identifer of a Sentinel 2 image

Parameters

image_name (str) – The Sentinel image name or path

Returns

orbit_number – The orbit. (eg: ‘R012’)

Return type

str

pyeo.filesystem_utilities.get_sen_2_image_tile(image_name)

Returns the tile number of a Sentinel 2 image or path :param image_name: The Sentinel image name or path :type image_name: str

Returns

tile_id – The tile ID (eg: ‘T13QFB’)

Return type

str

pyeo.filesystem_utilities.get_sen_2_image_timestamp(image_name)

Returns the timestamps part of a Sentinel image

Parameters

image_name (str) – The Sentinel image name or path

Returns

timestamp – The timestamp (yyyymmddThhmmss)

Return type

str

pyeo.filesystem_utilities.get_sen_2_tiles(image_dir)

Returns a list of Sentinel-2 tile IDs present in a folder

Parameters

image_dir (str) – Path to the folder containing images

Returns

tiles – A list of the tile identifiers present in the folder

Return type

list of str

pyeo.filesystem_utilities.init_log(log_path)

Sets up the log format and log handlers; one for stdout and to write to a file, ‘log_path’. Returns the log for the calling script. :param log_path: The path to the file output of this log. :type log_path: str

Returns

log – The logging object.

Return type

logging.Logger

pyeo.filesystem_utilities.sort_by_timestamp(strings, recent_first=True)

Takes a list of strings that contain sen2 timestamps and returns them sorted, from most recent on back by default. Removes any string that does not contain a timestamp

Parameters
  • strings (list of str) – list of timestamped strings to sort

  • recent_first (bool, optional) – If True, sorted_strings[0] is the most recent image. If False, sorted_strings[0] is the least recent image.

Returns

sorted_strings – The strings sorted by timestamp.

Return type

list of str

Notes

Does not guarantee preservation of ordering of strings with the same timestamp.