PyDriosm: an open-source tool for downloading, reading and PostgreSQL-based I/O of OpenStreetMap data

PyDriosm is an open-source tool for researchers/practitioners to easily download and read OpenStreetMap (OSM) data in popular file formats such as protobuf binary format (PBF) and shapefile, which are available for free download from Geofabrik and BBBike. The package also provides a convenient way for PostgreSQL-based I/O and storage of parsed OSM data.

This release:

2.1.1 (9 January 2022)

      Notable changes since 2.1.0:

• Improved the following methods/modules (with bug fixes):
  • the methods .write_to_shapefile(), .read_layer_shps(), and .merge_layer_shps() of the class SHPReadParse;
  • the modules downloader and _updater.

      For more information and detailed specifications, check out PyDriosm 2.1.1 documentation.

Release history:

2.1.0 (20 November 2022)

      Note that this release introduces a highly modified version, which is not compatible with any previous versions.

      Notable changes since 2.0.3:

• Made major modifications and sweeping changes to the modules: downloader, reader, ios and utils.
• Removed the module settings.
• Replaced the module updater with _updater.
• Added a new module errors.

      For more information and detailed specifications, check out PyDriosm 2.1.0 documentation.

2.0.3 (25 April 2021)

      Notable changes since 2.0.2:

• Renamed the function get_default_shp_crs() to get_epsg4326_wgs84_crs_ref() in the module reader.
• Removed the function get_osm_geom_object_dict() from the module utils
• Changed the default package for reading/writing shapefiles from GeoPandas to PyShp (Note that GeoPandas would not be required for installing PyDriosm 2.0.3+ but would still be reserved as an alternative option if available.
• Modified the default download directories.
• Improved the class PostgresOSM in the module ios and the module downloader with bug fixes.
• Added the following new functions:
  • get_epsg4326_wgs84_prj_ref(), specify_pyshp_fields() and write_to_shapefile() to the module reader;
  • shp_shape_types_dict() and shp_shape_types_geom_dict() to the module utils.

      For more information and detailed specifications, check out PyDriosm 2.0.3 documentation.

2.0.2 (24 November 2020)

      Notable changes since 2.0.1:

• Added a new parameter max_tmpfile_size for setting the maximum size of in-memory temporary file while instantiating the classes GeofabrikReader and BBBikeReader for reading OSM data.
• Added a new function validate_shp_layer_names() to the module utils.
• Optimized import statements for all modules.

      For more information and detailed specifications, check out PyDriosm 2.0.2 documentation.

2.0.1 (19 November 2020)

      Notable changes since 2.0.0:

• Optimized import statements for the modules downloader and reader.

2.0.0 (19 November 2020)

      Note that this release is a highly modified version and not compatible with any previous versions.

      Notable changes since 1.0.20:

• Featured with the following three new modules:
  • downloader, modified from the former download_GeoFabrik and download_BBBike, for downloading data;
  • reader, modified from the former read_GeoFabrik, for reading the data;
  • ios, modified from the former osm_psql and dump_GeoFabrik, for PostgreSQL-based I/O and storage of the data.
• Renamed the rest of the modules, fixed known bugs and added a number of new functions/classes.

1.0.20 (27 January 2020)

      Note that 1.0.19 had been removed from GitHub Releases.

      Notable changes since 1.0.18:

• Removed the function split_list() from the module utils.
• Improved the following class and functions with bug fixes:
  • OSM in the module osm_psql;
  • regulate_input_subregion_name() in download_GeoFabrik;
  • read_pbf() and read_pbf() in the module read_GeoFabrik;
  • psql_osm_pbf_data_extracts() in dump_GeoFabrik.

1.0.18 (9 January 2020)

      Notable changes since 1.0.17:

• Integrated the function collect_bbbike_subregion_download_catalogue() into collect_bbbike_download_catalogue() in the module download_BBBike.
• Modified the module download_GeoFabrik, allowing it to download data of a deep or shallow set of subregions.
• Improved the following functions with bug fixes:
  • get_subregion_table() in the module download_GeoFabrik
  • find_osm_shp_file(), merge_multi_shp() and read_pbf() in the module read_GeoFabrik
• Added a new module update.
• Added default parameters for PostgreSQL database connection.

1.0.17 (29 November 2019)

      Notable changes since 1.0.16:

• Improved the following functions with bug fixes:
  • collect_subregion_info_catalogue() and get_default_pathname() in the module download_GeoFabrik;
  • merge_multi_shp(), read_shp_zip() and read_pbf() in the module read_GeoFabrik.

1.0.16 (6 October 2019)

      Notable changes since 1.0.15:

• Fixed some known bugs.

1.0.15 (29 August 2019)

      Note that 1.0.14, 1.0.13 and 1.0.12 had been removed from GitHub Releases.

      Notable changes since 1.0.11:

• Improved the functions: extract_shp_zip(), read_shp() and read_shp_zip() with bug fixes in the module read_GeoFabrik.
• Added a new method OSM.db_exists(), allowing OSM.create_db() to check whether a database exists.
• Updated the LICENSE.

1.0.11 (4 April 2019)

      Note that 1.0.10 and 1.0.9 had been removed from GitHub Releases.

      Notable changes since 1.0.8:

• Fixed a minor bug for creating a default data directory.
• Improved the following functions (with bug fixes):
  • get_default_pathname() in the module download_GeoFabrik;
  • parse_layer_data() in the module read_GeoFabrik;
  • dump_osm_pbf_data_by_layer() in the module osm_psql;
  • psql_osm_pbf_data_extracts() in the module dump_GeoFabrik, with a new parameter database_name for specifying a database name when dumping data to a PostgreSQL server.
• Added a function regulate_table_name(), which regulates PostgreSQL table names, to the module osm_psql.

1.0.8 (18 Mar 2019)

      Note that 1.0.6 and 1.0.7 had been removed from GitHub Releases.

      Notable changes since 1.0.5:

• Made some major changes (with potential bug fixes).
• Fixed minor bugs in the following functions:
  • parse_layer_data() and read_shp_zip() in the module read_GeoFabrik;
  • retrieve_subregions() and psql_subregion_osm_data_extracts() in the module dump_GeoFabrik.
• Added a function regulate_input_data_dir() to the module utils.

1.0.5 (11 March 2019)

      Note that 1.0.4, 1.0.3, 1.0.2 and 1.0.1 had been removed from GitHub Releases.

      Notable changes since 1.0.0:

• Integrated the function read_parsed_osm_pbf() into read_pbf() in the module read_GeoFabrik.
• Improved the following functions
  • dump_osm_pbf_data() in the module osm_psql, with a new parameter chunk_size allowing users to parse/read/dump data in a chunk-wise way;
  • psql_subregion_osm_data_extracts() in the module dump_GeoFabrik;
  • gdal_configurations() in the module settings.
• Added new function:
  • retrieve_subregions(), which retrieves a list of subregions of a given (sub)region name, to the module dump_GeoFabrik;
  • split_list() to the module utils.

1.0.0 (4 March 2019)

      This is a release of a brand-new version.

      Note that the initial releases (of early versions up to 0.2.9) had been permanently deleted.