PyHelpers: an open-source toolkit for facilitating Python users' data manipulation tasks

PyHelpers is an open-source Python package designed as a lite toolkit for facilitating data (pre)processing. It offers a miscellaneous collection of handy utilities, which could assist us in performing many common data manipulation tasks, such as reading/writing of file-like objects, handling of various types of data (e.g. geographical data and textual data) and communication with relational databases (e.g. PostgreSQL).

 

This release:

1.3.0 (6 January 2022)

      Notable changes since 1.2.18:

• Renamed the module sql to dbms.
• Renamed the following functions:
  • in the module dir:
    • validate_input_data_dir() to validate_dir();
  • in the module ops:
    • update_nested_dict() to update_dict();
    • get_all_values_from_nested_dict() to get_dict_values();
    • remove_multiple_keys_from_dict() to remove_dict_keys();
    • instantiate_requests_session() to init_requests_session().
• Improved the following functions/class:
  • PostgreSQL in the module dbms (previously sql);
  • wgs84_to_osgb36() and osgb36_to_wgs84() in the module geom.
  • parse_size() (newly added), init_requests_session() and download_file_from_url() in the module ops;
  • load_json() in the module store;
  • find_similar_str() in the module text;
• Added the following functions to the module ops: parse_size(), get_number_of_chunks(), loop_in_pairs(), is_url() and update_dict_keys().
• Reduced the number of essential dependencies.

      For more information and detailed specifications, check out PyHelpers 1.3.0 documentation.

 

Release history:

1.2.18 (20 October 2021)

      Notable changes since 1.2.17:

• Improved the module ops by:
  • fixing the issue in fake_requests_headers() that may occasionally raise IndexError;
  • adding a new function get_fake_user_agent().

      For more information and detailed specifications, check out PyHelpers 1.2.18 documentation.

1.2.17 (1 October 2021)

      Notable changes since 1.2.16:

• Improved the function fake_requests_headers() to avoid raising IndexError.

      For more information and detailed specifications, check out PyHelpers 1.2.17 documentation.

1.2.16 (20 September 2021)

      Notable changes since 1.2.15:

• Renamed the function is_dirname() to is_dir() in the module dir.
• Improved the following functions (with bug fixes):
  • cd() in the module dir;
  • find_closest_points() in the module geom;
  • confirmed(), swap_cols(), swap_rows(), fake_requests_headers() and download_file_from_url() in the module ops;
  • all the functions in the module settings;
  • save_multiple_spreadsheets(), save_json() and load_json() in the module store;
  • find_similar_str() in the module text.
• Added the following new functions:
  • go_from_altered_cwd() to the module dir;
  • drop_axis() to the module geom;
  • instantiate_requests_session() to the module ops;
  • save_joblib() and load_joblib() to the module store.

      For more information and detailed specifications, check out PyHelpers 1.2.16 documentation.

1.2.15 (19 April 2021)

      Notable changes since 1.2.14:

• Improved the following functions/methods with bug fixes:
  • find_similar_str() and convert_md_to_rst() in the module text;
  • download_file_from_url() in the module ops;
  • save_svg_as_emf() in the module store;
  • PostgreSQL.import_data() and PostgreSQL.drop_schema() in the module sql.
• Added a new function is_downloadable() to the module ops.

      For more information and detailed specifications, check out PyHelpers 1.2.15 documentation.

1.2.14 (27 March 2021)

      Notable changes since 1.2.13:

• Rectified an error in the specification of dependencies for package installation.

      For more information and detailed specifications, check out PyHelpers 1.2.14 documentation.

1.2.13 (26 March 2021)

      Notable changes since 1.2.12:

• Improved the module sql with a bug fix for errors raised when importing SQLAlchemy 1.4.0+.

      For more information and detailed specifications, check out PyHelpers 1.2.13 documentation.

1.2.12 (22 March 2021)

      Notable changes since 1.2.11:

• Resolved an issue that failed importing modules (after updating or reinstalling the package).
• Added a new function gps_to_utc() to the module ops.

      For more information and detailed specifications, check out PyHelpers 1.2.12 documentation.

1.2.11 (3 March 2021)

      Notable changes since 1.2.10:

• Fixed a bug in the method PostgreSQL.import_data() in the module sql.

      For more information and detailed specifications, check out PyHelpers 1.2.11 documentation.

1.2.10 (1 February 2021)

      Notable changes since 1.2.9:

• Improved the following functions/method with bug fixes to:
  • np_preferences() in the module settings;
  • save_multiple_spreadsheets() in the module store;
  • PostgreSQL.read_sql_query() in the module sql.
• Added new functions/methods:
  • eval_dtype(), swap_cols() and swap_rows() to the module ops;
  • get_rectangle_centroid() to the module geom;
  • .list_table_names() and .alter_table_schema() to the class PostgreSQL in the module sql.

      For more information and detailed specifications, check out PyHelpers 1.2.10 documentation.

1.2.9 (5 January 2021)

      Note that the release 1.2.8 had been permanently deleted.

      Notable changes since 1.2.7:

• Made modifications with bug fixes to the following functions/class:
  • validate_input_data_dir() in the module dir;
  • confirmed() and fake_requests_headers() in the module ops;
  • PostgreSQL in the module sql.
• Added new functions:
  • get_obj_attr() to the module ops;
  • get_db_address() to the module sql.

1.2.7 (17 November 2020)

      Notable changes since 1.2.6:

• Fixed a minor bug in the method PostgreSQL.import_data() in the module sql
• Added new functions:
  • merge_dicts(), is_network_connected() and is_url_connectable() to the module ops;
  • get_acronym() and extract_words1upper() to the module text.

1.2.6 (6 November 2020)

      Notable changes since 1.2.5:

• Renamed two functions in the module geom:
  • find_closest_point_from() to find_closest_point();
  • find_closest_points_between() to find_closest_points().
• Reduced the number of essential dependencies for installing the package (see also the installation note).
• Enabled direct access to all functions/classes from importing the package without having to specifying the modules they reside in.
• Fixed a minor bug in the function fake_requests_headers() in the module ops.

1.2.5 (12 October 2020)

      Notable changes since 1.2.4:

• Renamed the function rm_dir() to delete_dir() in the module dir.
• Moved the function save_web_page_as_pdf() from the module text to store.
• Removed two functions from the module ops: get_variable_name() and get_variable_names().
• Improved the following function/class:
  • save() in the module store;
  • PostgreSQL in the module sql.

1.2.4 (7 September 2020)

      Notable changes since 1.2.3:

• Improved the function fake_requests_headers() in the module ops.

1.2.3 (7 September 2020)

      Notable changes since 1.2.2:

• Fixed a few bugs in the following functions/class:
  • download_file_from_url() in the module ops;
  • get_specific_filepath_info() in the module store;
  • PostgreSQL in the module sql.
• Added a new function fake_requests_headers() to the module ops.

1.2.2 (18 July 2020)

      Notable changes since 1.2.1:

• Improved the following functions:
  • get_specific_filepath_info() in the module store;
  • split_list() and get_all_values_from_nested_dict() in the module ops.

1.2.1 (1 July 2020)

      Note that the release 1.2.0 had been permanently deleted.

      Notable changes since 1.1.2:

• Integrated the module download into ops and made the following changes:
  • renamed the function download() to download_file_from_url();
  • added a new function convert_md_to_rst();
  • improved the following functions: split_list_by_size(), split_list(), get_extreme_outlier_bounds(),
    interquartile_range(), find_closest_date() and detect_nan_for_str_column().
• Moved the function save_web_page_as_pdf() from the module store to text.
• Improved the following functions/class:
  • get_specific_filepath_info(), save_fig(), save_svg_as_emf(), seven_zip() and load_csr_matrix() in the module store;
  • PostgreSQL in the module sql.

1.1.2 (30 May 2020)

      Note that the release 1.1.1 had been permanently deleted.

      Notable changes since 1.1.0:

• Renamed the following function/method:
  • save_excel() to save_spreadsheet() in the module store;
  • .disconnect() to .disconnect_database() of the class PostgreSQL in the module sql.
• Removed the function csr_matrix_to_dict() from the module text.
• Removed redundant parameters from functions in the module store.
• Improved the following functions by adding optional parameters:
  • cd(), cdd(), cd_data and rm_dir() in the module dir;
  • download() in the module download;
  • save_svg_as_emf() and save_web_page_as_pdf() in the module store.
• Added new functions:
  • to the module geom: sketch_square();
  • to the module ops: create_rotation_matrix() and dict_to_dataframe();
  • to the module store: save_multiple_spreadsheets(), load_multiple_spreadsheets(),
    unzip(), seven_zip() and load_csr_matrix();
  • to the module text: remove_punctuation(), count_words(), calculate_idf(), calculate_tf_idf(),
    euclidean_distance_between_texts() and cosine_similarity_between_texts().

1.1.0 (22 April 2020)

      Notable changes since 1.0.26:

• Made a few modifications to the class sql.PostgreSQL to speed up importing data into, and retrieving data from, a PostgreSQL database.
• Fixed a minor bug in the function text.find_matched_str().
• Improved the two modules store and text.

1.0.26 (12 March 2020)

      Notable changes since 1.0.25:

• Fixed a few bugs in the module store.

1.0.25 (6 March 2020)

      Notable changes since 1.0.24:

• Improved the functions, osgb36_to_wgs84() and wgs84_to_osgb36(), in the module geom.

1.0.24 (28 January 2020)

      Note that the release 1.0.23 had been permanently deleted.

      Notable changes since 1.0.22:

• Fixed a bug in the module sql.

1.0.22 (27 January 2020)

      Notable changes since 1.0.21:

• Renamed the function divide_list_into_chunks() to split_list_by_size() in the module ops.
• Added two new functions to the module ops: split_list() and split_iterable().
• Improved the following function/class:
  • gdal_configurations() in the module settings;
  • PostgreSQL in the module sql.

1.0.21 (20 January 2020)

      Notable changes since 1.0.20:

• Added a new module sql.
• Added a new function detect_nan_for_str_column() to the module ops.

1.0.20 (7 January 2020)

      Notable changes since 1.0.19:

• Fixed a few minor bugs in the module store.

1.0.19 (28 November 2019)

      Note that the release 1.0.18 had been permanently deleted.

      Notable changes since 1.0.17:

• Renamed the module misc to ops; moved the function csr_matrix_to_dict() to the module text.
• Fixed minor bugs:
  • save_feather() and load_feather() in the module store;
  • download() in the module download;
  • mpl_preferences() in the module settings.
• Added new functions:
  • show_square(), locate_square_vertices() and locate_square_vertices_calc() to the module geom;
  • is_dirname() to the module dir.
• Improved the module store with modifications to the functions: save_pickle(), load_pickle(), save_json(), load_json() and save().

1.0.17 (10 September 2019)

      Notable changes since 1.0.16:

• Improved the function rm_dir() in the module dir.
• Tidied up the code of the three modules: misc, geom and store.

1.0.16 (3 September 2019)

      This is a release of a brand-new version.

      Note that the initial releases (of early versions up to 1.0.15) had been permanently deleted.