pysal/pysal: pysal 2.4.0

PySAL 2.4.0 represents 6 months of enhancements, bug-fixes, widening of test coverage, and improved documentation. Three Google Summer of Code projects made substantial contributions to this release (see below). All users are encouraged to upgrade to this version as there are numerous optimizations as well as new features (see below) that have been implemented.

Overall, there were 1155 commits that closed 352 issues, together with 121 pull requests since our last release on 2020-07-29.

Highlights New Package: pysal/spopt

Provides methods for solving optimization problems with spatial data. Currently, regionalization methods are supported with planned support for facility location and transportation-oriented modeling.

The regionalization models implemented include:

• Max-p-regions: It involves the aggregation or clustering of a set of small areas into the maximum number of homogeneous and spatially contiguous regions such that the value of a regional attribute is higher than a predefined threshold. The number of regions will be endogenized in order to satisfy the threshold. (Duque, Anselin, and Rey, 2012 and Wei, Rey, and Knaap, 2020)
• Automatic Zoning Procedure (AZP) algorithm: It can work with any type of objective function that is sensitive to the aggregation of data for a larger number of zones into a pre-specified smaller number of regions (Openshaw, 1977 and Openshaw and Rao, 1995)
• Region-K-means: It is K-means clustering for regions with the constraint that each cluster forms a spatially connected component.
• Skater: It is a constrained spatial regionalization algorithm based on spanning tree pruning. Specifically, the number of edges is prespecified to be cut in a continuous tree to group spatial units into contiguous regions. (AssunCao et al., 2006)
• Spenc: spatially-encouraged spectral clustering is an approach to balance spatial and feature coherence using kernel combination in spectral clustering.([Wolf, 2020] (https://osf.io/yzt2p/))
• WardSpatial: It is an Agglomerative Clustering using Ward linkage with a spatial connectivity constraint. Basically, it is a "bottom-up" approach: each observation starts in its own cluster, and pairs of clusters are chosen to merge at each step in order to minimize the variance of the clusters. (sklearn.cluster.AgglomerativeClustering)

Enhancements to existing packages libpysal

Thanks to the Google Summer of Code 2020 led by Mragank Shekhar (@MgeeeeK), we have built an interface with xarray that allows you to create spatial weights matrices from xarray.DataArray objects. This means it is possible to build a weights matrix from raster data. We are using numba and multi-core where possible so the implementation is performant and scalable within memory limits. Current weights supported include:

• libpysal.weights.Rook.from_xarray
• libpysal.weights.Queen.from_xarray

In addition, the following methods have been added and are exposed to end users:

• [RECOMMENDED] da2WSP, to build a thin weights matrix (WSP) from a xarray.DataArray da2W, to build a weights matrix from a xarray.DataArray
• [RECOMMENDED] wsp2da, to reconstruct a xarray.DataArray object from a thin weights matrix and a pandas.Series (e.g. with mapclassify or esda.Moran_Local outcomes
• w2da, to reconstruct a xarray.DataArray object from a weights matrix and a pandas.Series (e.g. with mapclassify or esda.Moran_Local outcomes testDataArray, to generate a toy xarray.DataArray object for testing purposes.

esda

• Integration of the work of @jeffcsauer's 2020 GSOC project. This adds tons of new statistical estimators, such as:
  • the local heteroskedasticity estimator, esda.LOSH
  • local geary and multivariate geary estimators, esda.Geary_Local and esda.Geary_Local_MV
  • local join counts in univariate, bivariate, and multivariate flavors, esda.Join_Counts_Local, esda.Join_Counts_Local_BV, and esda.Join_Counts_Local_MV.
• "analytical" moments for Moran's I for replication/comparison to R. Forms are provided in Sokal 1998.

spreg

Thanks to the Google Summer of Code 2020 led by paboloestrada, we have added the following functionality:

• Panel_RE_Lag: random effects panel estimation with spatial lagged dependent variable.
• Panel_RE_Error: random effects panel estimation with spatial error interaction.
• Lagrange Multiplier test: classic and robust version.
• Hausman test.

spaghetti

The highlights of this release include a bug fix for how network segments were being split (raised in #526) and additions to several notebooks (spatial network segmentation, caveats, and network-constrained spatial autocorrelation). Also, spaghetti is now tested against Python 3.9.

tobler

• interpolation to hexagons using h3f
• more efficient unary_union in h3f

segregation

• segregation measures on network distances

mapclassify

• streamlined api

Detailed Changes by Package

<a name="libpysal"></a>

libpysal

• #385: Adding raster interface to docs API + notebooks
• #384: [Doc]: Update raster example notebook and docstrings
• #383: [WIP]: id_order as a property of WSP class
• #343: [WIP]: Optimized raster-based weights builder
• #382: remove dup matplotlib in environment.yml
• #366: fix typo
• #376: Bump PYPI version to 4.3.1
• #381: adjust duplicated rst link in README
• #380: rst link & syntax issue in README
• #379: Correct readme.rst
• #378: Troubleshoot release GHA
• #377: bump to v4.3.2 for release action
• #374: Adding deps for examples.
• #373: update unittests.yml
• #372: release action is broken due to gh deprecation of env var
• #371: Fix release action
• #370: Bump actions/setup-python from v2.1.4 to v2.2.0
• #364: examples.explain returning 404
• #368: This address a move in remote datasets
• #335: ENH: use query_bulk in fuzzy_contiguity
• #334: BUG: fuzzy_contiguity picks more neighbours than it should
• #367: ENH: use active geometry in from_dataframe
• #342: standardizing libpysal/io docs & ibpysal/cg docs
• #363: Testing matrix
• #359: Bump codecov/codecov-action from v1.0.14 to v1.0.15
• #358: Bump actions/checkout from v2.3.3 to v2.3.4
• #355: Add codecov.yml for customized reports/testing
• #357: Add codecov.yml
• #286: Blacken codebase
• #291: [WIP] DEV: blacken codebase
• #354: Bump codecov/codecov-action from v1.0.13 to v1.0.14
• #257: street network-based weights
• #351: Bump actions/setup-python from v2.1.3 to v2.1.4
• #279: GHA for PyPI release
• #324: Update PR template
• #347: ENH: include lower order contiguities in higher_order_sp
• #313: higher_order weights of <= k
• #348: Bump actions/setup-python from v2.1.2 to v2.1.3
• #345: Add xarray to requirements_plus_conda.txt
• #346: Bump actions/checkout from v2.3.2 to v2.3.3
• #344: Bump actions/create-release from v1.1.3 to v1.1.4
• #318: [WIP] : Base Raster Interface
• #339: Bump codecov/codecov-action from v1.0.12 to v1.0.13
• #338: Update actions/checkout requirement to v2.3.2
• #336: Bump actions/create-release from v1.1.2 to v1.1.3
• #337: Bump actions/setup-python from v2.0.1 to v2.1.2
• #326: Updating PULL_REQUEST_TEMPLATE.md
• #298: Add github action for releasing
• #316: adding release_and_publish.yml
• #333: Rook contiguity weights fails when polygons border at two points

<a name="access"></a>

access

• #10: capitalize Access class
• #15: Changes case on Access and Datasets classess
• #2: Ensure that the meta import for access exposes intended classes and functions
• #3: Confirm access version to freeze in meta
• #11: sphinx enforced to install w/ packages
• #12: The package does not depend on scipy
• #13: Adjust dependenies
• #1: Release on conda-forge

<a name="esda"></a>

esda

• #160: bump version for release
• #157: Use tags now in gitcount
• #139: [ENH][DOC] local join count and LOSH statistics
• #143: document minimum numba version
• #159: Moments for Moran's I_i following Sokal 1998
• #145: [ENH][DOC] local Geary statistics
• #158: Bump actions/setup-python from v1 to v2.2.1
• #156: Bump to 2.3.5
• #155: Bump version
• #154: Prepping for 2.3.2 release.
• #142: Issue with Numba < 0.47
• #146: parallelcrand ValueError
• #147: fix concatenation for random statistics
• #153: Unexpected LISA results for polygons contained by another polygon
• #152: Dropping 3.6 from testing.
• #151: CI: update conda setup
• #150: Bump codecov/codecov-action from v1.0.14 to v1.0.15
• #149: Bump codecov/codecov-action from v1.0.13 to v1.0.14
• #144: Bump codecov/codecov-action from v1.0.12 to v1.0.13
• #141: Spatial cross-correlations?

<a name="giddy"></a>

giddy

<a name="inequality"></a>

inequality

<a name="pointpats"></a>

pointpats

<a name="segregation"></a>

segregation

• #159: bump ver
• #79: add network, util, and profile functions to readthedocs
• #158: update docs
• #153: update docstring for decomposition plotting
• #157: Update release action for deprecation of set env
• #156: update github actions
• #155: bump to Version 1.4
• #154: add docstring to decomp plotting
• #152: more flexible decomp plotting
• #151: move testing to gha
• #148: col names hardcoded in _generate_counterfactual
• #150: updates docs link
• #149: Website is not rendering correctly

<a name="spaghetti"></a>

spaghetti

• #572: bump 1.5.5 --> v1.5.6 after syntax error
• #571: Prep 1.5.4 rel
• #569: new documentation build GHA schema
• #568: keep docs/notebooks directory
• #556: Revert "Python 3.9 testing for CI"
• #567: documentation maintenance and notebook generation
• #566: update docs build process
• #558: Is sphinx_gallery needed for docs?
• #565: remove sphinx_gallery from reqs
• #564: Update unittests.yml (GitHub Action)
• #557: revisit #555 (adding Python 3.9 testing)
• #555: Python 3.9 testing for CI
• #552: update GHA workflows and drop Py3.6 Windows testing
• #551: Bump codecov/codecov-action from v1.0.14 to v1.0.15
• #548: Request inclusion for GitHub Discussions.
• #546: Bump actions/checkout from v2.3.3 to v2.3.4
• #101: Network K Functionality
• #543: changed binder's link to reflect the new name of master branches from…
• #544: Update environment.yml
• #437: [JOSS] time to start thinking about a JOSS paper
• #542: minor edits and updated figure references
• #541: preparing JOSS paper for submission
• #540: Libpysal req
• #539: forgot to sync docs for v1.5.1 release
• #538: Version bump for bug fix (#535)
• #535: Vertex/Arc ID sorting bug in Network.split_arcs()
• #537: Bump codecov/codecov-action from v1.0.13 to v1.0.14
• #526: Strange count_per_link() results for networks generated using ntw.split_arcs()
• #536: Bump actions/setup-python from v2.1.3 to v2.1.4
• #533: Clarify the Network._extractnetwork() creation procedure
• #534: Add more documentation for Network._extractnetwork()
• #532: Network creation splits arcs at each node
• #531: Network parameter extractgraph has no effect
• #528: Correcting unit typos and errors in notebooks
• #529: updating landing page images
• #525: Wrong distance units in split_arcs description
• #524: Bump actions/setup-python from v2.1.2 to v2.1.3
• #523: update docstrings and rebuild docs
• #521: bump Python version for notebooks
• #520: Bump actions/checkout from v2.3.2 to v2.3.3
• #519: Bump actions/create-release from v1.1.3 to v1.1.4
• #518: updating test_nearest_neighbor_distances()
• #517: remove black check in unittest
• #516: updating Python version in pre-commit-config.yaml
• #515: Bump codecov/codecov-action from v1.0.12 to v1.0.13
• #514: Bump actions/create-release from v1.1.2 to v1.1.3
• #513: Bump actions/setup-python from v2.1.1 to v2.1.2
• #512: Bump actions/checkout from v2.3.1 to v2.3.2

<a name="mgwr"></a>

mgwr

• #86: version bump
• #83: check_constant change
• #85: model.predict => AttributeError: 'list' object has no attribute 'shape'

<a name="spglm"></a>

spglm

• #30: update log
• #29: version bump
• #28: fix check_constant returns

<a name="spint"></a>

spint

• #26: version/log bump
• #25: update travis

<a name="spreg"></a>

spreg

• #54: Docs To Do
• #63: Missing classes in API docs
• #64: notebook and API docs cleanup
• #60: Add tutorials to docs
• #56: Travis --> GHA
• #55: CI with GitHub Actions
• #53: Migrate from RTD to GH-pages for automated doc builds and hosting
• #52: update env variable syntax
• #47: Spatial dependence diagnostics returns nan
• #46: Random Effects for Spatial Panels
• #50: Random Effects Panel
• #38: Add GM_Lag_example notebook
• #44: Fixed Effects Panel - Spatial Error model
• #45: Fixed Effects Panel - Spatial Error
• #49: Not being able to release to PyPI using release_and_publish.yml
• #48: fix doctest of check_constant

<a name="spvcm"></a>

spvcm

<a name="tobler"></a>

tobler

• #129: bump version
• #128: badges out of date
• #127: update docs
• #94: remove accidental notebook commit
• #121: h3fy example/dep
• #126: more efficient unary_union in h3fy
• #124: hexify multipolygons
• #125: h3fy function fails if input contains multipolygons
• #123: BUG: fix parallel binning
• #122: Untitled notebook
• #104: numba/parallel implementation of _area_tables_binning
• #112: [WIP] STRTree parallel implementation
• #119: ENH: add area_join for join based on the largest intersection
• #101: Pick attribute based on largest intersection
• #120: add missing deps for example notebooks.
• #117: maintenance/admin for setup/README/etc.
• #111: add hexify function
• #116: Update & enhance syntax/options in GitHub Actions
• #115: move dasymetric test back into test folder
• #114: GHA for docs
• #113: add h3fy to docs
• #110: REF: use STRtree.query_bulk in _area_tables_binning
• #109: fix path in release workflow
• #108: increment version
• #107: increment version number
• #106: update gitcount notebook
• #105: prepare 0.5.0 release
• #103: dont hardcode geometry name
• #102: BUG: area_interpolate fails in geometry column not named "geometry"
• #98: update setup-miniconda
• #99: Refactor test suite to be more modular for quicker development
• #11: enhance readme
• #27: Assume Poisson in the XGBoost approach
• #82: Reorganize repository
• #83: Add a tutorial to the documentation site
• #92: area interpolation question
• #91: don try to init a df if columns are missing
• #89: bumping release version and disambiguation of names
• #90: deps to run notebooks added to requirements_docs.txt
• #88: fix docs
• #87: Tutorial
• #86: reorg notebooks
• #85: BUG: have to reset_index on geometry after interpolation
• #70: Faster raster
• #81: DOC: Update notebooks for api change
• #72: jupyter example
• #69: add tests for intensive, extensive, and both
• #68: Problem running area_interpolate with both intensive and extensive vars
• #6: Consider usage of data.store_rasters() for examples
• #80: fix concatenation issue and copy inputs

<a name="mapclassify"></a>

mapclassify

• #103: 2.4.2 prep
• #102: Update change log and version
• #104: REGR: invariant array fix
• #99: BUG: support series in sampled classifiers
• #98: BUG: FisherJenksSampled returns ValueError if Series is passed as y
• #101: REGR: fix invariant array regression
• #100: REGR: UserDefined classifier returns ValueError("Minimum and maximum of input data are equal, cannot create bins.")
• #91: [DOC] add example nb for new classify API
• #97: 2.4.0 Release
• #96: Remove timeout on tests.
• #92: BUG: HeadTailBreaks RecursionError due to floating point issue
• #95: Handle recursion error for head tails.
• #72: Add streamlined API
• #90: [API] add high-level API mapclassify.classify()
• #89: BUG: Fix mapclassify #88
• #94: exclude Python 3.6 for Windows
• #93: CI: update conda action
• #88: EqualInterval unclear error when max_y - min_y = 0
• #87: BUG: fix unordered series in greedy
• #86: BUG: greedy(strategy='balanced') does not return correct labels

<a name="splot"></a>

splot

<a name="spopt"></a>

spopt

• #138: fresh cut release?
• #139: bump version 0.1.0 -> 0.1.1
• #135: High level description of spopt needed for meta release change log
• #137: Fixing import structure
• #118: availability on conda-forge
• #134: spopt is now on conda-forge
• #136: bump v0.1.0
• #69: Installing for Google Collab
• #119: manual release on PYPI
• #132: pulp omitted from requirements
• #133: pulp to reqs
• #128: spopt sub-sub modules not included on PyPI?
• #131: Include all packages.
• #129: add token user to pypi step
• #126: revert to old pysal-styles.ccs
• #127: Revert pysal-styles.ccs, etc.
• #125: Narrative for notebook and handling a single attribute.
• #124: update docs & README
• #115: Add Zenodo DOI
• #116: pre-releases for workflow testing, etc.
• #123: Add Zenodo DOI and citation
• #122: update changelog notebook
• #121: fix changelog notebook
• #120: bump version to v0.0.2
• #117: bump initial version down to 0.0.1
• #113: extra reqs for binder environment + binder badge
• #114: Update notebook env
• #111: Polish up and minor docs/reqs fixes
• #110: various tasks for package admin/cleanup
• #109: Update sphinxcontrib-bibtex requirement from <2.0.0 to <3.0.0
• #107: update formatting on Installation
• #108: Update installation.rst
• #106: corrections for README
• #102: README is incomplete
• #105: change python version in installation.rst
• #104: add docs badge to README.md
• #103: yml-->yaml
• #99: updated docs handling
• #51: Funding information in README.md
• #56: adding Support info.
• #98: Adding support information.
• #100: add images for funding
• #97: remove ortools
• #39: Implementation
• #33: inventory models for consideration
• #37: identify models for implementation in region
• #40: Release Polishing
• #54: consider moving spenc to spopt
• #73: attempt 3.9 testing
• #95: update for 39
• #96: Why was ci/ changed to .ci/?
• #89: replace deleted logos
• #90: Add back images that were deleted
• #92: revise pictures
• #94: handle deprecation of discretize in skl.
• #93: Add spenc as a lib rather than using via an import
• #91: Environment for development.
• #64: change < to ==
• #66: add ward and test_ward
• #83: update .rst files
• #88: add description of models
• #86: update docs and docscr
• #87: Update GitHub Actions
• #76: Python error (Ubuntu): Segmentation fault (core dumped)
• #85: update methods and notebooks
• #75: add azp into spopt
• #84: Exclude failing 3.8 on ubuntu due to seg falt.
• #79: An issue about "module 'spopt' has no attribute 'version'"
• #81: migrate spenc code
• #80: add spenc to requirements.txt
• #68: rename master branch to main branch
• #74: Updating links from master branch to main branch
• #60: Issue with PuLP for Python 3.8 on conda-forge
• #72: Request inclusion for GitHub Discussions.
• #71: attempt 3.9 testing
• #70: Issues caused by GitHub Actions: Deprecating set-env and add-path commands
• #67: add spenc and test_spenc
• #65: skater
• #62: removing python-mip language from REAMDE
• #63: an error with the maxp code
• #59: adding .github/dependabot.yml
• #58: Setup correction
• #55: Updating README.md & adding CONTRIBUTING.md

<a name="pysal"></a>

pysal

• #1217: add spopt description in changelog
• #1215: 2.4.0rc2
• #1214: 2.4.0rc2
• #1211: AttributeError: module 'pysal' has no attribute 'common'
• #1213: Update changelog instructions
• #1212: 2.4.0rc1
• #1169: Import problem in Google Colab
• #1137: conda install does not install plus requirements
• #1139: Raise warnings on use, not import
• #1133: AttributeError: module 'pysal' has no attribute 'open' -- Python3 on Fedora
• #1092: integrate/incorporate geodacenter/spatial_access
• #1179: Migrate CI to GHA and pytest
• #1186: draft rc1 release notes
• #1154: multiplatform testing
• #1208: spopt now on conda-forge
• #1209: Update changelog building and grab spopt from conda-forge not pip
• #1210: wrong link to access in v2.3.0 Release Notes
• #1207: removing codecov action & streamlining CI
• #1205: remove codecov
• #1206: Update README for spopt and new project homepage
• #1202: Update tooling to handle master or main branches in ecosystem
• #1204: 2.4.0
• #1203: updating urllib3 to 1.26 minimum
• #1201: Python 3.7 pip install has an error
• #1200: ModuleNotFoundError: No module named 'pysal.model.mgwr.gwr'
• #1185: dependency conflicts on 3.8
• #1060: Handling coincident points
• #678: W stitch
• #1146: Improved documentation...
• #1196: GM_Lag
• #1195: splot seaborn failure
• #1193: Image missing from readme on pypi
• #1194: updating image link in README.md
• #1192: DOCS: 2.3.0
• #1191: 2.3.0
• #1189: Subpackages

<a name="contributors"></a>

Contributors

Many thanks to all of the following individuals who contributed to this release:

• Andrew Annex
• Dani Arribas-Bel
• Eli Knaap
• James Gaboardi
• Jamie Saxon
• Jeffcsauer
• Jkoschinsky
• Levi John Wolf
• Martin Fleischmann
• Mragank Shekhar
• Omar
• Pabloestradac
• Pedro Amaral
• Rwei5
• Serge Rey
• Stefanie Lumnitz
• Taylor Oshan
• Vidal Anguiano Jr
• Wei Kang
• Xin Feng
• Ziqi Li