scipy/scipy: SciPy 1.7.0

SciPy 1.7.0 Release Notes

SciPy 1.7.0 is the culmination of 6 months of hard work. It contains many new features, numerous bug-fixes, improved test coverage and better documentation. There have been a number of deprecations and API changes in this release, which are documented below. All users are encouraged to upgrade to this release, as there are a large number of bug-fixes and optimizations. Before upgrading, we recommend that users check that their own code does not use deprecated SciPy functionality (to do so, run your code with python -Wd and check for DeprecationWarning s). Our development attention will now shift to bug-fix releases on the 1.7.x branch, and on adding new features on the master branch.

This release requires Python 3.7+ and NumPy 1.16.5 or greater.

For running on PyPy, PyPy3 6.0+ is required.

Highlights of this release

• A new submodule for quasi-Monte Carlo, scipy.stats.qmc, was added
• The documentation design was updated to use the same PyData-Sphinx theme as NumPy and other ecosystem libraries.
• We now vendor and leverage the Boost C++ library to enable numerous improvements for long-standing weaknesses in scipy.stats
• scipy.stats has six new distributions, eight new (or overhauled) hypothesis tests, a new function for bootstrapping, a class that enables fast random variate sampling and percentile point function evaluation, and many other enhancements.
• cdist and pdist distance calculations are faster for several metrics, especially weighted cases, thanks to a rewrite to a new C++ backend framework
• A new class for radial basis function interpolation, RBFInterpolator, was added to address issues with the Rbf class.

We gratefully acknowledge the Chan-Zuckerberg Initiative Essential Open Source Software for Science program for supporting many of the improvements to scipy.stats.

New features scipy.cluster improvements

An optional argument, seed, has been added to kmeans and kmeans2 to set the random generator and random state.

scipy.interpolate improvements

Improved input validation and error messages for fitpack.bispev and fitpack.parder for scenarios that previously caused substantial confusion for users.

The class RBFInterpolator was added to supersede the Rbf class. The new class has usage that more closely follows other interpolator classes, corrects sign errors that caused unexpected smoothing behavior, includes polynomial terms in the interpolant (which are necessary for some RBF choices), and supports interpolation using only the k-nearest neighbors for memory efficiency.

scipy.linalg improvements

An LAPACK wrapper was added for access to the tgexc subroutine.

scipy.ndimage improvements

scipy.ndimage.affine_transform is now able to infer the output_shape from the out array.

scipy.optimize improvements

The optional parameter bounds was added to _minimize_neldermead to support bounds constraints for the Nelder-Mead solver.

trustregion methods trust-krylov, dogleg and trust-ncg can now estimate hess by finite difference using one of ["2-point", "3-point", "cs"].

halton was added as a sampling_method in scipy.optimize.shgo. sobol was fixed and is now using scipy.stats.qmc.Sobol.

halton and sobol were added as init methods in scipy.optimize.differential_evolution.

differential_evolution now accepts an x0 parameter to provide an initial guess for the minimization.

least_squares has a modest performance improvement when SciPy is built with Pythran transpiler enabled.

When linprog is used with method 'highs', 'highs-ipm', or 'highs-ds', the result object now reports the marginals (AKA shadow prices, dual values) and residuals associated with each constraint.

scipy.signal improvements

get_window supports general_cosine and general_hamming window functions.

scipy.signal.medfilt2d now releases the GIL where appropriate to enable performance gains via multithreaded calculations.

scipy.sparse improvements

Addition of dia_matrix sparse matrices is now faster.

scipy.spatial improvements

distance.cdist and distance.pdist performance has greatly improved for certain weighted metrics. Namely: minkowski, euclidean, chebyshev, canberra, and cityblock.

Modest performance improvements for many of the unweighted cdist and pdist metrics noted above.

The parameter seed was added to scipy.spatial.vq.kmeans and scipy.spatial.vq.kmeans2.

The parameters axis and keepdims where added to scipy.spatial.distance.jensenshannon.

The rotation methods from_rotvec and as_rotvec now accept a degrees argument to specify usage of degrees instead of radians.

scipy.special improvements

Wright's generalized Bessel function for positive arguments was added as scipy.special.wright_bessel.

An implementation of the inverse of the Log CDF of the Normal Distribution is now available via scipy.special.ndtri_exp.

scipy.stats improvements Hypothesis Tests

The Mann-Whitney-Wilcoxon test, mannwhitneyu, has been rewritten. It now supports n-dimensional input, an exact test method when there are no ties, and improved documentation. Please see "Other changes" for adjustments to default behavior.

The new function scipy.stats.binomtest replaces scipy.stats.binom_test. The new function returns an object that calculates a confidence intervals of the proportion parameter. Also, performance was improved from O(n) to O(log(n)) by using binary search.

The two-sample version of the Cramer-von Mises test is implemented in scipy.stats.cramervonmises_2samp.

The Alexander-Govern test is implemented in the new function scipy.stats.alexandergovern.

The new functions scipy.stats.barnard_exact and scipy.stats. boschloo_exact respectively perform Barnard's exact test and Boschloo's exact test for 2x2 contingency tables.

The new function scipy.stats.page_trend_test performs Page's test for ordered alternatives.

The new function scipy.stats.somersd performs Somers' D test for ordinal association between two variables.

An option, permutations, has been added in scipy.stats.ttest_ind to perform permutation t-tests. A trim option was also added to perform a trimmed (Yuen's) t-test.

The alternative parameter was added to the skewtest, kurtosistest, ranksums, mood, ansari, linregress, and spearmanr functions to allow one-sided hypothesis testing.

Sample statistics

The new function scipy.stats.differential_entropy estimates the differential entropy of a continuous distribution from a sample.

The boxcox and boxcox_normmax now allow the user to control the optimizer used to minimize the negative log-likelihood function.

A new function scipy.stats.contingency.relative_risk calculates the relative risk, or risk ratio, of a 2x2 contingency table. The object returned has a method to compute the confidence interval of the relative risk.

Performance improvements in the skew and kurtosis functions achieved by removal of repeated/redundant calculations.

Substantial performance improvements in scipy.stats.mstats.hdquantiles_sd.

The new function scipy.stats.contingency.association computes several measures of association for a contingency table: Pearsons contingency coefficient, Cramer's V, and Tschuprow's T.

The parameter nan_policy was added to scipy.stats.zmap to provide options for handling the occurrence of nan in the input data.

The parameter ddof was added to scipy.stats.variation and scipy.stats.mstats.variation.

The parameter weights was added to scipy.stats.gmean.

Statistical Distributions

We now vendor and leverage the Boost C++ library to address a number of previously reported issues in stats. Notably, beta, binom, nbinom now have Boost backends, and it is straightforward to leverage the backend for additional functions.

The skew Cauchy probability distribution has been implemented as scipy.stats.skewcauchy.

The Zipfian probability distribution has been implemented as scipy.stats.zipfian.

The new distributions nchypergeom_fisher and nchypergeom_wallenius implement the Fisher and Wallenius versions of the noncentral hypergeometric distribution, respectively.

The generalized hyperbolic distribution was added in scipy.stats.genhyperbolic.

The studentized range distribution was added in scipy.stats.studentized_range.

scipy.stats.argus now has improved handling for small parameter values.

Better argument handling/preparation has resulted in performance improvements for many distributions.

The cosine distribution has added ufuncs for ppf, cdf, sf, and isf methods including numerical precision improvements at the edges of the support of the distribution.

An option to fit the distribution to data by the method of moments has been added to the fit method of the univariate continuous distributions.

Other

scipy.stats.bootstrap has been added to allow estimation of the confidence interval and standard error of a statistic.

The new function scipy.stats.contingency.crosstab computes a contingency table (i.e. a table of counts of unique entries) for the given data.

scipy.stats.NumericalInverseHermite enables fast random variate sampling and percentile point function evaluation of an arbitrary univariate statistical distribution.

New scipy.stats.qmc module

This new module provides Quasi-Monte Carlo (QMC) generators and associated helper functions.

It provides a generic class scipy.stats.qmc.QMCEngine which defines a QMC engine/sampler. An engine is state aware: it can be continued, advanced and reset. 3 base samplers are available:

• scipy.stats.qmc.Sobol the well known Sobol low discrepancy sequence. Several warnings have been added to guide the user into properly using this sampler. The sequence is scrambled by default.
• scipy.stats.qmc.Halton: Halton low discrepancy sequence. The sequence is scrambled by default.
• scipy.stats.qmc.LatinHypercube: plain LHS design.

And 2 special samplers are available:

• scipy.stats.qmc.MultinomialQMC: sampling from a multinomial distribution using any of the base scipy.stats.qmc.QMCEngine.
• scipy.stats.qmc.MultivariateNormalQMC: sampling from a multivariate Normal using any of the base scipy.stats.qmc.QMCEngine.

The module also provide the following helpers:

• scipy.stats.qmc.discrepancy: assess the quality of a set of points in terms of space coverage.
• scipy.stats.qmc.update_discrepancy: can be used in an optimization loop to construct a good set of points.
• scipy.stats.qmc.scale: easily scale a set of points from (to) the unit interval to (from) a given range.

Deprecated features scipy.linalg deprecations

• scipy.linalg.pinv2 is deprecated and its functionality is completely subsumed into scipy.linalg.pinv
• Both rcond, cond keywords of scipy.linalg.pinv and scipy.linalg.pinvh were not working and now are deprecated. They are now replaced with functioning atol and rtol keywords with clear usage.

scipy.spatial deprecations

• scipy.spatial.distance metrics expect 1d input vectors but will call np.squeeze on their inputs to accept any extra length-1 dimensions. That behaviour is now deprecated.

Other changes

We now accept and leverage performance improvements from the ahead-of-time Python-to-C++ transpiler, Pythran, which can be optionally disabled (via export SCIPY_USE_PYTHRAN=0) but is enabled by default at build time.

There are two changes to the default behavior of scipy.stats.mannwhitenyu:

• For years, use of the default alternative=None was deprecated; explicit alternative specification was required. Use of the new default value of alternative, "two-sided", is now permitted.
• Previously, all p-values were based on an asymptotic approximation. Now, for small samples without ties, the p-values returned are exact by default.

Support has been added for PEP 621 (project metadata in pyproject.toml)

We now support a Gitpod environment to reduce the barrier to entry for SciPy development; for more details see :ref:quickstart-gitpod.

Authors

• @endolith
• Jelle Aalbers +
• Adam +
• Tania Allard +
• Sven Baars +
• Max Balandat +
• baumgarc +
• Christoph Baumgarten
• Peter Bell
• Lilian Besson
• Robinson Besson +
• Max Bolingbroke
• Blair Bonnett +
• Jordão Bragantini
• Harm Buisman +
• Evgeni Burovski
• Matthias Bussonnier
• Dominic C
• CJ Carey
• Ramón Casero +
• Chachay +
• charlotte12l +
• Benjamin Curtice Corbett +
• Falcon Dai +
• Ian Dall +
• Terry Davis
• droussea2001 +
• DWesl +
• dwight200 +
• Thomas J. Fan +
• Joseph Fox-Rabinovitz
• Max Frei +
• Laura Gutierrez Funderburk +
• gbonomib +
• Matthias Geier +
• Pradipta Ghosh +
• Ralf Gommers
• Evan H +
• h-vetinari
• Matt Haberland
• Anselm Hahn +
• Alex Henrie
• Piet Hessenius +
• Trever Hines +
• Elisha Hollander +
• Stephan Hoyer
• Tom Hu +
• Kei Ishikawa +
• Julien Jerphanion
• Robert Kern
• Shashank KS +
• Peter Mahler Larsen
• Eric Larson
• Cheng H. Lee +
• Gregory R. Lee
• Jean-Benoist Leger +
• lgfunderburk +
• liam-o-marsh +
• Xingyu Liu +
• Alex Loftus +
• Christian Lorentzen +
• Cong Ma
• Marc +
• MarkPundurs +
• Markus Löning +
• Liam Marsh +
• Nicholas McKibben
• melissawm +
• Jamie Morton
• Andrew Nelson
• Nikola Forró
• Tor Nordam +
• Olivier Gauthé +
• Rohit Pandey +
• Avanindra Kumar Pandeya +
• Tirth Patel
• paugier +
• Alex H. Wagner, PhD +
• Jeff Plourde +
• Ilhan Polat
• pranavrajpal +
• Vladyslav Rachek
• Bharat Raghunathan
• Recursing +
• Tyler Reddy
• Lucas Roberts
• Gregor Robinson +
• Pamphile Roy +
• Atsushi Sakai
• Benjamin Santos
• Martin K. Scherer +
• Thomas Schmelzer +
• Daniel Scott +
• Sebastian Wallkötter +
• serge-sans-paille +
• Namami Shanker +
• Masashi Shibata +
• Alexandre de Siqueira +
• Albert Steppi +
• Adam J. Stewart +
• Kai Striega
• Diana Sukhoverkhova
• Søren Fuglede Jørgensen
• Mike Taves
• Dan Temkin +
• Nicolas Tessore +
• tsubota20 +
• Robert Uhl
• christos val +
• Bas van Beek +
• Ashutosh Varma +
• Jose Vazquez +
• Sebastiano Vigna
• Aditya Vijaykumar
• VNMabus
• Arthur Volant +
• Samuel Wallan
• Stefan van der Walt
• Warren Weckesser
• Anreas Weh
• Josh Wilson
• Rory Yorke
• Egor Zemlyanoy
• Marc Zoeller +
• zoj613 +
• 秋纫 +

A total of 126 people contributed to this release. People with a "+" by their names contributed a patch for the first time. This list of names is automatically generated, and may not be fully complete.