sherpa: Sherpa 4.8.0

Release Notes Sherpa 4.8.0

This version of Sherpa introduces 'wstat' statistics which is an extension of 'cstat' with Poisson background data. It also provides the ability to include background data with the 'user statistics'.

Many changes were aimed at improving the Xspec extension by making it more robust, intuitive, and by fixing several bugs. Sherpa 4.8.0 supports Xspec 12.9.0d and was also tested against versions 12.8.2e, 12.8.2l, and 12.8.2q.

Most of the codebase was reviewed and cleaned up, in particular to remove the use of deprecated functionality and to comply with Python's PEP8 standard. More tests were added to the test suite, and a new testing infrastructure was put in place in order to simplify writing and running tests, and to measure the test suite code coverage.

More details below (infrastructure changes are not shown):

32: Fix segfault from CRATES update in 4.8b1. Since v4.8b1 CRATES returns

variable length arrays by default, rather than the zero-padded fixed length ones it used to return. Sherpa manipulated the arrays so to remove the zero-padding and obtaining variable length arrays. The change in the CRATES API resulted in Sherpa segfaulting when trying to manipulate the data coming from CRATES. In the patch, we use a new API offered by CRATES to get the old-style fixed-length arrays instead of the new default ones. In the future, we may want to update the Sherpa code to deal with the new arrays directly.

44: save_quality now correctly outputs 'QUALITY' as the column name,

instead of 'GROUPS'.

48: Fix up several issues seen in plot labels - titles and Y-axis labels -

for commands such as sherpa.ui.plot_data, sherpa.ui.plot_fit_resid, and sherpa.ui.plot_chisqr.

59: Fix bug #38 (grouping twice gives an IndexError exception). An un-handled

corner case in one of the Sherpa internal methods (utils.create_expr) was triggering an IndexError when two group_counts operations were performed back to back. The fix handles the case so that applying group_counts twice does not result in an Exception.

77: Replace == and != comparisons to None with is and is not. 78: OutOfBoundErr exceptions in some sherpa.utils

functions are properly caught. There were several places where screen output used either print or sys.stderr.write.

81: Ensure that XSPEC models which fail - for instance, because a data file

it needs is missing - return 0's for all bins, rather than random values. This should make it more obvious that something has gone wrong (for instance if the XSPEC chatter level is low enough not to show any error messages, as is the case for the default setting used by Sherpa, namely 0).

82: The XSpec "spectrum number" value is now set to 1 rather than 0, as this

value is 1-based in Xspec.

83: Removed S-Lang scripts, files, and references in the code. 84: Clarified error messages in Xspec extension. Also, changed the class of

the exception from RuntimeError to more appropriate exception types, in particular LookupError, ValueError, KeyError. This is a backwards-incompatible change, in that code that caught the RuntimeError will not catch the new error.

87: Some methods in sherpa/fit.py assigned mutable objects to default

arguments. This has now been fixed. More instances of this issue have been identified (Bug #95) and will removed in the future.

90: Added background data to the UserStat class. 94: Implement wstat statistic as described at the following url:

https://heasarc.gsfc.nasa.gov/xanadu/xspec/manual/XSappendixStatistics.html

96: Remove the unused myoptfct module. 99: Correct the documentation for the set_exposure function. 100 Fix bug #97, whereby the save_all function would not create the necessary

set_source() line. This does not fix all cases, but it does the simple PHA case such as

load_pha('src.pi') set_source(xsphabs.gal * powlaw1d.pl) save_all('test.out).

It also ensures that files created by save_all can be run using IPython's %run directive, by explicitly importing the sherpa.astro.ui module.

101: Fix handling of non-contiguous bins in Xspec - i.e. when a model is

called with both xlo and xhi arguments but the bins do not fully cover the energy, or wavelenth, range. This fixes #62 (for XSPEC 12.8.2; switching to XSPEC 12.9.0 should also fix it) and #56. It also fixes an (un-reported) problem with handling of non-contiguous grids when using a table model, where a crash was likely. When an XSPEC model is called with both low and high values for the grid - i.e. with two arguments - then the two arrays are checked to have the same length, and a ValueError is raised if this condition does not hold. This is a breaking change, but the results are not guaranteed to be correct if the two arrays are not the same length. The experimental interface for XSPEC convolution models has changed, so that the function call takes pars, fluxes, xlo, with optional xhi whereas before it was pars, xlo, xhi, fluxes. This is a breaking change, but this is in the low-level API that is not documented to users, and adds useful functionality (the ability to have xhi be optional). The cpflux convolution model has been added. Note that these models do not have Python classes associated with them as they are still an experimental interface. The test suite has been updated to test the new and changed functionality in this PR. The choice of models is made so as to avoid known problematic models (with a version check where relevant). It is believed that the changes in this PR fixes #42, although this is hard to prove conclusively given the erratic nature of the bug.

110: Update the sherpa.astro.datastack module documentation to include information from the

CIAO ahelp documentation and to match the style used by the sherpa.astro.ui module.

111: Update the documentation to include more information about the pyBLoCXS code. Caveats

These caveats are being fixed for the 4.8.1 release.

Incompatibility with matplotlib 1.5: Sherpa 4.8.0 is not compatible with matplotlib 1.5. Unfortunately, this version is currently the default package installed by conda. Users should install sherpa with matplotlib=1.4 numpy=1.9.

Test requirements are not installed automatically: sherpa_test does not work out of the box. Users should issue "pip install pytest-cov" in order for sherpa_test to run.

SAO DS9 issue on Ubuntu 14.04: the ds9 binaries shipped with Ubuntu and installed through apt-get install do not seem to work as expected. Binaries downloaded directly from the SAO ds9 page seem to work instead.