cta-observatory/ctapipe: v0.16.0

Summary

This next major release of ctapipe includes a large number of improvements and new features, as well as significant changes to the API and data model that fix outstanding problems with both. Due to the nature of these required changes and to keep the code reasonably maintainable (in this pre-1.0 release), files written by earlier versions of ctapipe are not guaranteed to be readable with this one.

Contributors

@aknierim, @kosack, @maxnoe, @nbiederbeck, @satoshifukami0115 and @vuillaut

What has changed since v0.15.0

Pull-requests containing changes of multiple nature are repeated.

New features

• ObsConfig data model (#1973) @kosack
• Add abstract InvalidPixelHandler and NeighborAverage implementation, apply in CameraCalibrator and TwoPassWindowSum. Pass invalid pixels to extractors. (#2023) @maxnoe
• Get reference_location and optics/camera name from simtel metadata if available (#2018) @maxnoe
• Add methods to read run config and shower distribution to TableLoader (#2010) @maxnoe
• added a better template for release-drafter (#1880) @kosack
• Add another cleaning method in cleaning.py (#1672) @satoshifukami0115
• Round floats for hashing in instrument module, use __slots__ (#1986) @maxnoe
• Add location of subarray reference point to SubarrayDescription (#1980) @maxnoe
• recursively validate containers (#1976) @kosack
• Improve units code in hillas_parameters (#1988) @maxnoe
• Add TableLoader.read_telescope_events_by_type_chunked (#1984) @maxnoe
• Maintain event order of input file in TableLoader, fixes #1966 (#1968) @maxnoe
• Chunked table loader (#1935) @maxnoe
• Read dl2 in hdf5eventsource (#1940) @maxnoe
• add joined option to SubarrayDescription.to_table (#1939) @maxnoe
• compute impact distance to shower axis (#1899) @kosack
• Improved configuration of reference metadata (#1889) @kosack
• Allow passing already opened h5file to table loader (#1933) @maxnoe
• Add overwrite option to write_table (#1932) @maxnoe
• Support reading files created with extract_calibevent (#1929) @maxnoe
• improve array display (#1911) @kosack

API Changes

• Use n_ prefix consistently, (rename all num_ to n_), fixes #986 (#2046) @maxnoe
• Remove cutflow (#1766) @maxnoe
• Add pixel, channel, sample information to CameraReadout, update OpticsDescription (#2019) @maxnoe
• Remove support for by_type split files (#2032) @maxnoe
• Rework h5 metadata (#2003) @maxnoe
• Use pytest fixtures in unit tests instead of from_name methods (#1985) @maxnoe
• Store both equivalent and effective focal length in OpticsDescription, add __slots__ (#1923) @maxnoe
• Replace deepcopying defaults with default_factory (#1991) @maxnoe
• Use module of Tool to setup logging (#1964) @maxnoe
• Add QualityQuery.get_table_mask, compile python code (#1943) @maxnoe
• Load all dl2 available groups in TableLoader (#1934) @maxnoe
• Use container classes for TableReader.read method, do not reuse same container instance (#1921) @maxnoe

Data Model Changes

• ObsConfig data model (#1973) @kosack
• Rename ReconstructedContainer.tel_ids -> telescopes, fixes 1953 (#2044) @maxnoe
• Use n_ prefix consistently, (rename all num_ to n_), fixes #986 (#2046) @maxnoe
• Add pixel, channel, sample information to CameraReadout, update OpticsDescription (#2019) @maxnoe
• Rework h5 metadata (#2003) @maxnoe
• Fix datalevels (#2004) @maxnoe
• Store both equivalent and effective focal length in OpticsDescription, add __slots__ (#1923) @maxnoe
• Rework container prefix handling (#1949) @maxnoe
• Add location of subarray reference point to SubarrayDescription (#1980) @maxnoe
• compute impact distance to shower axis (#1899) @kosack

Bug Fixes

• Fix missing f-string (#2043) @nbiederbeck
• Fix columns order when string column present, fixes #2041 (#2042) @maxnoe
• Fix writing and merging of dl2 statistics, fixes #1884 (#1885) @maxnoe
• Fix pixel rotation for very large offsets in NominalFrame (#2037) @maxnoe
• Fix wrong pixel area and rotation in nominal frame, fixes #2028 (#2029) @maxnoe
• Fix error message for missing effective focal length (#2026) @maxnoe
• Remove restriction on existing file in EventSource.input_url, fixes #1821 (#1998) @maxnoe
• Fix TableLoader reading empty table (#2013) @nbiederbeck
• Fix datalevels (#2004) @maxnoe
• Include number of mirror tiles in guessing keys to be able to distinguish between MAGIC 1 and MAGIC 2 (#1977) @maxnoe
• Use correct traitlet for event type enum in eventtypefilter (#2002) @maxnoe
• Add dl2 datalevel in hdf5 source, add datalevel tests (#2001) @maxnoe
• Ignore unknown column metadata in read_table (#1992) @maxnoe
• Fix comparison of timestamps in calibration code (#1987) @maxnoe
• Fix missing start/stop for reading impact table, use correct type in simteleventsource (#1982) @maxnoe
• Hash rounded versions of floats for CameraGeometry (#1956) @maxnoe
• Fix impact being written for non-triggered telescopes, fixes #1961 (#1967) @maxnoe
• fixed bug that prevented printing of QualityQuery (#1958) @kosack
• Fix ArrayDisplay creating legend in wrong axes with multiple axes active (#1946) @maxnoe
• Store true_image_sum also when true image is not written (#1893) @maxnoe
• Better error message in case unknown telescopes are selected, fixes #1924 (#1925) @maxnoe
• Minor visual bugfix: better html repr for Tool and Component (#1905) @kosack

Refactoring and Optimization

• Use tel_id consistently (replacing telid -> tel_id), fixes #1457 (#2045) @maxnoe
• Remove cutflow (#1766) @maxnoe
• Add 'raises' to run_tool, which raises exceptions instead of logging them. (#2038) @nbiederbeck
• ExpressionEngine for QualityQuery and FeatureGenerator (#2014) @nbiederbeck
• Return tuples from SubArray properties, add suport for single value in get_tel_ids (#2027) @maxnoe
• Use pytest fixtures in unit tests instead of from_name methods (#1985) @maxnoe
• Ignore containers and maps by default in HDF5Writer (#1945) @maxnoe
• Hdf5EventSource performance improvements and skip empty events, fixes #1916 (#1917) @maxnoe
• Directly return peak position from neighbor average waveform (#1914) @maxnoe

Maintenance

• updated incorrect comments in base_config.yaml (#2040) @kosack
• Remove support for by_type split files (#2032) @maxnoe
• Fixed Typo in docstring (#2009) @aknierim
• added a better template for release-drafter (#1880) @kosack
• Add flake8 pre-commit hook. Change black hook to also format notebooks. (#1993) @nbiederbeck
• Avoid warning in case of reading DL1a only files with HDF5EventSource (#1994) @maxnoe
• Automatically include fields in docstrings for containers (#1990) @maxnoe
• Create a fresh ArrayEventContainer for each event in HDF5EventSource (#1970) @maxnoe
• Create a fresh ArrayEventContainer for each event in SimTelEventSource (#1952) @maxnoe
• Update CI actions versions (#1951) @maxnoe
• Simplify units code in subarray (#1950) @maxnoe
• Use a simple kwarg instead of a traitlet for TableLoader.h5file (#1937) @maxnoe
• Improve TableLoader docs, fixes #1930 (#1931) @maxnoe
• Print version info in pytest header (#1919) @maxnoe