Published April 15, 2024 | Version v2.6.0
Software Open

System for automated Quality Control - SaQC

  • 1. Helmholtz Centre for Environmental Research
  • 2. Helmholtz Centre for Environmental Research - UFZ
  • 1. Helmholtz Centre for Environmental Research

Description

Added

  • reindex: base reindexer function
  • flagGeneric, processGeneric: target broadcasting and numpy array support
  • SaQC: automatic translation of incoming flags
  • Option to change the flagging scheme after initialization
  • flagByClick: manually assign flags using a graphical user interface
  • SaQC: support for selection, slicing and setting of items by subscription on SaQC objects
  • transferFlags is a multivariate function
  • plot: added yscope keyword
  • setFlags: function to replace flagManual
  • flagUniLOF: added parameter slope_correct to correct for overflagging at relatively steep data value slopes
  • History: added option to change aggregation behavior
  • Translation scheme AnnotatedFloatScheme

Changed

  • SaQC.flags always returns a DictOfSeries

Removed

  • SaQC methods deprecated in version 2.4: interpolate, interpolateIndex, interpolateInvalid, roll, linear,shift, flagCrossStatistics
  • Method Flags.toDios deprecated in version 2.4
  • Method DictOfSeries.index_of method deprecated in version 2.4
  • Option "complete" for parameter history of method plot
  • Option "cycleskip" for parameter ax_kwargs of method plot
  • Parameter phaseplot from method plot

Fixed

  • flagConstants: fixed flagging of rolling ramps
  • Flags: add meta entry to imported flags
  • group operations were overwriting existing flags
  • SaQC._construct : was not working for inherited classes
  • processgeneric: improved numpy function compatability

Deprecated

  • flagManual in favor of setFlags
  • inverse_** options for concatFlags parameter method in favor of invert=True
  • flagRaise with delegation to better replacements flagZScore, flagUniLOF, flagJumps or flagOffset
  • flagByGrubbs with delegation to better replacements flagZScore, flagUniLOFs
  • flagMVScore with delegation to manual application of the steps

 

Notes

Cite as: Lennart Schmidt, David Schäfer, Juliane Geller, Peter Lünenschloss, Bert Palm, Karsten Rinke, Corinna Rebmann, Michael Rode, Jan Bumberger, System for automated Quality Control (SaQC) to enable traceable and reproducible data streams in environmental science, Environmental Modelling & Software, 2023, 105809, ISSN 1364-8152, https://doi.org/10.1016/j.envsoft.2023.105809.

Other

Acknowledgements

We thank the Helmholtz Association and the Federal Ministry of Education and Research (BMBF) for supporting the DataHub Initiative of the Research Field Earth and Environment. The DataHub enables an overarching and comprehensive research data management, following FAIR principles, for all Topics in the Program Changing Earth – Sustaining our Future.

Files

saqc-v2.6.0.zip

Files (6.0 MB)

Name Size Download all
md5:4aa25be130fde68b3fd049a4c1d3f684
6.0 MB Preview Download

Additional details

Related works

Funding

Federal Ministry of Education and Research
Helmholtz Association of German Research Centres

Software

Repository URL
https://git.ufz.de/rdm-software/saqc/
Development Status
Active