Published May 26, 2020
| Version v1.0.0
Software
Open
RADAR-base/Restructure-HDFS-topic: radar-output-restructure version 1.0.0
Authors/Creators
- 1. @thehyve
- 2. @RADAR-base @KHP-Informatics
Description
Changes since radar-hdfs-restructure version 0.6.0:
- Added storage drivers for S3 API input and output
- Storage is more consistent to configure
- Changed synchronisation and accounting to be handled with Redis
- With more consistent locking
- Faster write times
- Redis is required.
- Added integration tests
- Configurable automatic deletion after a threshold amount of time, to avoid storage filling up.
- Reprocesses files that were modified after the last processed modification time.
- Customizable time bin format
Files
RADAR-base/Restructure-HDFS-topic-v1.0.0.zip
Additional details
Related works
- Is supplement to
- https://github.com/RADAR-base/Restructure-HDFS-topic/tree/v1.0.0 (URL)