Published August 3, 2020
| Version 0.6.1
Software
Open
kiwiPy: Robust, high-volume, messaging for big-data and computational science workflows
Creators
- 1. Department of Energy Conversion and Storage, Technical University of Denmark, 2800 Kgs. Lyngby, Denmark
- 2. National Centre for Computational Design and Discovery of Novel Materials (MARVEL), École Polytechnique Fédérale de Lausanne, CH-1015 Lausanne, Switzerland and Theory and Simulation of Materials (THEOS), Faculté des Sciences et Techniques de l'Ingénieur, École Polytechnique Fédérale de Lausanne, CH-1015 Lausanne, Switzerland
Contributors
Other:
- 1. Department of Physics, South China University of Technology, Guangzhou 510640, China
Description
In this work we present kiwiPy, a Python library designed to support robust message based communication for high-throughput, big-data, applications while being general enough to be useful wherever high-volumes of messages need to be communicated in a predictable manner. KiwiPy relies on the RabbitMQ protocol, an industry standard message broker, while providing a simple and intuitive interface that can be used in both multithreaded and coroutine based applications. To demonstrate some of kiwiPy's functionality we give examples from AiiDA, a high-throughput simulation platform, where kiwiPy is used as a key component of the workflow engine.
Notes
This is the version referred to by the kiwiPy publication in JOSS.
Files
Files
(44.0 kB)
| Name | Size | Download all |
|---|---|---|
|
md5:7e523733729d9200961e2fe9ebbc276d
|
44.0 kB | Download |
Additional details
Related works
- Is referenced by
- Journal article: arXiv:2005.07475 (arXiv)