Published April 10, 2019
| Version 1.0
Software
Open
Software Artifacts for Performance Feedback Autoscaling Experiments with Workloads of Workflows in Apache Airflow
Creators
- 1. Delft University of Technology
- 2. University of Würzburg
- 3. Mälardalen University
- 4. University of Southern California
- 5. Vrije Universiteit Amsterdam, Delft University of Technology
Description
These software artifacts are related to the computational artifacts DOI:10.5281/zenodo.2635573
The software artifacts are packed in the sa.tar.gz archive with the following directories:
- airflow contains the source code of the Airflow system v1.9.0 with the added autoscaling support, including the source code of all the considered autoscaling policies: PLF, SCF, and PFA. The code of the autoscalers is located in `airflow/airflow/autoscalers`.
- gurobi contains our implementation of the MIP model used in the paper for the Gurobi solver v8.0.1.
- tools contains the scripts we used to generate workloads, analyze the results, plot data, and deploy and run the Airflow experimental setup. For the experimental purposes the Airflow code can be just run directly without installation (e.g., without using the `airflow/INSTALL`, `airflow/setup.py` files). And that is how we were running it. The airflow code from the `airflow` directory should be correctly positioned, so that the deployments scripts can reach it. Check the paths inside the scripts within `tools/deployment`, and correct them, if necessary.