There is a newer version of the record available.

Published May 4, 2023 | Version v1

SLASH - Serverless Apache Spark Hub

Authors/Creators

  • 1. Zurich University of Applied Sciences

Description

This folder contains SLASH, the research prototype implementation of
Serverless Apache Spark Hub.

To make it run, install the dependencies for OpenStack access, e.g.:
 % sudo apt-get install python3-novaclient

SLASH consists of three components: The Python module 'slash' which
augments 'pyspark' magically with reactive autoscaling capabilities, the
'slashhub' which collects on-demand scaling actions to calculate
reactive autoscaling in conjunction with predictive ones, and the
'slashjobserver' which executes the calendar scheduling and forecasting.

A calendar file is expected as 'calendar' with dates or date ranges per
line. Furthermore, the functionality is determined by environment
variables: OS_* (see openstackclient.py for a list) and SLASH pointing
'slash' to the node running 'slashhub' on port 11111.

Files

Files (4.6 kB)

Name Size Download all
md5:1f52898ab89187a4f7eb5fcb82da7520
4.6 kB Download