Conference paper Open Access

Learning Workflow Scheduling on Multi-Resource Clusters

Hu, Yang; de Laat, Cees; Zhao, Zhiming

Workflow scheduling is one of the key issues in

the management of workflow execution. Typically, a workflow

application can be modeled as a Directed-Acyclic Graph (DAG).

In this paper, we present GoDAG, an approach that can learn

to well schedule workflows on multi-resource clusters. GoDAG

directly learns the scheduling policy from experience through

deep reinforcement learning. In order to adapt deep reinforcement

learning methods, we propose a novel state representation,

a practical action space and a corresponding reward definition

for workflow scheduling problem. We implement a GoDAG

prototype and a simulator to simulate task running on multiresource

clusters. In the evaluation, we compare the GoDAG with

three state-of-the-art heuristics. The results show that GoDAG

outperforms the baseline heuristics, leading to less average

makespan to different workflow structures.

Files (390.9 kB)
Name Size
390.9 kB Download
Views 549
Downloads 201
Data volume 78.6 MB
Unique views 534
Unique downloads 192


Cite as