Conference paper Open Access

Learning Workflow Scheduling on Multi-Resource Clusters

Hu, Yang; de Laat, Cees; Zhao, Zhiming

Citation Style Language JSON Export

  "DOI": "10.1109/NAS.2019.8834720", 
  "title": "Learning Workflow Scheduling on Multi-Resource Clusters", 
  "issued": {
    "date-parts": [
  "abstract": "<p>Workflow scheduling is one of the key issues in</p>\n\n<p>the management of workflow execution. Typically, a workflow</p>\n\n<p>application can be modeled as a Directed-Acyclic Graph (DAG).</p>\n\n<p>In this paper, we present GoDAG, an approach that can learn</p>\n\n<p>to well schedule workflows on multi-resource clusters. GoDAG</p>\n\n<p>directly learns the scheduling policy from experience through</p>\n\n<p>deep reinforcement learning. In order to adapt deep reinforcement</p>\n\n<p>learning methods, we propose a novel state representation,</p>\n\n<p>a practical action space and a corresponding reward definition</p>\n\n<p>for workflow scheduling problem. We implement a GoDAG</p>\n\n<p>prototype and a simulator to simulate task running on multiresource</p>\n\n<p>clusters. In the evaluation, we compare the GoDAG with</p>\n\n<p>three state-of-the-art heuristics. The results show that GoDAG</p>\n\n<p>outperforms the baseline heuristics, leading to less average</p>\n\n<p>makespan to different workflow structures.</p>", 
  "author": [
      "family": "Hu, Yang"
      "family": "de Laat, Cees"
      "family": "Zhao, Zhiming"
  "version": "Camera ready", 
  "type": "paper-conference", 
  "id": "3466676"
Views 49
Downloads 45
Data volume 17.6 MB
Unique views 45
Unique downloads 42


Cite as