Conference paper Open Access

Learning Workflow Scheduling on Multi-Resource Clusters

Hu, Yang; de Laat, Cees; Zhao, Zhiming


JSON-LD (schema.org) Export

{
  "description": "<p>Workflow scheduling is one of the key issues in</p>\n\n<p>the management of workflow execution. Typically, a workflow</p>\n\n<p>application can be modeled as a Directed-Acyclic Graph (DAG).</p>\n\n<p>In this paper, we present GoDAG, an approach that can learn</p>\n\n<p>to well schedule workflows on multi-resource clusters. GoDAG</p>\n\n<p>directly learns the scheduling policy from experience through</p>\n\n<p>deep reinforcement learning. In order to adapt deep reinforcement</p>\n\n<p>learning methods, we propose a novel state representation,</p>\n\n<p>a practical action space and a corresponding reward definition</p>\n\n<p>for workflow scheduling problem. We implement a GoDAG</p>\n\n<p>prototype and a simulator to simulate task running on multiresource</p>\n\n<p>clusters. In the evaluation, we compare the GoDAG with</p>\n\n<p>three state-of-the-art heuristics. The results show that GoDAG</p>\n\n<p>outperforms the baseline heuristics, leading to less average</p>\n\n<p>makespan to different workflow structures.</p>", 
  "license": "https://creativecommons.org/licenses/by/4.0/legalcode", 
  "creator": [
    {
      "affiliation": "University of Amsterdam", 
      "@type": "Person", 
      "name": "Hu, Yang"
    }, 
    {
      "affiliation": "University of Amsterdam", 
      "@type": "Person", 
      "name": "de Laat, Cees"
    }, 
    {
      "affiliation": "University of Amsterdam", 
      "@id": "https://orcid.org/0000-0002-6717-9418", 
      "@type": "Person", 
      "name": "Zhao, Zhiming"
    }
  ], 
  "headline": "Learning Workflow Scheduling on Multi-Resource Clusters", 
  "image": "https://zenodo.org/static/img/logos/zenodo-gradient-round.svg", 
  "datePublished": "2019-08-16", 
  "url": "https://zenodo.org/record/3466676", 
  "version": "Camera ready", 
  "keywords": [
    "Workflow Scheduling", 
    "Multi-resource Clusters", 
    "Directed-Acyclic Graph (DAG)", 
    "Deep Reinforcement Learning"
  ], 
  "@context": "https://schema.org/", 
  "identifier": "https://doi.org/10.1109/NAS.2019.8834720", 
  "@id": "https://doi.org/10.1109/NAS.2019.8834720", 
  "@type": "ScholarlyArticle", 
  "name": "Learning Workflow Scheduling on Multi-Resource Clusters"
}
47
43
views
downloads
Views 47
Downloads 43
Data volume 16.8 MB
Unique views 44
Unique downloads 40

Share

Cite as