Conference paper Open Access

Learning Workflow Scheduling on Multi-Resource Clusters

Hu, Yang; de Laat, Cees; Zhao, Zhiming


Dublin Core Export

<?xml version='1.0' encoding='utf-8'?>
<oai_dc:dc xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:oai_dc="http://www.openarchives.org/OAI/2.0/oai_dc/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.openarchives.org/OAI/2.0/oai_dc/ http://www.openarchives.org/OAI/2.0/oai_dc.xsd">
  <dc:creator>Hu, Yang</dc:creator>
  <dc:creator>de Laat, Cees</dc:creator>
  <dc:creator>Zhao, Zhiming</dc:creator>
  <dc:date>2019-08-16</dc:date>
  <dc:description>Workflow scheduling is one of the key issues in

the management of workflow execution. Typically, a workflow

application can be modeled as a Directed-Acyclic Graph (DAG).

In this paper, we present GoDAG, an approach that can learn

to well schedule workflows on multi-resource clusters. GoDAG

directly learns the scheduling policy from experience through

deep reinforcement learning. In order to adapt deep reinforcement

learning methods, we propose a novel state representation,

a practical action space and a corresponding reward definition

for workflow scheduling problem. We implement a GoDAG

prototype and a simulator to simulate task running on multiresource

clusters. In the evaluation, we compare the GoDAG with

three state-of-the-art heuristics. The results show that GoDAG

outperforms the baseline heuristics, leading to less average

makespan to different workflow structures.</dc:description>
  <dc:identifier>https://zenodo.org/record/3466676</dc:identifier>
  <dc:identifier>10.1109/NAS.2019.8834720</dc:identifier>
  <dc:identifier>oai:zenodo.org:3466676</dc:identifier>
  <dc:relation>info:eu-repo/grantAgreement/EC/H2020/676247/</dc:relation>
  <dc:relation>info:eu-repo/grantAgreement/EC/H2020/654182/</dc:relation>
  <dc:relation>info:eu-repo/grantAgreement/EC/H2020/643963/</dc:relation>
  <dc:relation>info:eu-repo/grantAgreement/EC/H2020/825134/</dc:relation>
  <dc:rights>info:eu-repo/semantics/openAccess</dc:rights>
  <dc:rights>https://creativecommons.org/licenses/by/4.0/legalcode</dc:rights>
  <dc:subject>Workflow Scheduling</dc:subject>
  <dc:subject>Multi-resource Clusters</dc:subject>
  <dc:subject>Directed-Acyclic Graph (DAG)</dc:subject>
  <dc:subject>Deep Reinforcement Learning</dc:subject>
  <dc:title>Learning Workflow Scheduling on Multi-Resource Clusters</dc:title>
  <dc:type>info:eu-repo/semantics/conferencePaper</dc:type>
  <dc:type>publication-conferencepaper</dc:type>
</oai_dc:dc>
46
43
views
downloads
Views 46
Downloads 43
Data volume 16.8 MB
Unique views 43
Unique downloads 40

Share

Cite as