Journal article Open Access

Design and implementation of GXP make — A workflow system based on make

Taura, Kenjiro; Matsuzaki, Takuya; Miwa, Makoto; Kamoshida, Yoshikazu; Yokoyama, Daisaku; Dun, Nan; Shibata, Takeshi; Jun, Choi Sung; Tsujii, Jun'ichi

MARC21 XML Export

<?xml version='1.0' encoding='UTF-8'?>
<record xmlns="">
  <datafield tag="540" ind1=" " ind2=" ">
    <subfield code="u"></subfield>
    <subfield code="a">Creative Commons Attribution Share Alike 4.0 International</subfield>
  <datafield tag="260" ind1=" " ind2=" ">
    <subfield code="c">2013-02-01</subfield>
  <controlfield tag="005">20200120172324.0</controlfield>
  <controlfield tag="001">890130</controlfield>
  <datafield tag="909" ind1="C" ind2="O">
    <subfield code="p">openaire</subfield>
    <subfield code="o"></subfield>
  <datafield tag="520" ind1=" " ind2=" ">
    <subfield code="a">This paper describes the rational behind designing
workflow systems based on the Unix make by showing a number
of idioms useful for workflows comprising many tasks. It also
demonstrates a specific design and implementation of such a
workflow system called GXP make. GXP make supports all
the features of GNU make and extends its platforms from
single node systems to clusters, clouds, supercomputers, and
distributed systems. Interestingly, it is achieved by a very small
code base that does not modify GNU make implementation at
all. While being not ideal for performance, it achieved a useful
performance and scalability of dispatching one million tasks in
approximately 16,000 seconds (60 tasks per second, including
dependence analysis) on an 8 core Intel Nehalem node. For
real applications, recognition and classification of protein-protein
interactions from biomedical texts on a supercomputer with more
than 8,000 cores are described.</subfield>
  <datafield tag="700" ind1=" " ind2=" ">
    <subfield code="a">Matsuzaki, Takuya</subfield>
  <datafield tag="700" ind1=" " ind2=" ">
    <subfield code="a">Miwa, Makoto</subfield>
  <datafield tag="700" ind1=" " ind2=" ">
    <subfield code="a">Kamoshida, Yoshikazu</subfield>
  <datafield tag="700" ind1=" " ind2=" ">
    <subfield code="a">Yokoyama, Daisaku</subfield>
  <datafield tag="700" ind1=" " ind2=" ">
    <subfield code="a">Dun, Nan</subfield>
  <datafield tag="700" ind1=" " ind2=" ">
    <subfield code="a">Shibata, Takeshi</subfield>
  <datafield tag="700" ind1=" " ind2=" ">
    <subfield code="a">Jun, Choi Sung</subfield>
  <datafield tag="700" ind1=" " ind2=" ">
    <subfield code="a">Tsujii, Jun'ichi</subfield>
  <datafield tag="856" ind1="4" ind2=" ">
    <subfield code="s">843588</subfield>
    <subfield code="z">md5:3259da71febed6c900500db90a5a59a7</subfield>
    <subfield code="u"></subfield>
  <datafield tag="542" ind1=" " ind2=" ">
    <subfield code="l">open</subfield>
  <datafield tag="980" ind1=" " ind2=" ">
    <subfield code="a">publication</subfield>
    <subfield code="b">article</subfield>
  <datafield tag="100" ind1=" " ind2=" ">
    <subfield code="a">Taura, Kenjiro</subfield>
  <datafield tag="024" ind1=" " ind2=" ">
    <subfield code="a">10.1016/j.future.2011.05.026</subfield>
    <subfield code="2">doi</subfield>
  <datafield tag="245" ind1=" " ind2=" ">
    <subfield code="a">Design and implementation of GXP make — A workflow system based on make</subfield>
  <datafield tag="650" ind1="1" ind2="7">
    <subfield code="a">cc-by</subfield>
    <subfield code="2"></subfield>
Views 86
Downloads 48
Data volume 40.5 MB
Unique views 86
Unique downloads 47


Cite as