el Khaldi Ahanach, Elias
Koulouzis, Spiros
Zhao, Zhiming
2019-10-01
<p>When executing scientific workflows in a distributed</p>
<p>environment, anomalies of the workflow behavior are often</p>
<p>caused by a mixture of different issues, e.g., careless design</p>
<p>of the workflow logic, buggy workflow components, unexpected</p>
<p>performance bottlenecks or resource failure at the underlying</p>
<p>infrastructure. The provenance information only defines data</p>
<p>evolution at the workflow level, which does not have an explicit</p>
<p>connection with the system logs provided by the underlying</p>
<p>infrastructure. Analyzing provenance information and apposite</p>
<p>system metrics requires expertise and a considerable amount of</p>
<p>manual effort. Moreover, it is often time-consuming to aggregate</p>
<p>this information and correlate events occurring at different levels</p>
<p>in the infrastructure. In this paper, we propose an architecture</p>
<p>to automate the integration among the workflow provenance</p>
<p>information with the performance information collected from</p>
<p>infrastructure nodes running workflow tasks. Our architecture</p>
<p>enables workflow developers or domain scientists to effectively</p>
<p>browse workflow execution information together with the system</p>
<p>metrics, and analyze contextual information for possible anomalies.</p>
https://doi.org/10.5281/zenodo.3466766
oai:zenodo.org:3466766
Zenodo
https://zenodo.org/communities/eu
https://doi.org/10.5281/zenodo.3466765
info:eu-repo/semantics/openAccess
Creative Commons Attribution 4.0 International
https://creativecommons.org/licenses/by/4.0/legalcode
IWSG19, 11th International Workshop on Science Gateways, Ljubljana, Slovenia, 12-14, June 2019
Linking provenance with system logs: a context aware information integration and exploration framework for analyzing workflow execution
info:eu-repo/semantics/conferencePaper