UPDATE: Zenodo migration postponed to Oct 13 from 06:00-08:00 UTC. Read the announcement.

Conference paper Open Access

Linking provenance with system logs: a context aware information integration and exploration framework for analyzing workflow execution

el Khaldi Ahanach, Elias; Koulouzis, Spiros; Zhao, Zhiming

When executing scientific workflows in a distributed

environment, anomalies of the workflow behavior are often

caused by a mixture of different issues, e.g., careless design

of the workflow logic, buggy workflow components, unexpected

performance bottlenecks or resource failure at the underlying

infrastructure. The provenance information only defines data

evolution at the workflow level, which does not have an explicit

connection with the system logs provided by the underlying

infrastructure. Analyzing provenance information and apposite

system metrics requires expertise and a considerable amount of

manual effort. Moreover, it is often time-consuming to aggregate

this information and correlate events occurring at different levels

in the infrastructure. In this paper, we propose an architecture

to automate the integration among the workflow provenance

information with the performance information collected from

infrastructure nodes running workflow tasks. Our architecture

enables workflow developers or domain scientists to effectively

browse workflow execution information together with the system

metrics, and analyze contextual information for possible anomalies.

Files (633.3 kB)
Name Size
633.3 kB Download
All versions This version
Views 548548
Downloads 6565
Data volume 41.2 MB41.2 MB
Unique views 531531
Unique downloads 6363


Cite as