Conference paper Open Access

Linking provenance with system logs: a context aware information integration and exploration framework for analyzing workflow execution

el Khaldi Ahanach, Elias; Koulouzis, Spiros; Zhao, Zhiming

When executing scientific workflows in a distributed

environment, anomalies of the workflow behavior are often

caused by a mixture of different issues, e.g., careless design

of the workflow logic, buggy workflow components, unexpected

performance bottlenecks or resource failure at the underlying

infrastructure. The provenance information only defines data

evolution at the workflow level, which does not have an explicit

connection with the system logs provided by the underlying

infrastructure. Analyzing provenance information and apposite

system metrics requires expertise and a considerable amount of

manual effort. Moreover, it is often time-consuming to aggregate

this information and correlate events occurring at different levels

in the infrastructure. In this paper, we propose an architecture

to automate the integration among the workflow provenance

information with the performance information collected from

infrastructure nodes running workflow tasks. Our architecture

enables workflow developers or domain scientists to effectively

browse workflow execution information together with the system

metrics, and analyze contextual information for possible anomalies.

Files (633.3 kB)
Name Size
633.3 kB Download
All versions This version
Views 249249
Downloads 4141
Data volume 26.0 MB26.0 MB
Unique views 241241
Unique downloads 4141


Cite as