Conference paper Open Access

Linking provenance with system logs: a context aware information integration and exploration framework for analyzing workflow execution

el Khaldi Ahanach, Elias; Koulouzis, Spiros; Zhao, Zhiming

When executing scientific workflows in a distributed

environment, anomalies of the workflow behavior are often

caused by a mixture of different issues, e.g., careless design

of the workflow logic, buggy workflow components, unexpected

performance bottlenecks or resource failure at the underlying

infrastructure. The provenance information only defines data

evolution at the workflow level, which does not have an explicit

connection with the system logs provided by the underlying

infrastructure. Analyzing provenance information and apposite

system metrics requires expertise and a considerable amount of

manual effort. Moreover, it is often time-consuming to aggregate

this information and correlate events occurring at different levels

in the infrastructure. In this paper, we propose an architecture

to automate the integration among the workflow provenance

information with the performance information collected from

infrastructure nodes running workflow tasks. Our architecture

enables workflow developers or domain scientists to effectively

browse workflow execution information together with the system

metrics, and analyze contextual information for possible anomalies.

Files (633.3 kB)
Name Size
2019.conference.workshop.iwsg-cam.pdf
md5:20ab8e0bfc5f7dd1bd094fe5fa1b4991
633.3 kB Download
3
3
views
downloads
All versions This version
Views 33
Downloads 33
Data volume 1.9 MB1.9 MB
Unique views 33
Unique downloads 33

Share

Cite as