Conference paper Open Access
Linking provenance with system logs: a context aware information integration and exploration framework for analyzing workflow execution
el Khaldi Ahanach, Elias;
Koulouzis, Spiros;
Zhao, Zhiming
When executing scientific workflows in a distributed
environment, anomalies of the workflow behavior are often
caused by a mixture of different issues, e.g., careless design
of the workflow logic, buggy workflow components, unexpected
performance bottlenecks or resource failure at the underlying
infrastructure. The provenance information only defines data
evolution at the workflow level, which does not have an explicit
connection with the system logs provided by the underlying
infrastructure. Analyzing provenance information and apposite
system metrics requires expertise and a considerable amount of
manual effort. Moreover, it is often time-consuming to aggregate
this information and correlate events occurring at different levels
in the infrastructure. In this paper, we propose an architecture
to automate the integration among the workflow provenance
information with the performance information collected from
infrastructure nodes running workflow tasks. Our architecture
enables workflow developers or domain scientists to effectively
browse workflow execution information together with the system
metrics, and analyze contextual information for possible anomalies.
| Name | Size | |
|---|---|---|
|
2019.conference.workshop.iwsg-cam.pdf
md5:20ab8e0bfc5f7dd1bd094fe5fa1b4991 |
633.3 kB | Download |
| All versions | This version | |
|---|---|---|
| Views | 548 | 548 |
| Downloads | 65 | 65 |
| Data volume | 41.2 MB | 41.2 MB |
| Unique views | 531 | 531 |
| Unique downloads | 63 | 63 |
- Publication date:
- October 1, 2019
- DOI:
-
- Grants:
-
European Commission:
-
VRE4EIC - A Europe-wide Interoperable Virtual Research Environment to Empower Multidisciplinary Research Communities and Accelerate Innovation and Collaboration (676247)
-
ENVRI PLUS - Environmental Research Infrastructures Providing Shared Solutions for Science and Society (654182)
-
SWITCH - Software Workbench for Interactive, Time Critical and Highly self-adaptive cloud applications (643963)
-
ARTICONF - smART socIal media eCOsytstem in a blockchaiN Federated environment (825134)
- ENVRI-FAIR - ENVironmental Research Infrastructures building Fair services Accessible for society, Innovation and Research (824068)
-
VRE4EIC - A Europe-wide Interoperable Virtual Research Environment to Empower Multidisciplinary Research Communities and Accelerate Innovation and Collaboration (676247)
- Meeting:
- 11th International Workshop on Science Gateways (IWSG19), Ljubljana, Slovenia, 12-14, June 2019
- License (for files):
- Creative Commons Attribution 4.0 International