Seminar Paper Artifacts for: Collecting and Exploiting Performance Metrics of Kafka Streams Applications

Paper Artifacts for: Collecting and Exploiting Performance Metrics of Kafka Streams Applications

A detailed description can be found in the README.md

Abstract Stream processing engines are increasingly used in huge and distributed systems, especially when a continuous flow of data is to be processed. Benchmarking can be used to analyze and evaluate the performance of a system and to make it comparable with other systems by measuring various indicators. To execute a benchmark, it is necessary to use performance metrics that give information about the analyzed system characteristics. In this paper, we present a monitoring approach that allows acquiring extensive metric data which can be used for executing benchmarks. Visualization can be helpful for controlling and monitoring systems and it is particularly useful for showing benchmark results. For that reason, our approach provides of a dashbord solution that allows to visualize the monitored data. Based on this background, we focus on the collection of metric data, which are particularly relevant for benchmarking and monitoring Kafka Streams. In an experimental overhead evaluation we analyze if the usage of the monitoring solution causes a relevant perfomance overhead.The evaluation showes that exposing debug-level metrics in Kafka Streams causes a noticeable overhead of 30 - 50% in comparision to exposing info-level metrics.