Log Analysis and Classification of CERN Control Systems

Project Specification

CERN is running several large WinCC OA distributed sys- tems, each of them counting ~100-150 computers and several thousands of processes. The log files currently produced by these processes are only looked at very seldom to debug prob- lems. The project will consist of introducing a modern log analysis tool to analyze these log files in real time and inform the operators/experts of possible problems. The tools will also be used to extract statistics and produce reports about the control systems. The selected tool will be Elastic Search with Kibana.

Abstract

This project has consisted in examining different alternatives for monitoring and ana- lyzing the log messages produced by the control system software using at CERN. Its main accomplishments have been a performance measurement of the Logstash tool un- der both realistic and extreme load scenarios, the development of a flexible and powerful performance testing tool, and the proposal of a system that automatically determines the importance that a given log message might have for an operator based on a Bayesian filter.