Modernise log collection of OKD deployments
Authors/Creators
Description
To address future processing demands, it's essential to enhance logging capabilities by implementing a more robust, efficient, and maintainable system, in replacement of our current Fluentd system. In addition, OKD maintainer RedHat already deprecated Fluentd log collector Operator and plans to phase out and remove it in a future release, providing only bug fixes and support in the meantime. Their official alternative is Vector for the internal log collector Operator, which is our chosen option for researching and proof of concept. Fluentd is an open source data collector and processor used as a general purpose logging tool written in Ruby, under the CNCF. Fluentbit serves the same purpose, also under the CNCF, but is written in C, which makes it faster and more memory efficient. Vector is also a logging collector and processor that serves the same purpose, but is written in Rust and is developed by DataDog. This report details the research and modifications to the internal OKD logging pipeline out of Fluentd towards FluentBit and Vector in order to match the current system requirements, make it more robust and efficient, and also be compatible with future releases of OKD. Also, we will explore ways to simplify the logging pipeline itself, merging different components into one aggregator. Our OKD telemetry pipeline collects logs from containers and Kubernetes itself, parses them, and sends them to OpenSearch, through the Central Monitoring system. Our Kubernetes Magnum clusters implement almost the same pipeline, in addition to enriching the data with metrics about the logs themselves, in order to send them to Prometheus. Therefore, we will try to make our solution cross-compatible for OKD and Kubernetes Magnum.