Accelerating HEP Workloads on Kubernetes

Deploying a new open-source based service is an extensive and challenging process, from both technical and user-support perspective. The work includes understanding the service architecture and features, deploying the service on local servers, debugging errors and customizing the open source code to fit the specific requirements of the users' ecosystem.

Integration of the new machine learning service based on Kubeflow is an ongoing process to offer a better user experience for machine learning developers across CERN. Providing on-demand access to Python environments and hardware resources such as GPUs, the new service reduces users' need to set up local infrastructures and allows more time for scientific research. Additionally, the service offers features such as pipelines, automated hyperparameter search, inference services, that can be utilized for developing complex machine learning use cases which go beyond model training.