The Road to Devops HPC Cluster Management

EMSL has been using xCAT for cluster administration since 2011. As supercomputing strategies have changed, we've developed a configuration approach which addresses many of xCAT’s shortcomings and enables us to manage heterogeneous cluster configurations in a manner concordant with software engineering best practices. Some changes have been submitted upstream to the xCAT project, and others exist as a suite of automation tools and architectural practices. This presentation is an overview of some of the challenges we've overcome and best practices derived from 8 years of real-world xCAT deployment across multiple clusters.