Optimization Studies of Millepede - a Detector Alignment Application

Project Specification

Millepede is a multi-threaded detector alignment and calibration software based on the Linear Least Squares Fit algorithm for fitting a large number of parameters (5M+). Developed at DESY, the application is used in many HEP sites, including CERN. Due to computational and memory complexity, the application uses an in-house principle of submatrix partitioning. While this is the way to make the computation feasible, it is not possible to deliver uncertainty for every estimated parameter. The project concerns characterizing the application in terms of parallelization efficiency, including the multi-threading and vectorization aspects.  The project was also ported to Intel Xeon Phi, which allowed conducting a performance comparison between Xeon and Xeon Phi.

Abstract

Optimization of a scientific application is not an easy task. It requires a broad knowledge and following a multiple-step methodology. In this work we have characterized and optimized a detector alignment software called Millepede II. We have focused on different performance levels: vectorization, parallelism, memory distribution. Also, the software was ported to Xeon Phi. For the characterization of the application we have used several tools including Intel Advisor and Intel VTune Amplifier.

In this paper we show that there is some room for optimization without introducing massive modifications to the code. We obtain a speed-up of 15% by introducing changes in the code to ease the vectorization and by choosing the right scheduling for each parallel section of the code.

Finally, we run the software on Xeon Phi, obtaining not satisfactory results due to the lack of adaptation of the software to the card’s hardware.