D7.5: Evaluation of Accelerated and Non-accelerated Benchmarks

The Unified European Application Benchmark Suite (UEABS) provides a publicly available benchmark suite. One of the key results of this activity is the re-unification of the UEABS and the accelerator benchmark suite so that the UEABS lives up to its Unified name again. This new release is migrated to the PRACE GitLab server (next to the CodeVault repository). We present benchmark results and performance analyses on PRACE Tier-0 systems, on two PRACE PCP prototypes, on a DEEP-ER prototype, and on a Mont-Blanc 3 prototype. (If you want to select the optimal system/architecture for a given UEABS application, please have a look at these results.) Furthermore, we compare the energy efficiency from an application point of view of systems where energy measurements at job level are possible. Finally, we conclude with a high-level comparison of the benchmark systems: starting with the ubiquitous LINPACK performance; followed by both application performance (time to solution, or speed) as well as energy efficiency (energy to solution). For this we combine all benchmark results and derive a comparison of the overall performance of the systems, and a comparison of the energy efficiency for the systems where we obtained energy measurements.
The energy efficiency of the two benchmarked PCP prototypes strongly depends on the application benchmark / data set / problem size / node count. Overall, the GPU based system (DAVIDE) is somewhat more energy efficient than the KNL system (Frioul). If we add the GPU based Piz Daint system to the comparison, then Piz Daint clearly is the most energy efficient system.
As expected, the optimal system/architecture also strongly depends on the application benchmark / data set / problem size / node count. Overall the most recent Intel Skylake systems are the most performant, JUWELS being the fastest. For applications that can exploit GPUs, Piz Daint is most performant. On the other end of the spectrum, the systems based on the discontinued Knights Landing in general are least performant. The conclusion might be that LINPACK performance still is a reasonable indicator for application performance, but most people – including the LINPACK originators themselves – will disagree.