Planned intervention: On Wednesday April 3rd 05:30 UTC Zenodo will be unavailable for up to 2-10 minutes to perform a storage cluster upgrade.
Published February 15, 2022 | Version v1
Report Open

Heterogeneous High Performance Computing

  • 1. BSC
  • 2. HPE
  • 3. Seagate Systems
  • 4. Forschungszentrum Jülich

Description

Modern HPC systems are becoming increasingly heterogeneous, affecting all components of HPC systems, from the processing units, through memory hierarchies and network components to storage systems. This trend is on the one hand due to the need to build larger, yet more energy efficient systems, and on the other hand it is caused by the need to optimise (parts of the) systems for certain workloads. In fact, it is not only the systems themselves that are becoming more heterogeneous, but also scientific and industrial applications are increasingly combining different technologies into complex workflows, including simulation, data analytics, visualisation, and artificial intelligence/machine learning. Different steps in these workflows call for different hardware and thus today’s HPC systems are often composed of different modules optimised to suit certain stages in these workflows.

While the trend towards heterogeneity is certainly helpful in many aspects, it makes the task of programming these systems and using them efficiently much more complicated. Often, a combination of different programming models is required and selecting suitable technologies for certain tasks or even parts of an algorithm is difficult. Novel methods might be needed for heterogeneous components or be only facilitated by them. And this trend is continuing, with new technologies around the corner that will further increase heterogeneity, e.g. neuromorphic or quantum accelerators, in-memory-computing, and other non-von-Neumann approaches.

In this paper, we present an overview of the different levels of heterogeneity we find in HPC technologies and provide recommendations for research directions to help deal with the challenges they pose. We also point out opportunities that particularly applications can profit from by exploiting these technologies. Research efforts will be needed over the full spectrum, from system architecture, compilers and programming models/languages, to runtime systems, algorithms and novel mathematical approaches.

Files

ETP4HPC_WP_Heterogeneous-HPC_20220216.pdf

Files (3.2 MB)

Name Size Download all
md5:b2f9858906fe0c9711db4603206de8b5
3.2 MB Preview Download

Additional details

Funding

MAESTRO – Middleware for memory and data-awareness in workflows 801101
European Commission
DEEP-SEA – DEEP – SOFTWARE FOR EXASCALE ARCHITECTURES 955606
European Commission
EuroEXA – Co-designed Innovation and System for Resilient Exascale Computing in Europe: From Applications to Silicon 754337
European Commission
FocusCoE – Concerted action for the European HPC CoEs 823964
European Commission

References