ETP4HPC
Published May 11, 2022 | Version v1
Report Open

Modular Supercomputing Architecture

Authors/Creators

  • 1. Jülich Supercomputing Centre
  • 2. ParTec AG
  • 3. LuxProvide
  • 4. JSC/ RWTH Aachen University

Description

The European Community and its member states regularly invest large volumes of funding and effort in the development of HPC technologies in Europe. However, some observers express the criticism that these investments are either unfocused, lack long-term perspectives, or that their results are not mature enough to be adopted by the mainstream developments, which limits their benefit for the European HPC community, industry and society. This paper is intended as a counterexample to this pessimistic view. It describes the success story of Modular Supercomputing Architecture, which started in 2011 with the EU-funded R&D project “DEEP”, and is now being adopted by large-scale supercomputing centres across the old continent and worldwide. Main hardware and software characteristics of the architecture and some of the systems using it are described, complemented by a historical view of its development, the lessons learned in the process and future prospects.

Files

ETP4HPC_WP_MSA_final.pdf

Files (3.4 MB)

Name Size Download all
md5:7a76d295062c90682e5906061d63d827
3.4 MB Preview Download

Additional details

Funding

European Commission
DEEP - Dynamical Exascale Entry Platform 287530
European Commission
HPCQS - High Performance Computer and Quantum Simulator hybrid 101018180
European Commission
DEEP-EST - DEEP - Extreme Scale Technologies 754304
European Commission
DEEP-SEA - DEEP – SOFTWARE FOR EXASCALE ARCHITECTURES 955606
European Commission
DEEP-ER - DEEP Extended Reach 610476

References

  • [1] E. Suarez, N. Eicker and T. Lippert, Modular Supercomputing Architecture: from idea to production, Chapter 9 in Contemporary High Performance Computing: from Petascale toward Exascale, vol. 3, J. S. Vetter, Ed., CRC Press, 2019, pp. 223-251.
  • [2] E. Suarez, N. Eicker and T. Lippert, "Supercomputer Evolution at JSC," in Proceedings of the 2018 NIC Symposium, 2018.
  • [3] N. Eicker, T. Lippert, T. Moschny and E. Suarez, "The DEEP Project - An alternative approach to heterogeneous cluster-computing in the many-core era," Concurrency and computation: Practice and Experience, vol. 28, pp. 2394–-2411, 2016.
  • [4] A. Kreuzer, J. Amaya, N. Eicker, R. Léger and E. Suarez, "The DEEP-ER project: I/O and resiliency extensions for the Cluster-Booster architecture," in Proceedings of 2018 IEEE 20th International Conference on High Performance Computing and Communications (HPCC), Exeter, United Kingdom, 2018.
  • [5] E. Suarez, A. Kreuzer, N. Eicker and T. Lippert, The DEEP-EST project, Chapter 1 in Porting applications to a Modular Supercomputer - Experiences from the DEEP-EST project, Forschungszentrum Jülich GmbH Zentralbibliothek, Verlag, Schriften des Forschungszentrums Jülich IAS Series 48, 2021, pp. 9-25.
  • [6] N. Eicker, A. Galonska and M. N. J. Hauke, Bridging the DEEP Gap – Implementation of an Efficient Forwarding Protocol. In: Intel European Exascale Labs - Report 2013, 2014, p. 34–41.
  • [7] E. Suarez, N. Eicker, T. Moschny and T. Lippert, Critical Analysis of the Modular Supercomputing Architecture. Chapter 9 in Porting applications to a Modular Supercomputer - Experiences from the DEEP-EST project, Forschungszentrum Jülich GmbH Zentralbibliothek, Verlag Jülich, IAS Series 48, 2021, pp. 233-245.
  • [8] W. Gropp, "MPICH2: a new start for MPI implementations," Recent Advances in Parallel Virtual Machine and Message Passing Interface, vol. 2474, p. 7, 2002.
  • [9] S. Pickartz, C. Clauss, S. Lankes, S. Krempel, T. Moschny and A. Monti, "Non- Intrusive Migration of MPI Processes in OS-bypass Networks," in IEEE International Parallel and Distributed Processing Symposium Workshops (IPDPSW), 2016.
  • [10] "InfiniBand (IB) architecture specification. Tech. rep., ITA," 2016. [Online]. Available: https://www.infinibandta.org/ibta-specification/.
  • [11] M. S. Birrittella, M. Debbage, R. Huggahalli, J. Kunz, T. Lovett, T. Rimmer, K. D. Underwood and R. C. Zak, "Intel® Omni-path Architecture: Enabling Scalable, High Performance Fabrics," in 2015 IEEE 23rd Annual Symposium on High-Performance Interconnects, Santa Clara, CA, USA, 2015.
  • [12] H. Fröning, M. Nüssle, H. Litz, C. Leber and U. Brüning, "On achieving high message rates," in 13th IEEE/ACM International Symposium on Cluster, Cloud, and Grid Computing, 2013.
  • [13] A. Kreuzer, E. Suarez, N. Eicker et T. Lippert, Porting applications to a Modular Supercomputer - Experiences from the DEEP-EST project, Forschungszentrum Jülich GmbH Zentralbibliothek, Verlag Jülich, IAS Series 48, 2021, p. 254.
  • [14] Jülich Supercomputing Centre, «JURECA: General-purpose supercomputer at Jülich Supercomputing Centre,» Journal of large-scale research facilities, vol. 2, p. A62, 2016.
  • [15] Jülich Supercomputing Centre, "JURECA: Modular supercomputer at Jülich Supercomputing Centre," Journal of large-scale research facilities, vol. 4, p. A132, 2018.
  • [16] Jülich Supercomputing Centre, "JURECA: Data Centric and Booster Modules implementing the Modular Supercomputing Architecture at Jülich Supercomputing Centre," Journal of large-scale research facilities, vol. 7, p. A182, 2021.
  • [17] Jülich Supercomputing Centre, "JUWELS Cluster and Booster: Exascale Pathfinder with Modular Supercomputing Architecture at Jülich Supercomputing Centre," Journal of large-scale research facilities, vol. 7, p. A183, 2021.
  • [18] A. Herten, "JUWELS Booster - Early User Experiences," in The 30th International Symposium on High-Performance Parallel and Distributed Computing, PERMAVOST Workshop, HPDC21, Virtual, Sweden, 2021.
  • [19] S. Kesselheim, A. Herten and K. Krajsek, "JUWELS Booster – A Supercomputer for Large-Scale AI Research, High Performance Computing," in ISC High Performance 2021, Digital, Germany, 2021.