2024-03-28T18:28:56Z
https://zenodo.org/oai2d
oai:zenodo.org:6470840
2022-04-27T01:49:55Z
user-etp4hpc
Becker, Tobias
Haas, Robert
Schemmel, Johannes
Furber, Steve
Dolas, Sagar
2022-04-25
<p>Moore’s Law, which stated that “<em>the complexity for minimum component costs has increased at a rate of roughly a factor of two per year </em>“, is slowing down due to the enormous cost of developing new process generations along with feature sizes approximating silicon interatomic distances. With the end of Dennard scaling (i.e; the rather constant power density across technology nodes, and the increase of operating frequency with each technology node) more than 15 years ago, using more transistors to build increasingly parallel architectures was a key focus. Now, other ways need to be found to deliver further advances in performance. This can be achieved by a combination of innovations at all levels: technology (3D stacking, using physics carry out computations, etc.), architecture (e.g., specialization, computing in/near memory, dataflow), software, algorithms and new ways to represent information (e.g., neuromorphic – coding information “in time” with “spikes”, quantum, mixed precision). A closely related challenge is the energy required to move data, which can be orders of magnitude higher than the energy of the computation itself.</p>
<p>These challenges give rise to new unconventional HPC architectures, which, through some form of specialisation, achieve more efficient and faster computations. This paper covers a range of new unconventional HPC architectures which are currently emerging. While these architectures and their underlying requirements are naturally diverse, AI emerges as a technology that drives the development of both novel architectures and computational techniques due to its dissemination and computing requirements. The ever-increasing demand of AI applications requires more efficient computation of data centric algorithms. Furthermore, minimising data movement and improving data locality plays an important role in achieving high performance while limiting power dissipation and this is reflected in both architectures and programming models. We also cover models of computation that differ from those used in conventional CPU architectures, or models that are purely focussed on achieving performance through parallelisation. Finally, we address the challenges of developing, porting and maintaining applications for these new architectures.</p>
https://doi.org/10.5281/zenodo.6470840
oai:zenodo.org:6470840
eng
Zenodo
https://zenodo.org/communities/etp4hpc
https://doi.org/10.5281/zenodo.6470839
info:eu-repo/semantics/openAccess
Creative Commons Attribution 4.0 International
https://creativecommons.org/licenses/by/4.0/legalcode
Unconventional HPC Architectures
info:eu-repo/semantics/report
oai:zenodo.org:5470479
2021-09-15T13:48:22Z
user-etp4hpc
Hartmann, Dirk
2021-09-15
<p>We live in a world of exploding complexity driven by technical evolution as well as highly volatile socio-economic environments. Managing complexity is a key issue in everyday decision-making such as providing safe, sustainable, and efficient industrial control solutions as well as solving today's global grand challenges such as the climate change. However, the level of complexity has reached our cognitive capability to take informed decisions. Digital Twins, tightly integrating the real and the digital world, are a key enabler to support decision making for complex systems. They allow informing operational as well as strategic decisions upfront through accepted virtual predictions and optimisations of their real-world counter parts.</p>
<p><strong><em>Digital Twins</em></strong> [6] <em>are specific virtual representations of physical objects. A Digital Twin integrates all data, models, and other information of a physical asset generated along its life cycle for a dedicated purpose. This is typically reproducing the state and behaviour of the corresponding system as well as predicting and optimising its performance. To this purpose, simulation methods and data-based methods are used.</em></p>
<p>Depending on the specific nature, application, and context a wide variety of nomenclature has been introduced, see e.g. [2,4,7,14]. Here, we focus on real-time Digital Twins for online prediction and optimisation of highly dynamic industrial assets and processes. By their nature, Digital Twins integrate and tightly connect several digital key technologies including mathematical modelling, simulation, and optimisation; data analytics, machine learning, and artificial intelligence; data and compute platforms from edge to cloud computing; cybersecurity; human computer interaction; and many more. Only a coordinated research effort as envisaged by the TransContinuum Initiative will allow the realisation of the full potential of Digital Twins - a key tool for decision making addressing today's industrial as well as global challenges.</p>
A Transcontinuum Initiative Use Case
https://doi.org/10.5281/zenodo.5470479
oai:zenodo.org:5470479
eng
Zenodo
https://zenodo.org/communities/etp4hpc
https://doi.org/10.5281/zenodo.5470478
info:eu-repo/semantics/openAccess
Creative Commons Attribution 4.0 International
https://creativecommons.org/licenses/by/4.0/legalcode
Real-Time Digital Twins
info:eu-repo/semantics/report
oai:zenodo.org:6107362
2022-02-21T13:50:00Z
user-etp4hpc
user-eu
Marazakis, Manolis
Duranton, Marc
Pleiter, Dirk
Taffoni, Giuliano
Hoppe, Hans-Christian
2022-02-21
<p>Emerging use cases from incident response planning and broad-scope European initiatives (e.g. Destination Earth [1,2], European Green Deal and Digital Package [21]) are expected to require federated, distributed infrastructures combining computing and data platforms. These will provide elasticity enabling users to build applications and integrate data for thematic specialisation and decision support, within ever shortening response time windows.</p>
<p>For prompt and, in particular, for urgent decision support, the conventional usage modes of HPC centres is not adequate: these rely on relatively long-term arrangements for time-scheduled exclusive use of HPC resources, and enforce well-established yet time-consuming policies for granting access. In urgent decision support scenarios, managers or members of incident response teams must initiate processing and control the resources required based on their real-time judgement on how a complex situation evolves over time. This circle of clients is distinct from the regular users of HPC centres, and they must interact with HPC workflows on-demand and in real-time, while engaging significant HPC and data processing resources in or across HPC centres.</p>
<p>This white paper considers the technical implications of supporting urgent decisions through establishing flexible usage modes for computing, analytics and AI/ML-based applications using HPC and large, dynamic assets.</p>
<p>The target decision support use cases will involve ensembles of jobs, data-staging to support workflows, and interactions with services/facilities external to HPC systems/centres. Our analysis identifies the need for flexible and interactive access to HPC resources, particularly in the context of dynamic workflows processing large datasets. This poses several technical and organisational challenges: short-notice secure access to HPC and data resources, dynamic resource allocation and scheduling, coordination of resource managers, support for data-intensive workflow (including data staging on node-local storage), preemption of already running workloads and interactive steering of simulations. Federation of services and resources across multiple sites will help to increase availability, provide elasticity for time-varying resource needs and enable leverage of data locality.</p>
The authors would like to thank Maria S. Perez (Professor at Universidad Politécnica de Madrid, Spain) for her insightful critique on earlier drafts of this whitepaper.
https://doi.org/10.5281/zenodo.6107362
oai:zenodo.org:6107362
eng
Zenodo
https://zenodo.org/communities/eu
https://zenodo.org/communities/etp4hpc
https://doi.org/10.5281/zenodo.6107361
info:eu-repo/semantics/openAccess
Creative Commons Attribution 4.0 International
https://creativecommons.org/licenses/by/4.0/legalcode
HPC for Urgent Decision-Making
info:eu-repo/semantics/report
oai:zenodo.org:5555960
2021-10-14T01:48:32Z
user-etp4hpc
Bartsch, Valeria
Colin de Verdière, Guillaume
Nominé, Jean-Philippe
Ottaviani, Daniele
Dragoni, Daniele
Bouazza, Chayma
Magugliani, Fabrizio
Bowden, David
Allouche, Cyril
Johansson, Mikael
Terzo, Olivier
Scarabosio, Andrea
Vitali, Giacomo
Shagieva, Farida
Michielsen, Kristel
2021-10-08
<p>Quantum Computing (QC) describes a new way of computing based on the principles of quantum mechanics. From a High Performance Computing (HPC) perspective, QC needs to be integrated:</p>
<ul>
<li>at a system level, where quantum computer technologies need to be integrated in HPC clusters;</li>
<li>at a programming level, where the new disruptive ways of programming devices call for a full hardware-software stack to be built;</li>
<li>at an application level, where QC is bound to lead to disruptive changes in the complexity of some applications so that compute-intensive or intractable problems in the HPC domain might become tractable in the future.</li>
</ul>
<p>The White Paper QC for HPC focuses on the technology integration of QC in HPC clusters, gives an overview of the full hardware-software stack and QC emulators, and highlights promising customised QC algorithms for near-term quantum computers and its impact on HPC applications. In addition to universal quantum computers, we will describe non-universal QC where appropriate. Recent research references will be used to cover the basic concepts. Thetarget audience of this paper is the European HPC community: members of HPC centres, HPC algorithm developers, scientists interested in the co-design for quantum hardware, benchmarking, etc.</p>
https://doi.org/10.5281/zenodo.5555960
oai:zenodo.org:5555960
Zenodo
https://zenodo.org/communities/etp4hpc
https://doi.org/10.5281/zenodo.5555959
info:eu-repo/semantics/openAccess
Creative Commons Attribution 4.0 International
https://creativecommons.org/licenses/by/4.0/legalcode
< QC | HPC >: Quantum for HPC
info:eu-repo/semantics/report
oai:zenodo.org:6451288
2022-04-21T13:49:14Z
user-etp4hpc
Haus, Utz-Uwe
Narasimharmurthy, Sai
Perez, Maria S.
Wierse, Andreas
2022-04-11
<p>An increasing interest is observed in making a diversity of compute and storage resources, which are geographic spread, available in a federated manner. A common services layer can facilitate easier access, more elasticity as well as lower response times, and improved utilisation of the underlying resources. In this white paper, current trends are analysed both from an infrastructure provider as well as an end-user perspective. Here the focus is on federated e-infrastructures that among others include high-performance computing (HPC) systems as compute resources. Two initiatives, namely Fenix and GAIA-X, are presented as illustrative examples. Based on a more detailed exploration of selected topical areas related to federated e-infrastructures, various R&I challenges are identified and recommendations for further efforts formulated.</p>
https://doi.org/10.5281/zenodo.6451288
oai:zenodo.org:6451288
eng
Zenodo
https://zenodo.org/communities/etp4hpc
https://doi.org/10.5281/zenodo.6451287
info:eu-repo/semantics/openAccess
Creative Commons Attribution 4.0 International
https://creativecommons.org/licenses/by/4.0/legalcode
Federated HPC, Cloud and Data infrastructures
info:eu-repo/semantics/report
oai:zenodo.org:7347009
2022-11-22T14:26:33Z
user-etp4hpc
Malms, Michael
Cargemel, Laurent
Suarez, Estela
Mittenzwey, Nico
Duranton, Marc
Sezer, Sakir
Prunty, Craig
Rossé-Laurent, Pascale
Pérez-Harnandez, Maria
Marazakis, Manolis
Lonsdale, Guy
Carpenter, Paul
Antoniu, Gabriel
Narasimharmurthy, Sai
Brinkman, André
Pleiter, Dirk
Haus, Utz-Uwe
Krueger, Jens
Hoppe, Hans-Christian
Laure, Erwin
Wierse, Andreas
Bartsch, Valeria
Michielsen, Kristel
Allouche, Cyril
Becker, Tobias
Haas, Robert
2022-10-18
<p>This document feeds research and development priorities devel-oped by the European HPC ecosystem into EuroHPC’s Research and Innovation Advisory Group with an aim to define the HPC Technology research Work Programme and the calls for proposals included in it and to be launched from 2023 to 2026.<br>
This SRA also describes the major trends in the deployment of HPC and HPDA methods and systems, driven by economic and societal needs in Europe, taking into account the changes ex-pected in the technologies and architectures of the expanding underlying IT infrastructure. The goal is to draw a complete pic-ture of the state of the art and the challenges for the next three to four years rather than to focus on specific technologies, implementations or solutions.</p>
https://doi.org/10.5281/zenodo.7347009
oai:zenodo.org:7347009
eng
Zenodo
https://zenodo.org/communities/etp4hpc
https://doi.org/10.5281/zenodo.7347008
info:eu-repo/semantics/openAccess
Creative Commons Attribution 4.0 International
https://creativecommons.org/licenses/by/4.0/legalcode
ETP4HPC's SRA 5 - Strategic Research Agenda for High-Performance Computing in Europe - 2022
info:eu-repo/semantics/report
oai:zenodo.org:6508394
2022-05-14T01:49:43Z
user-etp4hpc
user-eu
Suarez, Estela
Eicker, Norbert
Moschny, Thomas
Pickartz, Simon
Clauss, Carsten
Plugaru, Valentin
Herten, Andreas
Michielsen, Kristel
Lippert, Thomas
2022-05-11
<p>The European Community and its member states regularly invest large volumes of funding and effort in the development of HPC technologies in Europe. However, some observers express the criticism that these investments are either unfocused, lack long-term perspectives, or that their results are not mature enough to be adopted by the mainstream developments, which limits their benefit for the European HPC community, industry and society. This paper is intended as a counterexample to this pessimistic view. It describes the success story of Modular Supercomputing Architecture, which started in 2011 with the EU-funded R&D project “DEEP”, and is now being adopted by large-scale supercomputing centres across the old continent and worldwide. Main hardware and software characteristics of the architecture and some of the systems using it are described, complemented by a historical view of its development, the lessons learned in the process and future prospects.</p>
https://doi.org/10.5281/zenodo.6508394
oai:zenodo.org:6508394
eng
Zenodo
https://zenodo.org/communities/eu
https://zenodo.org/communities/etp4hpc
https://doi.org/10.5281/zenodo.6508393
info:eu-repo/semantics/openAccess
Creative Commons Attribution 4.0 International
https://creativecommons.org/licenses/by/4.0/legalcode
Modular Supercomputing Architecture
info:eu-repo/semantics/report
oai:zenodo.org:5549731
2021-10-06T13:48:29Z
user-etp4hpc
user-eu
Aumage, Olivier
Carpenter, Paul
Benkner, Siegfried
2021-10-05
<p>As HPC hardware continues to evolve and diversify and workloads become more dynamic and complex, applications need to be expressed in a way that facilitates high performance across a range of hardware and situations. The main application code should be platform-independent, malleable and asynchronous with an open, clean, stable and dependable interface between the higher levels of the application, library or programming model and the kernels and software layers tuned for the machine. The platform-independent part should avoid direct references to specific resources and their availability, and instead provide the information needed to optimise behaviour.</p>
<p>This paper summarises how task abstraction, which first appeared in the 1990s and is already mainstream in HPC, should be the basis for a composable and dynamic performance-portable interface. It outlines the innovations that are required in the programming model and runtime layers, and highlights the need for a greater degree of trust among application developers in the ability of the underlying software layers to extract full performance. These steps will help realise the vision for performance portability across current and future architectures and problems.</p>
This work was supported by the Spanish Government (contract PID2019-107255GB), Generalitat de Catalunya (contract 2014-SGR-1051), and the European Union's Horizon 2020 research and innovation programme under grant agreements No 955606 (DEEP-SEA) and No 754337 (EuroEXA). Paul Carpenter holds the Ramon y Cajal fellowship under contracts RYC2018-025628-I of the Ministry of Economy and Competitiveness of Spain. This work was supported by the French Government (contract ANR-19-CE46-0009), Région Nouvelle Aquitaine (contract 018-1R50119) and the European Union's Horizon 2020 research and innovation programme under grant agreements No 671602 (INTERTWinE) and No 801015 (EXA2PRO). This work was supported by the Austrian Science Fund grant P29783.
https://doi.org/10.5281/zenodo.5549731
oai:zenodo.org:5549731
eng
Zenodo
https://zenodo.org/communities/eu
https://zenodo.org/communities/etp4hpc
https://doi.org/10.5281/zenodo.5549730
info:eu-repo/semantics/openAccess
Creative Commons Attribution 4.0 International
https://creativecommons.org/licenses/by/4.0/legalcode
Task-Based Performance Portability in HPC
info:eu-repo/semantics/report
oai:zenodo.org:4605344
2021-03-15T16:30:30Z
user-etp4hpc
Malossi, Cristiano
Bodin, Francois
Lavignon, Jean-Francois
Nominé, Jean-Philippe
Asch, Mark
Vermesan, Ovidiu
Bauer, Peter
Requena, Stephane
Scionti, Alberto
Costan, Alexandru
Ferretti, Andrea
Bilas, Angelos
Anciaux-Sedrakian, Ani
Queralt, Anna
Peña, Antonio J.
Depardon, Benjamin
D'Amico, Carmine
Calvin, Christophe
Kozanitis, Christos
Morey, Colin
Molka, Daniel
Garcia-Gasulla, Dario
Hartmann, Dirk
Audit, Edouard
Brun, Emeric
Chaix, Fabien
Boillod-Cerneux, France
Shainer, Gilad
Wiber, Gilles
Colin de Verdière, Guillaume
Lafoucrière, Jacques-Charles
Denis, Jean-Marc
Acquaviva, Jean-Thomas
Guitart, Jordi
Bigot, Julien
Corbolan, Julita
Bautista Gomez, Leonardo Arturo
Axner, Lillit
Mason, Luke
Ploumidis, Manolis
Casas, Marc
Perache, Marc
Hautreux, Matthieu
Vazquez, Miguel
Bat, Nejc
Bergeret, Nicolas
Tonello, Nicolas
Wedi, Nils
Marsden, Olivier
Terzo, Olivier
Unsal, Osman
Carribault, Patrick
Radojkovic, Petar
Bricard, Philippe
Deniel, Philippe
Pratikakis, Polyvios
Nou, Ramon
Borrell, Ricard
Graham, Richard
Pinning, Robin
Apostolov, Rossen
Pllana, Sabri
Ryan, Sinead
Mazumdar, Somnath
Markidis, Stefano
Reinemo, Sven-Arne
Goubier, Thierry
Quintino, Tiago
Haus, Utz-Uwe
Plugaru, Valentin
Bartsch, Valeria
Alexandrov, Vassil
Papaefstathiou, Vassilis
Beltran, Vicenç
Martorell, Xavier
Cai, Xing
Papaefstathiou, Yannis
Becerra, Yolanda
Malms, Michael
Ostasz, Marcin
Gilliot, Maike
Bernier-Bruna, Pascale
Cargemel, Laurent
Suarez, Estela
Cornelius, Herbert
Duranton, Marc
Koren, Benny
Rosse-Laurent, Pascale
Pérez-Hernández, María S.
Marazakis, Manolis
Lonsdale, Guy
Carpenter, Paul
Antoniu, Gabriel
Narasimhamurthy, Sai
Brinkman, André
Pleiter, Dirk
Tate, Adrian
Krueger, Jens
Hoppe, Hans-Christian
Laure, Erwin
Wierse, Andreas
2020-03-03
<p>This Strategic Research Agenda (SRA) is the fourth High Performance Computing (HPC) technology roadmap developed and maintained by ETP4HPC, with the support of the <a href="https://exdci.eu/">EXDCI-2</a> project. It continues the tradition of a structured approach to the identification of key research objectives. The main objective of this SRA is to identify the European technology research priorities in the area of HPC and High-Performance Data Analytics (HPDA), which should be used by EuroHPC to build its 2021 – 2024 Work Programme.</p>
<p>Over eighty HPC experts associated with member organisations of ETP4HPC created this document in collaboration with external technical leaders representing those areas of technology that together with HPC form what we have come to call <strong>“The Digital Continuum”</strong>. This new concept well reflects the main trend of this SRA – it is not only about developing HPC technology in order to build competitive European HPC systems but also about making our HPC solutions work together with other related technologies - the material included in this SRA is also a result of our interactions with Big Data, Internet of Things (IoT), Artificial Intelligence (AI) and Cyber Physical Systems (CPS).</p>
https://doi.org/10.5281/zenodo.4605344
oai:zenodo.org:4605344
eng
Zenodo
https://zenodo.org/communities/etp4hpc
https://doi.org/10.5281/zenodo.4605343
info:eu-repo/semantics/openAccess
Creative Commons Attribution 4.0 International
https://creativecommons.org/licenses/by/4.0/legalcode
ETP4HPC's Strategic Research Agenda for High-Performance Computing in Europe 4
info:eu-repo/semantics/other
oai:zenodo.org:4683451
2021-05-10T11:31:45Z
user-etp4hpc
user-eu
Bauer, Peter
Duranton, Marc
Malms, Michael
2021-01-18
<p>Dealing responsibly with extreme events requires not only a drastic change in the ways society addresses its energy and population crises. It also requires a new capability for using present and future information on the Earth system to reliably predict the occurrence and impact of such events. A breakthrough in Europe’s predictive capability can be made manifest through science and technology solutions delivering as yet unseen levels of predictive reliability with real value for society.<br>
The “TransContinuum Initiative”, initiated by ETP4HPC, ECSO, BDVA, 5GIA, EU MATHS IN, CLAIRE, AIOTI and HiPEAC, offers unprecedented opportunities to overcome the technological limitations currently hampering progress in this area. Beyond providing this use case with better technology solutions, the initiative offers a foundation for an Earth system – computational science collaboration that will eventually lead to science and technology being truly co-developed, and thus to sustainable benefit for one of today’s most relevant applications and for European technology providers.</p>
This article was initially published as part of the HiPEAC Vision available at hipeac.net/vision.
V.1, January 2021:
P. Bauer, M. Duranton, and M. Malms. The extremes prediction use case. In M. Duranton et al., editors, HiPEAC Vision 2021, pages 44-49, Jan 2021.
https://doi.org/10.5281/zenodo.4683451
oai:zenodo.org:4683451
eng
Zenodo
isbn:978-90-78427-02-5
https://doi.org/10.5281/zenodo.4719362
https://zenodo.org/communities/eu
https://zenodo.org/communities/etp4hpc
https://doi.org/10.5281/zenodo.4683450
info:eu-repo/semantics/openAccess
Creative Commons Attribution 4.0 International
https://creativecommons.org/licenses/by/4.0/legalcode
The extremes prediction use case
info:eu-repo/semantics/report
oai:zenodo.org:4767489
2021-09-18T01:48:29Z
user-etp4hpc
user-eu
Radojković, Petar
Carpenter, Paul
Esmaili-Dokht, Pouya
Cimadomo, Rémy
Charles, Henri-Pierre
Sebastian, Abu
Amato, Paolo
2021-07-29
<p>Decades after being initially explored in the 1970s, Processing in Memory (PIM) is currently experiencing a renaissance. By moving part of the computation to the memory devices, PIM addresses a fundamental issue in the design of modern computing systems, the mismatch between the von Neumann architecture and the requirements of important data-centric applications. A number of industrial prototypes and products are under development or already available in the marketplace, and these devices show the potential for cost-effective and energy-efficient acceleration of HPC, AI and data analytics workloads. This paper reviews the reasons for the renewed interest in PIM and surveys industrial prototypes and products, discussing their technological readiness.</p>
<p>Wide adoption of PIM in production, however, depends on our ability to create an ecosystem to drive and coordinate innovations and co-design across the whole stack. European companies and research centres should be involved in all aspects, from technology, hardware, system software and programming environment, to updating of the algorithm and application. In this paper, we identify the main challenges that must be addressed and we provide guidelines to prioritise the research efforts and funding. We aim to help make PIM a reality in production HPC, AI and data analytics.</p>
This work was supported by the by the Spanish Government (contract PID2019-107255GB), Generalitat de Catalunya (contracts 2017-SGR-1328 and 2017-SGR-1414), and the European Union's Horizon 2020 research and innovation programme under grant agreements No 955606 (DEEP-SEA) and No 682675 (Projected Memristor European Research Council grant). Paul Carpenter holds the Ramon y Cajal fellowship under contracts RYC2018-025628-I of the Ministry of Economy and Competitiveness of Spain. This work was also supported by the Collaboration Agreement between Micron Technology, Inc. and BSC. The authors wish to thank Xavier Martorell from BSC for his technical support, and Manolis Marazakis and André Brinkmann for their feedback.
https://doi.org/10.5281/zenodo.4767489
oai:zenodo.org:4767489
eng
Zenodo
https://zenodo.org/communities/eu
https://zenodo.org/communities/etp4hpc
https://doi.org/10.5281/zenodo.4767488
info:eu-repo/semantics/openAccess
Creative Commons Attribution 4.0 International
https://creativecommons.org/licenses/by/4.0/legalcode
Processing in Memory
PIM
Processing in Memory: The Tipping Point
info:eu-repo/semantics/report
oai:zenodo.org:5534464
2021-10-01T01:48:38Z
user-etp4hpc
user-eu
Antoniu, Gabriel
Valduriez, Patrick
Hoppe, Hans-Christian
Krüger, Jens
2021-09-28
<p>Modern use cases such as autonomous vehicles, digital twins, smart buildings and precision agriculture, greatly increase the complexity of application workflows. They typically combine physics-based simulations, analysis of large data volumes and machine learning and require a hybrid execution infrastructure: edge devices create streams of input data, which are processed by data analytics and machine learning applications in the Cloud, and simulations on large, specialised HPC systems provide insights into and prediction of future system state. From these results, additional steps create and communicate output data across the infrastructure levels, and for some use cases, control devices or cyber-physical systems in the real world are controlled (as in the case of smart factories). All of these steps pose different requirements for the best suited execution platforms, and they need to be connected in an efficient and secure way. This assembly is called the <em>Computing Continuum</em> (CC) (1). It raises challenges at multiple levels: at the application level, innovative algorithms are needed to bridge simulations, machine learning and data-driven analytics; at the middleware level, adequate tools must enable efficient deployment, scheduling and orchestration of the workflow components across the whole distributed infrastructure; and, finally, a capable resource management system must allocate a suitable set of components of the infrastructure to run the application workflow, preferably in a dynamic and adaptive way, taking into account the specific capabilities of each component of the underlying heterogeneous infrastructure.</p>
<p>To address the challenges, we foresee an increasing need for integrated software ecosystems which combine current “island” solutions and bridge the gaps between them. These ecosystems must facilitate the full lifecycle of CC use cases, including initial modelling, programming, deployment, execution, optimisation, as well as monitoring and control. It will be important to ensure adequate reproducibility of workflow results and to find ways for creating and managing trust when sharing systems, software and data. All of these will in turn require novel or improved hardware capabilities. This white paper provides an initial discussion of the gaps. Our objective is to accelerate progress in both hardware and software infrastructures to build CC use cases, with the ultimate goals of accelerating scientific discovery, improving timeliness, quality and sustainability of engineering artefacts, and supporting decisions in complex and potentially urgent situations</p>
The authors would like to thank Rafael Mayo-García from CIEMAT and Marion Carrier from CybeleTech for their help in describing relevant use cases for the computing continuum.
https://doi.org/10.5281/zenodo.5534464
oai:zenodo.org:5534464
eng
Zenodo
https://zenodo.org/communities/eu
https://zenodo.org/communities/etp4hpc
https://doi.org/10.5281/zenodo.5534463
info:eu-repo/semantics/openAccess
Creative Commons Attribution 4.0 International
https://creativecommons.org/licenses/by/4.0/legalcode
Towards Integrated Hardware/Software Ecosystems for the Edge-Cloud-HPC Continuum
info:eu-repo/semantics/report
oai:zenodo.org:6090425
2022-03-08T01:49:06Z
user-etp4hpc
user-eu
Carpenter, Paul
Utz, Uwe-Haus
Narasimhamurthy, Sai
Suarez, Estela
2022-02-15
<p>Modern HPC systems are becoming increasingly heterogeneous, affecting all components of HPC systems, from the processing units, through memory hierarchies and network components to storage systems. This trend is on the one hand due to the need to build larger, yet more energy efficient systems, and on the other hand it is caused by the need to optimise (parts of the) systems for certain workloads. In fact, it is not only the systems themselves that are becoming more heterogeneous, but also scientific and industrial applications are increasingly combining different technologies into complex workflows, including simulation, data analytics, visualisation, and artificial intelligence/machine learning. Different steps in these workflows call for different hardware and thus today’s HPC systems are often composed of different modules optimised to suit certain stages in these workflows.</p>
<p>While the trend towards heterogeneity is certainly helpful in many aspects, it makes the task of programming these systems and using them efficiently much more complicated. Often, a combination of different programming models is required and selecting suitable technologies for certain tasks or even parts of an algorithm is difficult. Novel methods might be needed for heterogeneous components or be only facilitated by them. And this trend is continuing, with new technologies around the corner that will further increase heterogeneity, e.g. neuromorphic or quantum accelerators, in-memory-computing, and other non-von-Neumann approaches.</p>
<p>In this paper, we present an overview of the different levels of heterogeneity we find in HPC technologies and provide recommendations for research directions to help deal with the challenges they pose. We also point out opportunities that particularly applications can profit from by exploiting these technologies. Research efforts will be needed over the full spectrum, from system architecture, compilers and programming models/languages, to runtime systems, algorithms and novel mathematical approaches.</p>
https://doi.org/10.5281/zenodo.6090425
oai:zenodo.org:6090425
eng
Zenodo
https://zenodo.org/communities/eu
https://zenodo.org/communities/etp4hpc
https://doi.org/10.5281/zenodo.6090424
info:eu-repo/semantics/openAccess
Creative Commons Attribution 4.0 International
https://creativecommons.org/licenses/by/4.0/legalcode
Heterogeneous High Performance Computing
info:eu-repo/semantics/report