There is a newer version of the record available.

Published August 18, 2023 | Version v1
Video/Audio Open

Dragon Proxy Runtimes and Multi-system Workflows

  • 1. Hewlett Packard Enterprise

Description

We present a novel method for obtaining proxy access to remote instances of the Dragon distributed runtime. Dragon is a composable distributed runtime for managing dynamic processes, high-performance communication objects, memory and data at scale that is based on an abstraction of a distributed system. Proxy access to a remote instance of the Dragon runtime allows the client Dragon runtime to run any command that could be run directly by the remote Dragon runtime, but executes the command on the remote runtime. Commands to be run on a remote Dragon runtime are mediated by a Python object that acts as a proxy for the remote runtime, which we call a \textit{proxy runtime}. These proxy runtimes, combined with the ability to start and tear down remote Dragon runtimes both programmatically and via the command line interface, make a number of challenging workflows simple to program. Such workflows include edge-to-cloud scientific workflows, batch services and scientific applications based on Python multiprocessing. The ability to program complex workflows on systems that span clusters, scientific instruments and cloud resources is critical to the development of post-exascale applications, infrastuctures and frameworks.

Files

dragon_multisystem_workflows.mp4

Files (31.6 MB)

Name Size Download all
md5:704029c3a6df3d68e76f5f0a5958801e
31.6 MB Preview Download