Open Science with openPMD
Creators
- 1. Helmholtz-Zentrum Dresden - Rossendorf, Technische Universität Dresden
- 2. Lawrence Berkeley National Lab
- 3. Lawrence Livermore National Lab
- 4. Technische Universität Dresden, Max Planck Institute of Molecular Cell Biology and Genetics
- 5. Institute for Optics and Quantum Electronics Jena
- 6. Helmholtz-Zentrum Dresden - Rossendorf
Description
Nobody needs yet an other data format for HPC. But why have so-called self-describing data formats never provided out-of-the-box cross application portability? Why are most open-access datasets not self-describing for both the domain scientist and after-use? And why do communities need to implement their data readers in various post-processing, visualization and analysis frameworks over and over again?
We present the open meta data format openPMD for data format agnostic markup of particle-mesh data. Based on a minimal kernel of meta information and enriched with domain-specific extensions, we develop an open ecosystem of interoperable simulations and data processing frameworks from the domains of laser-plasma interaction, X-ray photon sciences, astrophysics up to systems biology. This poster presents our efforts to enable & establish workflows suitable to frictionless transposition between those domains, using highly scalable I/O methods (e.g. ADIOS BP or HDF5), a truly self-describing data markup and peer reviewed participation.
Files
openPMD_PASC17_Zenodo.pdf
Files
(1.4 MB)
Name | Size | Download all |
---|---|---|
md5:026a19350c86143bfed630948cec3f88
|
1.4 MB | Preview Download |
Additional details
Related works
- Cites
- 10.5281/zenodo.33624 (DOI)
- References
- http://openpmd.org (URL)
- http://github.com/openPMD (URL)
References
- A. Huebl et al. openPMD 1.0.0: A meta data standard for particle and mesh based data, technical specification (CC-BY 4.0), November 2015, DOI:10.5281/zenodo.33624
- A. Huebl et al. On the Scalability of Data Reduction Techniques in Current and Upcoming HPC Systems from an Application Perspective, ISC 2017, arXiv:1706.00522
- H. Abbasi et al. Datastager: scalable data staging services for petascale applications, Cluster Computing 13(3), DOI:10.1007/s10586-010-0135-6
- C. Docan et al. DataSpaces: An interaction and coordination framework or coupled simulation workflows, HPDC 2010, DOI:10.1007/s10586-011-0162-y