Published October 11, 2024 | Version v1
Conference paper Open

Modernising Australian geophysics datasets to enable Nationalscale multiphysics computation at Exascale: one size can no longer fit all.

  • 1. Australian National University, lesley.wyborn@anu.edu.au
  • 2. Australian National University, nigel.rees@anu.edu.au
  • 3. University of Melbourne, rebecca@auscope.org.au
  • 4. Australian National University, ben.evans@anu.edu.au
  • 5. Australian National University, hannes.hollmann@anu.edu.au
  • 6. Australian National University, jo.croucher@anu.edu.au
  • 7. rui.yang@anu.edu.au, Australian National University
  • 8. University of Melbourne, Tim@auscope.org.au

Description

Although available computational power has increased significantly in the last decade, the geophysics community has been slow to utilise the latest High Performance Computing (HPC) resources. The 2030 Geophysics Collections Project, a collaboration between the National Computational Infrastructure (NCI), AuScope, Australian Research Data Commons (ARDC) and Terrestrial Ecosystem Research Network (TERN), investigated what was required to make existing Australian geophysical data collections compliant with the FAIR Principles, and openly accessible on existing HPC resources at NCI, and ultimately scaling to Exascale resources by 2030. A selection of AuScope-funded time series datasets in Magnetotellurics (MT), Passive Seismic (PS) and Distributed Acoustic Sensing (DAS) were ingested and organised on NCI's Gadi Tier 1 supercomputer to facilitate multi-physics (re)processing, modelling and analysis at scale. For MT, optimisation for HPC required reformatting datasets into the new MTH5/ mt_metadata standard, with benchmark processing showing that generating different MT processing levels for entire surveys can now be done transparently and in a matter of minutes utilising current HPC capability. In parallel to these data activities, the NCI-geophysics and NCI-AI-ML software environments were developed to enable a seamless experience for geophysical data analysis, processing and modelling on HPC and to help lower the barrier to entry for geophysical researchers wanting to perform AI/ML analysis utilising Gadi GPU resources. The 2030 Geophysics Collections Project showed that to realise the full value of significant investments going back over 50 years in acquiring continental-scale geophysical datasets in government, industry and academia, we need to consider a collaborative effort from all three sectors to remaster our current geophysical assets into modern HPC-compatible, self-describing formats suitable for in-situ processing to enable high-resolution processing at scale, and readiness for new computational techniques. However, there is still a need to make datasets accessible in current formats: and hence one size can no longer fit all.

Files

ASEG_2024_ID100.pdf

Files (270.5 kB)

Name Size Download all
md5:b00accc057aac05bf5704b5f1ea5cd79
270.5 kB Preview Download