Supplementary material for "A Massively Parallel Time-Domain Coupled Electrodynamics-Micromagnetics Solver"

This directory contains the inputs, outputs, job submission scripts and github hashes used to generate perform our simulations and create the figures presented in "A Massively Parallel Time-Domain Coupled Electrodynamics-Micromagnetics Solver" by Yao et al. and submitted to "The International Journal of High Performance Computing Applications."

The source code for ARTEMIS is here

https://github.com/ECP-WarpX/artemis.git

(Commit hash used : 64e32d56767e6f8027de81a8c4897b808f211267)

and AMReX code is here :

https://github.com/AMReX-Codes/amrex.git

(Commit hash : 7c3e95210a1877c67139ea57c62fbcc717acc1c3)

The following modules were loaded for performing simulations on NERSC Cori GPU:

1) cgpu 2) gcc 3) cuda/10.2.89 4)mvapich2

and on NERSC Haswell Partition

  1) modules/3.2.11.4
  2) altd/2.0
  3) darshan/3.2.1
  4) craype-network-aries
  5) gcc/8.3.0
  6) craype/2.6.2
  7) cray-mpich/7.7.10
  8) craype-haswell
  9) craype-hugepages2M
 10) cray-libsci/19.06.1
 11) udreg/2.3.2-7.0.1.1_3.47__g8175d3d.ari
 12) ugni/6.0.14.0-7.0.1.1_7.49__ge78e5b0.ari
 13) pmi/5.0.14
 14) dmapp/7.1.1-7.0.1.1_4.61__g38cf134.ari
 15) gni-headers/5.0.12.0-7.0.1.1_6.36__g3b1768f.ari
 16) xpmem/2.2.20-7.0.1.1_4.20__g0475745.ari
 17) job/2.2.4-7.0.1.1_3.47__g36b56f4.ari
 18) dvs/2.12_2.2.165-7.0.1.1_14.4__ge967908e
 19) alps/6.6.58-7.0.1.1_6.19__g437d88db.ari
 20) rca/2.2.20-7.0.1.1_4.61__g8e3fb5b.ari
 21) atp/2.1.3
 22) PrgEnv-gnu/6.0.5

The input files, datasets, and jupyter notebooks (StrongScaling/strong_scaling_analysis.ipynb and WeakScaling/weak_scaling_plots.ipynb) used to generate the weak and strong scaling for CPU and GPU performance, shown in Figures 2, 3(a) and 3(b), are in the directory named "InputFilesAndDataSetForScalingStudies".

The simulation data sets and analysis scripts for Figures 5, 6, and 7 in the paper are in the directory named "DataSetForEmptyWaveguideAndTunableFilter.tar.gz". The runs for the waveguide simulations used to produce Figures 5, 6, and 7 were run on a local LBNL work-station with a single NVIDIA Quadro GV100 GPU. The input files are located in the ARTEMIS repository (https://github.com/ECP-WarpX/artemis.git) at Examples/Waveguide/.