April 2019 Darshan counters from the Cori supercomputer

Copyright (c) 2022, The Regents of the University of California, through Lawrence Berkeley National Laboratory (subject to receipt of any required approvals from the U.S. Dept. of Energy). All rights reserved.

 

GENERAL INFORMATION

 

1. Title of Dataset: April 2019 Darshan counters from the Cori supercomputer

 

2. Author Information:

 

    A. Principal Investigator Contact Information

        - Name: Suren Byna

        - Institution: Lawrence Berkeley National Laboratory

        - Address: 1 Cyclotron Rd, Berkeley, CA 94720

        - Email: sbyna@lbl.gov

 

    B. Associate or Co-investigator Contact

        - Name: Jean Luca Bez

        - Institution: Lawrence Berkeley National Laboratory

        - Address: 1 Cyclotron Rd, Berkeley, CA 94720

        - Email: jlbez@lbl.gov

 

3. Date of data collection: 2019-04-01

 

4. Geographic location of data collection: Berkeley, California, USA

 

5. Information about funding sources that supported the collection of the data:

 

    This research used resources of the National Energy Research Scientific Computing Center (NERSC), a U.S. Department of Energy Office of Science User Facility located at Lawrence Berkeley National Laboratory, operated under Contract No. DE-AC02-05CH11231.

 

SHARING/ACCESS INFORMATION

 

1. Licenses/restrictions placed on the data:

 

    - Creative Commons - Attribution-NonCommercial-NoDerivatives 4.0 International (CC BY-NC-ND 4.0)

 

2. Links to publications that cite or use the data:  N/A

 

3. Links to other publicly accessible locations of the data: N/A

 

4. Links/relationships to ancillary data sets: N/A

 

5. Was data derived from another source? YES

 

    This dataset is derived from raw Darshan logs collected in Cori during April 2020.

 

6. Recommended citation for this dataset:

 

    Jean Luca Bez, Ahmad Maroof Karimi, Arnab K. Paul, Bing Xie, Suren Byna, Philip Carns, Sarp Oral, Feiyi Wang, and Jesse Hanley. 2022. Access Patterns and Performance Behaviors of Multi-layer Supercomputer I/O Subsystems under Production Load. In Proceedings of the 31st International Symposium on High-Performance Parallel and Distributed Computing (HPDC '22). Association for Computing Machinery, New York, NY, USA.

 

DATA & FILE OVERVIEW

 

1. File List:

 

    1. `darshan`: Dataset derived from raw Darshan logs collected in Cori during April 2020. Months are organized as directories with a `Darshan.csv` file in each day containing all the counters collected and relevant to our investigation.

 

    2. `domains`: Dataset maps job identification to their primary associated science domain as reported by NERSC upon project registration. A single CSV comprises all jobs in April 2020 covered or not by Darshan.

 

2. Additional related data collected that was not included in the current data package:

 

    - The original raw Darshan logs are not included in this package.

 

4. Are there multiple versions of the dataset?

 

    - No, this is the first version of this dataset.

 

DESCRIPTIONS FOR VARIABLES USED IN THE FILES

 

1. Darshan counter dataset

 

* `uid`: User identifier that ran this job

* `jobid`: Job identifier from the SLURM scheduler

* `rank`: Rank identifier

* `recordid`: File record identifier

* `fstype`: File system type

* `POSIX_OPENS`: Number of open calls in POSIX

* `POSIX_FILENOS`: Number of POSIX fileno operations

* `POSIX_DUPS`: Number of POSIX dup operations

* `POSIX_READS`: Number of POSIX read operations

* `POSIX_WRITES`: Number of POSIX write operations

* `POSIX_SEEKS`: Number of POSIX seek operations

* `POSIX_STATS`: Number of POSIX stat operations

* `POSIX_MMAPS`: Number of POSIX mmap operations

* `POSIX_FSYNCS`: Number of POSIX fsync operations

* `POSIX_FDSYNCS`: Number of POSIX fdatasync operations

* `POSIX_BYTES_READ`: Total number of bytes that were read from the file

* `POSIX_BYTES_WRITTEN`: Total number of bytes written to the file

* `POSIX_CONSEC_READS`: Number of consecutive reads (that were immediately adjacent to the previous access)

* `POSIX_CONSEC_WRITES`: Number of consecutive writes (that were immediately adjacent to the previous access)

* `POSIX_SEQ_READS`: Number of sequential reads (at a higher offset than where the previous access left off)

* `POSIX_SEQ_WRITES`: Number of sequential writes (at a higher offset than where the previous access left off)

* `POSIX_SIZE_READ_0_100`: Number of read access between 0 and 100 bytes at POSIX level

* `POSIX_SIZE_READ_100_1K`: Number of read access between 100 bytes and 1KB at POSIX level

* `POSIX_SIZE_READ_1K_10K`: Number of read access between 1KB and 10KB at POSIX level

* `POSIX_SIZE_READ_10K_100K`: Number of read access between 10KB and 100KB at POSIX level

* `POSIX_SIZE_READ_100K_1M`: Number of read access between 100KB and 1MB at POSIX level

* `POSIX_SIZE_READ_1M_4M`: Number of read access between 1MB and 4MB at POSIX level

* `POSIX_SIZE_READ_4M_10M`: Number of read access between 4MB and 10MB at POSIX level

* `POSIX_SIZE_READ_10M_100M`: Number of read access between 10MB and 100MB at POSIX level

* `POSIX_SIZE_READ_100M_1G`: Number of read access between 100MB and 1GB at POSIX level

* `POSIX_SIZE_READ_1G_PLUS`: Number of read access greater than 1GB at POSIX level

* `POSIX_SIZE_WRITE_0_100`: Number of write access between 0 and 100 bytes at POSIX level

* `POSIX_SIZE_WRITE_100_1K`: Number of write access between 100 bytes and 1KB at POSIX level

* `POSIX_SIZE_WRITE_1K_10K`: Number of write access between 1KB and 10KB at POSIX level

* `POSIX_SIZE_WRITE_10K_100K`: Number of write access between 10KB and 100KB at POSIX level

* `POSIX_SIZE_WRITE_100K_1M`: Number of write access between 100KB and 1MB at POSIX level

* `POSIX_SIZE_WRITE_1M_4M`: Number of write access between 1MB and 4MB at POSIX level

* `POSIX_SIZE_WRITE_4M_10M`: Number of write access between 4MB and 10MB at POSIX level

* `POSIX_SIZE_WRITE_10M_100M`: Number of write access between 10MB and 100MB at POSIX level

* `POSIX_SIZE_WRITE_100M_1G`: Number of write access between 100MB and 1GB at POSIX level

* `POSIX_SIZE_WRITE_1G_PLUS`: Number of write access greater than 1GB at POSIX level

* `POSIX_F_OPEN_START_TIMESTAMP`: Timestamp that the first POSIX file open operation began

* `POSIX_F_READ_START_TIMESTAMP`: Timestamp that the first POSIX file read operation began

* `POSIX_F_WRITE_START_TIMESTAMP`: Timestamp that the first POSIX file write operation began

* `POSIX_F_CLOSE_START_TIMESTAMP`: Timestamp that the first POSIX file close operation began

* `POSIX_F_OPEN_END_TIMESTAMP`: Timestamp that the last POSIX file open operation ended

* `POSIX_F_READ_END_TIMESTAMP`: Timestamp that the last POSIX file read operation ended

* `POSIX_F_WRITE_END_TIMESTAMP`: Timestamp that the last POSIX file write operation ended

* `POSIX_F_CLOSE_END_TIMESTAMP`: Timestamp that the last POSIX file close operation ended

* `POSIX_F_READ_TIME`: Cumulative time spent reading at the POSIX level

* `POSIX_F_WRITE_TIME`: Cumulative time spent in write, fsync, and fdatasync at the POSIX level

* `POSIX_F_META_TIME`: Cumulative time spent in open, close, stat, and seek at the POSIX level

* `MPIIO_INDEP_OPENS`: Number of non-collective MPI opens

* `MPIIO_COLL_OPENS`: Number of collective MPI opens

* `MPIIO_INDEP_READS`: Number of non-collective MPI reads

* `MPIIO_INDEP_WRITES`: Number of non-collective MPI writes

* `MPIIO_COLL_READS`: Number of collective MPI reads

* `MPIIO_COLL_WRITES`: Number of collective MPI writes

* `MPIIO_SPLIT_READS`: Number of MPI split collective reads

* `MPIIO_SPLIT_WRITES`: Number of MPI split collective writes

* `MPIIO_NB_READS`: Number of MPI non-blocking reads

* `MPIIO_NB_WRITES`: Number of MPI non-blocking writes

* `MPIIO_SYNCS`: Number of MPI file syncs

* `MPIIO_MODE`: MPI mode that the file was last opened in

* `MPIIO_BYTES_READ`: Total number of bytes that were read from the file at MPI level

* `MPIIO_BYTES_WRITTEN`: Total number of bytes written to the file at MPI level

* `MPIIO_RW_SWITCHES`: Number of times that access toggled between read and write in consecutive MPI operations

* `MPIIO_SIZE_READ_0_100`: Number of read access between 0 and 100 bytes at MPIIO level

* `MPIIO_SIZE_READ_100_1K`: Number of read access between 100 bytes and 1KB at MPIIO level

* `MPIIO_SIZE_READ_1K_10K`: Number of read access between 1KB and 10KB at MPIIO level

* `MPIIO_SIZE_READ_10K_100K`: Number of read access between 10KB and 100KB at MPIIO level

* `MPIIO_SIZE_READ_100K_1M`: Number of read access between 100KB and 1MB at MPIIO level

* `MPIIO_SIZE_READ_1M_4M`: Number of read access between 1MB and 4MB at MPIIO level

* `MPIIO_SIZE_READ_4M_10M`: Number of read access between 4MB and 10MB at MPIIO level

* `MPIIO_SIZE_READ_10M_100M`: Number of read access between 10MB and 100MB at MPIIO level

* `MPIIO_SIZE_READ_100M_1G`: Number of read access between 100MB and 1GB at MPIIO level

* `MPIIO_SIZE_READ_1G_PLUS`: Number of read access greater than 1GB at MPIIO level

* `MPIIO_SIZE_WRITE_0_100`: Number of write access between 0 and 100 bytes at MPIIO level

* `MPIIO_SIZE_WRITE_100_1K`: Number of write access between 100 bytes and 1KB at MPIIO level

* `MPIIO_SIZE_WRITE_1K_10K`: Number of write access between 1KB and 10KB at MPIIO level

* `MPIIO_SIZE_WRITE_10K_100K`: Number of write access between 10KB and 100KB at MPIIO level

* `MPIIO_SIZE_WRITE_100K_1M`: Number of write access between 100KB and 1MB at MPIIO level

* `MPIIO_SIZE_WRITE_1M_4M`: Number of write access between 1MB and 4MB at MPIIO level

* `MPIIO_SIZE_WRITE_4M_10M`: Number of write access between 4MB and 10MB at MPIIO level

* `MPIIO_SIZE_WRITE_10M_100M`: Number of write access between 10MB and 100MB at MPIIO level

* `MPIIO_SIZE_WRITE_100M_1G`: Number of write access between 100MB and 1GB at MPIIO level

* `MPIIO_SIZE_WRITE_1G_PLUS`: Number of write access greater than 1GB at MPIIO level

* `MPIIO_F_OPEN_START_TIMESTAMP`: Timestamp that the first MPIIO file open operation began

* `MPIIO_F_READ_START_TIMESTAMP`: Timestamp that the first MPIIO file read operation began

* `MPIIO_F_WRITE_START_TIMESTAMP`: Timestamp that the first MPIIO file write operation began

* `MPIIO_F_CLOSE_START_TIMESTAMP`: Timestamp that the first MPIIO file close operation began

* `MPIIO_F_OPEN_END_TIMESTAMP`: Timestamp that the last MPIIO file open operation ended

* `MPIIO_F_READ_END_TIMESTAMP`: Timestamp that the last MPIIO file read operation ended

* `MPIIO_F_WRITE_END_TIMESTAMP`: Timestamp that the last MPIIO file write operation ended

* `MPIIO_F_CLOSE_END_TIMESTAMP`: Timestamp that the last MPIIO file close operation ended

* `MPIIO_F_READ_TIME`: Cumulative time spent reading at MPI level

* `MPIIO_F_WRITE_TIME`: Cumulative time spent write and sync at MPI level

* `MPIIO_F_META_TIME`: Cumulative time spent in open and close at MPI level

* `MPIIO_F_VARIANCE_RANK_TIME`: The population variance for MPI I/O time of all the ranks

* `MPIIO_F_VARIANCE_RANK_BYTES`: The population variance for bytes transferred of all the ranks at MPI level

* `STDIO_OPENS`: Number of stdio file open operations

* `STDIO_FDOPENS`: Number of stdio fdopen operations

* `STDIO_READS`: Number of stdio read operations

* `STDIO_WRITES`: Number of stdio write operations

* `STDIO_SEEKS`: Number of stdio seek operations

* `STDIO_STATS`: Number of stdio stats operations

* `STDIO_FLUSHES`: Number of stdio flush operations

* `STDIO_BYTES_READ`: Total number of bytes read from the file using stdio operations

* `STDIO_BYTES_WRITTEN`: Total number of bytes written to the file using stdio operations

* `STDIO_META_TIME`: Cumulative time spent in stdio open/close/seek operations

* `STDIO_READ_TIME`: Cumulative time spent in stdio read operations

* `STDIO_WRITE_TIM`: Cumulative time spent in stdio write operations

 

2. Science domain dataset

 

* `job_id`: Unique job identifier automatically generated by SLURM upon submission

* `date`: Date the job started to run as provided by SLURM

* `science_domain`: Science domain of the job based on the information provided upon project allocation

 

RESOURCES

 

Helpful resources in understanding and utilizing this dataset:

 

1. Data collection methodology, understanding of Darshan logs and counters

 

    - Darshan: Darshan HPC I/O Characterization Tool Documentation

 

2. General information about the NERSC Cori Supercomputer

 

    - CORI: General information about the Cori supercomputer

    - Darshan: Darshan I/O profiler documentation in Cori