[2019 QSM Reconstruction Challenge] Metrics and Submission Information

This repository contains information about submitted solutions and resulting analysis metrics of the 2019 Quantitative Susceptibility Mapping Reconstruction Challenge. The original susceptibility maps submitted for participation in the challenge are available here and here.

The package contains seven Comma-Separated Values (CSV) files and two PDF files:

• master_stage1_anonymized.csv: Results of stage 1 of the challenge at the time of presentation at the workshop (fully-blinded);
• master_stage2_snr1_anonymized.csv: Results of stage 2 of the challenge using the high noise dataset at the time of presentation at the workshop (fully-blinded);
• master_stage2_snr2_anonymized.csv: Results of stage 2 of the challenge using the low noise dataset at the time of presentation at the workshop (fully-blinded);
• submission_form_stage1.pdf: PDF export of the online form used in stage 1;
• submission_form_stage2.pdf: PDF export of the online form used in stage 2.

For the manuscript, we analyzed these CSV files with scripts reported here.

Each csv file contains metrics for all submitted solutions along with detailed information about the algorithm used, provided by the participant at the time of submission. The very first record in each file is a header containing a list of field names:

• normalized rmse: Whole-brain root-mean-squared error relative to ground truth;
• rmse_detrend_tissue: Root-mean-squared error relative to ground truth (after detrending) in grey and white matter mask;
• rmse_detrend_blood: Root-mean-squared error relative to ground truth (after detrending) using a one-pixel dilated vein mask;
• rmse_detrend_DGM: Root-mean-squared error relative to ground truth (after detrending) in a deep gray matter mask (substantia nigra & subthalamic nucleus, red nucleus, dentate nucleus, putamen, globus pallidus and caudate);
• DeviationFromLinearSlope: Absolute difference between the slope of the average value of the six deep gray matter regions vs. the prescribed mean value and 1.0;
• CalcStreak: Estimation of the impact of the streaking artifact in a region of interest surrounding the calcification through the standard deviation of the difference map between reconstruction and the ground truth;
• DeviationFromCalcMoment: Absolute deviation from the volumetric susceptibility moment of the reconstructed calcification, compared to the ground truth (computed at in the high-resolution model);
• Submission Identifier: Self-chosen unique identifier of the submission;
• Submission Identifier of the corresponding Stage 1 submission: This is the Submission Identifier of the solution submitted to Stage 2 that was calculated with a similar algorithm in Stage 1;
• Changes with respect to Stage 1 submission: Self-reported information about modifications made to the algorithm for Stage 2;
• Number of submissions in Stage 2: The number of solutions that were submitted to Stage 2 with a similar algorithm;
• Sim1/Sim2: Filename of the submitted solutions for Stage 1;
• File name of the zip-file you are going to upload: Filename of the file uploaded to Stage 2;
• Full name of the algorithm: Self-reported full name of the algorithm used;
• Preferred Acronym: Self-reported acronym of the algorithm used;
• Algorithm-type: Self-reported type of algorithm used;
• Does your algorithm incorporate information derived from magnitude images?: Self-reported Yes/No;
• Regularization terms: Self-reported types of regularization terms involved;
• Did your algorithm use the provided frequency map or the four individual echo phase images?: Self-reported information about involved magnitude information;
• Publication-ready description of the reconstruction technique: Self-reported description of the algorithm;
• Publications that describe the algorithm: Self-reported literature reference;
• Algorithm publicly available?: Self-reported public availability of the algorithm;
• If your algorithm is not yet publicly available, would you be willing to make it available at the end of the challenge?: Self-reported willingness to share the algorithm code with the public;
• Specific information about this solution: Self-reported detailed information about the solution;
• Herewith, I permit the QSM Challenge committee to publish my uploaded files (calculated maps) after the completion of the challenge: Self reported agreement with publication of submitted solution;
• Ground truth was not explicitly or implicitly incorporated into your algorithm or solution: Self-reported confirmation that the ground truth was not incorporated in the solution.