Dataset Open Access

2019 EMDataResource Model Metrics Challenge Dataset

Lawson, Catherine L; Kryshtafovych, Andriy; Adams, Paul D; Afonine, Pavel V; Baker, Matthew L; Barad, Benjamin A; Bond, Paul; Burnley, Tom; Cao, Renzhi; Cheng, Jianlin; Chojnowski, Grzegorz; Cowtan, Kevin; Dill, Ken; DiMaio, Frank; Farrell, Daniel P; Fraser, James S; Herzik, Mark A Jr; Hoh, Soon Wen; Hou, Jie; Hung, Li-Wei; Igaev, Maxim; Joseph, Agnel P; Kihara, Daisuke; Kumar, Dilip; Mittal, Sumit; Monastryrskyy, Bohdan; Olek, Mateusz; Palmer, Colin M; Patwardhan, Ardan; Perez, Alberto; Pfab, Jonas; Pintilie, Greg D; Richardson, Jane S; Sarkar, Daipayan; Schäfer, Luisa U; Schmid, Michael F; Schröder, Gunnar F; Shekhar, Mrinal; Si, Dong; Singharoy, Abishek; Terashi, Genki; Terwilliger, Thomas C; Vaiana, Andrea; Wang, Liguo; Wankowicz, Stephanie A; Williams, Christopher J; Winn, Martyn; Wu, Tianqi; Yu, Xiaodi; Zhang, Kaiming; Berman, Helen M; Chiu, Wah

This is the full dataset of the 2019 Cryo-EM Map-based Model Metrics Challenge sponsored by EMDataResource (www.emdataresource.org, challenges.emdataresource.org, model-compare.emdataresource.org). The goals of this challenge were (1) to assess the quality of models that can be produced using current modeling software, (2) to check the reproducibility of modeling results from different software developers and users, and (3) compare the performance of current metrics used for evaluation of models. The focus was on near-atomic resolution maps with an innovative twist: three of four target maps formed a resolution series (1.8 to 3.1 Å) from the same specimen and imaging experiment. Tools developed in previous challenges were expanded for managing, visualizing and analyzing the 63 submitted coordinate models, and several new metrics were introduced.

File Descriptions:

  • 2019-EMDataResource-Challenge-web.pdf: Archive of News, Goals, Timeline, Targets, Modelling Instructions, Process, FAQ, Submission Instructions, Submission Summary Statistics source from the EMDR Challenges website
  • correlation-images.tar.gz: Pairwise correlation tables for selected metric scores from the EMDR Model Compare website
  • maps.tar.gz: The maps used for Fit-to-Map analyses in the Challenge
  • models.tar.gz: The 63 models submitted by the modelling teams
  • results.tar.gz: The output logs for all of the analysis methods
  • Scores.xlsx: Scores for each model and analysis method, compiled into spreadsheet format
  • targets.tar.gz: The reference models used in the analysis

Post submission correction to the web archive PDF document: The full list of EMDataResource members on the model committee is as follows: Cathy Lawson, Andriy Kryshtafovych, Greg Pintilie, Mike Schmid, Helen Berman, Wah Chiu.

EMDataResource is supported by the US National Institutes of Health (NIH)/National Institute of General Medical Science, R01GM079429. The following additional grants are acknowledged for participant support. JSR and CW: NIH/ R35GM131883. PDA, PVA, L-WH, JSR, TCT and CW: NIH/P01GM063210. AS: National Science Foundation (NSF)/MCB-1942763 (CAREER), NIH/R01GM095583. The Singharoy team used supercomputing resources of the OLCF at the Oak Ridge National Laboratory, which is supported by the Office of Science at DOE under Contract No. DE-AC05-00OR22725. DKihara: NIH/R01GM123055, NIH/R01GM133840, NSF/DMS1614777, NSF/CMMI1825941, NSF/MCB1925643, NSF/DBI2003635, Purdue Institute of Drug Discovery. JSF: NIH/R01GM123159. MI: Max Planck Society German Research Foundation/IG 109/1-1. ACV: Max Planck Society German Research Foundation /FOR-1805. DKumar: NIH/R37AI36040 and Welch Foundation/Q1279 (PI: BVV Prasad). DS: NSF/DBI2030381. TB, CMP, MW: Medical Research Council MR/N009614/1. APJ, MW: Wellcome Trust 208398/Z/17/Z. KC: Biotechnology and Biological Sciences Research Council BB/P000517/1. MW: Biotechnology and Biological Sciences Research Council BB/P000975/1.
Files (270.9 MB)
Name Size
2019-EMDataResource-Challenge-web.pdf
md5:6c38dac86fc27b0ce976b4fe46297f18
726.5 kB Download
correlation-images.tar.gz
md5:53b53791192cf5095a6fc26d3cf268f1
730.6 kB Download
maps.tar.gz
md5:17cad0e8d75c05310114425877384dec
187.5 MB Download
models.tar.gz
md5:54e75cf33147d4d30c1ba6705e0ae534
4.5 MB Download
results.tar.gz
md5:e0a8091dd3a1b527b38a7715a58c50e9
77.2 MB Download
Scores.xlsx
md5:714f60634c66c3b8ac0eda6ce0e5d1a1
45.3 kB Download
targets.tar.gz
md5:ffc0df64d3e54d18b4530b21399f34c2
201.0 kB Download
67
56
views
downloads
All versions This version
Views 6767
Downloads 5656
Data volume 307.2 MB307.2 MB
Unique views 6060
Unique downloads 5050

Share

Cite as