Published December 2, 2024 | Version v1
Preprint Open

Make Data Count: Driving metrics for the meaningful evaluation of data

Description

In order to successfully evaluate the impact and reach of data, there is a need for metrics that treat data as primary research outputs. Make Data Count is an initiative that works to develop metrics that allow the community to understand how data are used and how they advance research and policy. Make Data Count collaborates with groups across the research data ecosystem to create and implement workflows for recording information about how data sets are accessed and used; those measures provide the necessary foundation for further contextualization leading into meaningful data metrics. Spanning across disciplines, sectors, and geographies, Make Data Count fosters innovation in data evaluation. We develop standards to capture comparable and consistent counts of data citations, downloads and views, and scalable infrastructure to collect, aggregate and share usage measures, openly and through integration with workflows to index research information. We support projects that build the research evidence needed to identify metadata gaps and to benchmark data metrics to respond to the needs of diverse users. And importantly, we believe that data must be incorporated into research evaluation processes such as funding applications or tenure and promotion processes so that researchers are rewarded for the data they share and their impact. We invite the community to contribute to the development of meaningful data metrics so that, through collaboration, we can create a system where data are regularly assessed, evaluated, and rewarded.

Files

Make Data Count_ Driving metrics for the meaningful evaluation of data.pdf

Files (161.2 kB)

Additional details

References

  • Lowenberg D, Chodacki J, Fenner M, Kemp J, Jones MB. Open Data Metrics: Lighting the Fire [Internet]. Zenodo; 2019 [cited 2023 Sep 22]. Available from: https://zenodo.org/record/3525349
  • Kratz JE, Strasser C. Making data count. Sci Data. 2015 Aug 4;2(1):150039.
  • Fenner M, Lowenberg D, Jones M, Needham P, Vieglais D, Abrams S, et al. Code of practice for research data usage metrics release 1 [Internet]. PeerJ Inc.; 2018 Feb [cited 2024 Nov 11]. Report No.: e26505v1. Available from: https://peerj.com/preprints/26505
  • Data Citation and Reuse [Internet]. Scholarly Communications Lab | ScholCommLab. [cited 2024 Nov 11]. Available from: https://www.scholcommlab.ca/research/data-citation/
  • White House Office of Science and Technology Policy (OSTP). Desirable Characteristics of Data Repositories for Federally Funded Research [Internet]. Executive Office of the President of the United States; 2022 May [cited 2024 Nov 11]. Available from: https://repository.si.edu/handle/10088/113528
  • EMBL. Research Assessment at EMBL [Internet]. [cited 2024 Nov 11]. Available from: https://embl.org/about/research-assessment/
  • Wilkinson MD, Dumontier M, Aalbersberg IjJ, Appleton G, Axton M, Baak A, et al. The FAIR Guiding Principles for scientific data management and stewardship. Sci Data. 2016 Mar 15;3(1):160018.
  • Puebla I, Lowenberg D. Building Trust: Data Metrics as a Focal Point for Responsible Data Stewardship. Harv Data Sci Rev [Internet]. 2024 Apr 2 [cited 2024 Nov 11];(Special Issue 4). Available from: https://hdsr.mitpress.mit.edu/pub/l3g0j3bk/release/1
  • Burton A, Fenner M, Haak W, Manghi P. Scholix Metadata Schema for Exchange of Scholarly Communication Links. 2017 Nov 21 [cited 2024 Nov 11]; Available from: https://zenodo.org/records/1120265
  • Fenner M, Crosas M, Grethe JS, Kennedy D, Hermjakob H, Rocca-Serra P, et al. A data citation roadmap for scholarly data repositories. Sci Data. 2019 Apr 10;6(1):28.
  • Cousijn H, Kenall A, Ganley E, Harrison M, Kernohan D, Lemberger T, et al. A data citation roadmap for scientific publishers. Sci Data. 2018 Nov 20;5(1):180259.
  • DataCite, Crossref, STM, STM D Crossref and. DataCite. 2023 [cited 2024 Nov 11]. Joint Statement on Research Data. Available from: https://datacite.org/blog/joint-statement-on-research-data/
  • Gregory K, Ninkov A, Ripp C, Roblin E, Peters I, Haustein S. Tracing data: A survey investigating disciplinary differences in data citation. Quant Sci Stud. 2023 Sep 12;1–51.
  • Ninkov A, Gregory K, Peters I, Haustein S. Datasets on DataCite - an Initial Bibliometric Investigation [Internet]. Zenodo; 2021 Apr 30 [cited 2024 Nov 11]. Available from: https://zenodo.org/records/4730857
  • DataCite, Make Data Count. Data Citation Corpus Data File (v2.0) [Data set]. DataCite. https://doi.org/10.5281/zenodo.13376773
  • Ninkov AB, Gregory K, Jambor M, Garza K, Strecker D, Schabinger R, et al. Mapping Metadata - Improving Dataset Discipline Classification [Internet]. Zenodo; 2022 Sep 7 [cited 2024 Nov 11]. Available from: https://zenodo.org/records/6948238
  • Alperin JP, Schimanski LA, La M, Niles MT, McKiernan EC. The value of data and other non-traditional scholarly outputs in academic review, promotion, and tenure in Canada and the United States. Open Handb Linguist Data Manag [Internet]. [cited 2024 Mar 28]; Available from: https://hcommons.org/deposits/item/hc:32909/
  • Puebla I, Ascoli GA, Blume J, Chodacki J, Finnell J, Kennedy DN, et al. Ten simple rules for recognizing data and software contributions in hiring, promotion, and tenure. PLOS Comput Biol. 2024 Aug 8;20(8):e1012296.
  • Evaluation, context and getting practical on data metrics: the Make Data Count Summit 2024 - Make Data Count. [Internet]. 2024 [cited 2024 Nov 11]. Available from: https://makedatacount.org/read-our-blog/evaluation-context-and-getting-practical-on-data-metrics-the-make-data-count-summit-2024/