Code "Reinterpretation Metadata Analysis" for producing Fig. 3 of the ESPPU 2025 contribution "Reinterpretation and preservation of data and analyses in HEP"

Description of the project

This project, which is also available on GitLab, allows to plot the number of publications as well as citations per publication on Inspire HEP. They are further sub-categorised by whether they have a HEPData entry and whether reinterpretation material in an appropriate tool (CheckMATE, CMS Combine, MadAnalysis, Nuisance, pyHF-HistFactory, Rivet, or SModelS) is available.

For Fig. 3, the following settings were used:

• Only ATLAS and CMS publications with centre-of-mass energy 13 TeV or 13.6 TeV are considered.
  • Only peer-reviewed and published analyses are taken into account. This excludes, among others, performance, trigger, and software publications.
  • The publication date (taken as the date of publication on arXiv) must lie between 2016 and 6 months prior to running the code (i.e. between January 2016 and September 2024).
• As tools, only those giving a reimplementation of the analysis logic (CheckMATE, MadAnalysis, Rivet, and SModelS) are taken into account.
• Publications are labelled as "search" if the term "search" appears in either title or abstract of the publication.

Description of folders and files contained in reinterpretation_metadata_analysis.zip

• *.py: the python executables run to produce the plots
• *.csv: the intermediate database files
• plots/: directory of the produced plots

Fig. 3 corresponds to plots/citation_count_without_self_citations_by_year_collATLAS,CMS_cms10+TeV_after16-01_before24-09_analysistypesAll_toolsImplementations.png and plots/citation_count_and_publications_by_year_collATLAS,CMS_cms10+TeV_after16-01_before24-09_analysistypesSearch_toolsImplementations.png.

Instructions to reproduce Fig. 3

# prepare environment
python3 -m venv .venv
source ./.venv/bin/activate
pip install hepdata-cli matplotlib pandas tqdm

# get information from inspire
python3 get_inspire.py

# amend with information from HEPData
python3 get_hepdata.py

# plot
python3 plot.py --cms 10+TeV -t implementations -a 6 --min_date 2016-01-01 --no_mean 
python3 plot.py --cms 10+TeV -t implementations -a 6 --min_date 2016-01-01 --no_mean --analysis_types search