Metrics from: Black-box language model explanation by context length probing
Authors/Creators
- 1. Zenith Team, LIRMM, Université de Montpellier, France
- 2. Zenith Team, LIRMM, Inria, France
Description
Token-level metrics from the paper Black-box language model explanation by context length probing. The metrics were computed on the UD_English_LinES development set using the preds_to_metrics script from the repository.
The archives were created using PyTorch 1.11.0 and can be loaded using torch.load. Each file contains a dictionary mapping metric names to PyTorch tensors. The first two dimensions of each tensor correspond to target token position (within the whole dataset) and context length, respectively.
Code for processing the metrics is included in the process_metrics notebook.
The metrics are provided for research purposes, in particular to enable reproducing results from the paper without having to recompute or store the model predictions.
Files
Files
(3.5 GB)
| Name | Size | Download all |
|---|---|---|
|
md5:dc2544da39e2d7bfa1d16be14a18600b
|
1.2 GB | Download |
|
md5:4a104d5c2b011e85ab9bb2f96b5878e8
|
1.2 GB | Download |
|
md5:f29f48c3c328d615310ed98358e2e4e6
|
1.2 GB | Download |
Additional details
Related works
- Is supplement to
- Preprint: arXiv:2212.14815 (arXiv)
- Software: https://github.com/cifkao/context-probing/ (URL)
Funding
- Agence Nationale de la Recherche
- NUMEV - Digital and Hardware Solutions and Modeling for the Environement and Life Sciences ANR-10-LABX-0020
- Agence Nationale de la Recherche
- MUSE - MUSE ANR-16-IDEX-0006