Records and scripts: provenance audit and training-catalog overlap for BioCLIP-family evaluations (A False Floor)
Description
Records and scripts behind the provenance work for A False Floor (preprint 10.5281/zenodo.23519300; the revised manuscript was submitted to PLOS ONE on 22 October 2026): a field audit of 24 codable papers citing BioCLIP or BioCLIP 2, from an OpenAlex frame of 146 works, a direct measurement of overlap between the study's scored evaluation images and the published TreeOfLife-200M training catalogs, and a post hoc out-of-fold count on the pair the original floor was measured on.
Version 4 (22 October 2026). Adds the license make-up of the scored hard-pair images (c5_license_by_dataset.py and its output: all 2,775 recovered scored images belong to the GBIF dataset iNaturalist Research-grade Observations, key 50c9509d-22c7-4a22-a47d-8c48425ef4a7, licensed CC BY-NC 4.0, read 22 October 2026; the licenses of the 825 unrecovered images were not resolved; the record also gives the eight hard-pair species' GBIF taxonKeys), which is the basis for sharing the archived feature caches for noncommercial research use; the saved known-answer output of the out-of-fold count (ten-split DINOv2 per-seed counts [0,0,0,1,0,0,0,0,1,0], matching the macOS 27 record); the author's later decisions appended to the decision record; the log of a note added to the OSF archive; and the author's clarification that Qwen3.8-Max was used as a model, not as an agent. The digest log now uses paths relative to the deposit root throughout. Everything else is carried from version 3 byte for byte.
Version 3 (21 October 2026). The overlap figures of version 2 are unchanged. Version 3 adds: the author's decision record keeping the BioCLIP 2/2.5 exclusion on every pair, as the overlap pre-commitment's consequence required (on source-distribution and label-supervision provenance, with no claim of image overlap); a pre-committed, post hoc out-of-fold count on field mushroom against Destroying Angel (DINOv2 passes 0 of 500 held-out Destroying Angels, one-sided 95% bound 0.60%; I-JEPA 14 of 500), with an appended erratum; the catalog facts file (bytes, SHA-256 and maximum GBIF key of both catalog revisions); a later check in which a separate language model (Qwen3.8-Max), run by the author after reading a draft, coded the 13 positive and unknown audit papers (10 of 13 agree with the final codes; the final codes stand); the clean-room record of a re-run from version 2; and the updated log of the preprint's correction notice. The digest log now uses relative paths. Everything else is carried from version 2 byte for byte.
The catalog extracts are not redistributed, because the GBIF occurrence snapshot they derive from is CC BY-NC 4.0; README.md gives the script and the catalog SHA-256 values that regenerate them. The records were written in dated internal files and were not publicly deposited in advance, so they are called pre-specified, not pre-registered. Records and documents are CC BY 4.0; scripts are MIT (see LICENSE.txt).
Post-submission update (22 October 2026): the preprint version history is available at https://doi.org/10.5281/zenodo.23519299. This metadata update leaves all four published version-4 files unchanged. Additional retained inputs for the stored-hash matching replay are supplied separately as records/provenance_inputs_2026-10-22.zip (OSF file version 2) in the archive described in the manuscript. That package contains no photographs, does not repeat photograph hashing or network aspect-ratio checks, and retains its source-specific terms and unresolved rights disclosures.
Files
LICENSE.txt
Additional details
Related works
- References
- Preprint: 10.5281/zenodo.23519300 (DOI)
- Dataset: 10.15468/dl.bfv433 (DOI)
- Dataset: https://huggingface.co/datasets/imageomics/TreeOfLife-200M (URL)