FKBench-EA: Benchmark, Code, and Results for "Factorial Evaluation of Evidence-Aware Foreign-Key Discovery"
Authors/Creators
Description
Reproducibility package for the paper "FKBench-EA: Factorial Evaluation of Evidence-Aware Foreign-Key Discovery".
FKBench-EA is a benchmark for directed, single-column foreign-key discovery with machine-checkable evidence pointers. Each source database is degraded deterministically (key constraints removed, identifiers transformed) and packaged in six evidence tracks that cross SQL availability with no diagram, a current ER diagram, or a deliberately stale ER diagram.
The archive (about 84 MB) contains:
- the benchmark constructor, the evaluator and its tests, and an offline verifier;
- adapters for BINDER, FAIDA, and HoPF, the pixel-based ER reader (CV-ER), and two diagnostic controls;
- the language-model runner with every recorded model response, so that its scores can be recomputed without any model call;
- 30 Spider and 5 BIRD base cases, the identifier-opacity and orphan-value cases, all predictions, score tables, and analysis outputs;
- the paper sources, generated figures, and TeX macros.
README.md gives the commands to rebuild the six evidence tracks, run the verifier, and recompute every reported number.
Licensing: the benchmark code is released under the Apache License 2.0 (LICENSE, NOTICE). The databases derived from Spider 1.0 and BIRD remain under CC BY-SA 4.0 (see LICENSES.md). The HoPF source is not redistributed because its repository contains no license file; the archive includes the adapter, build patches, and build instructions.
Files
edbt2027-fkbench-ea-artifact.zip
Additional details
Dates
- Issued
-
2026-09-28