Published November 2, 2026 | Version v2

FKBench-EA: Benchmark, Code, and Results for "Factorial Evaluation of Evidence-Aware Foreign-Key Discovery"

Authors/Creators

Description

Reproducibility package for the paper "FKBench-EA: Factorial Evaluation of Evidence-Aware Foreign-Key Discovery".

FKBench-EA is a benchmark for directed, single-column foreign-key discovery with machine-checkable evidence pointers. Each source database is degraded deterministically (key constraints removed, identifiers transformed) and packaged in six evidence tracks that cross SQL availability with no diagram, a current ER diagram, or a deliberately stale ER diagram.

The archive (about 84 MB) contains:
- the benchmark constructor, the evaluator and its tests, and an offline verifier;
- adapters for BINDER, FAIDA, and HoPF, the pixel-based ER reader (CV-ER), and two diagnostic controls;
- the language-model runner with every recorded model response, so that its scores can be recomputed without any model call;
- 30 Spider and 5 BIRD base cases, the identifier-opacity and orphan-value cases, all predictions, score tables, and analysis outputs;
- the paper sources, generated figures, and TeX macros.

README.md gives the commands to rebuild the six evidence tracks, run the verifier, and recompute every reported number.

Licensing: the benchmark code is released under the Apache License 2.0 (LICENSE, NOTICE). The databases derived from Spider 1.0 and BIRD remain under CC BY-SA 4.0 (see LICENSES.md). The HoPF source is not redistributed because its repository contains no license file; the archive includes the adapter, build patches, and build instructions.

Files

edbt2027-fkbench-ea-artifact.zip

Files (83.9 MB)

Name Size
md5:784ac7ee297cb5de28f519e97dced739
83.9 MB Preview Download

Additional details

Dates

Issued
2026-09-28

Software

Programming language
Python
Development Status
Active