Binary Grounded VEX Clearance Fails Three Times: Upstream of Evidence, on Internal Symbols, and on Version Applicability

Regulatory regimes including the EU Cyber Resilience Act make vulnerability
reachability load bearing: a flaw in a component that cannot be reached in a
shipped product does not start the mandatory reporting clock. This creates
demand for automated binary grounded VEX clearance, in which a tool inspects
the artifact that ships and decides whether the vulnerable code is present.

We built such a pipeline and measured where it fails. Across 55 scanned
directories covering 53 catalogued shipped artifacts, including 12 firmware
images, and 15,641 scanner reported CVEs, we attribute every decision to the
pipeline stage at which it terminated. Two kill tests, written into the
repository before any code existed and never adjusted, resolve against the
approach.

First, the pipeline terminates upstream of evidence. 91.93 percent of
attributed findings stop at identity resolution and a further 7.67 percent at
manifest lookup, before any binary is opened. Zero findings terminate at a
symbol table stage, despite 83 percent of corpus binaries being stripped. The
coverage kill test is therefore recorded as not evaluable rather than failed:
the instrument never reached the question it was built to answer. Termination
is not uniform across packaging ecosystems, with RPM based artifacts failing
earlier and more completely than Debian based ones.

Second, where the pipeline did decide, its evidence could not support its
claim. All 35 clearances justified vulnerable_code_not_present by absence of
the vulnerable function from the dynamic symbol table. Those functions are
internal to OpenSSL and never appear in any export map, so absence was
guaranteed a priori and the verdict was unfalsifiable rather than merely wrong.
Hand verification of 100 statements found 20 of 20 clearances false and 80 of
80 refusals correct.

Third, and found only after the second result was written up, the pipeline
never computes whether a CVE applies to the version on the artifact. Two of
four manifest CVEs do not apply to the versions where they were decided. This
reclassifies 7 of the 35 clearances as correct for a reason never computed,
leaving 28 unsound, and corrects 4 of the 20 hand labels, leaving 16. The
soundness test still fails: its bar was one. The guard added in response to the
second result closed symbol observability and left version applicability
untouched.

We publish the corpus, the hand labelled ground truth, and all figures as
re derivable artifacts. The binding constraint on binary grounded VEX clearance
is CVE to component identity, not evidence availability, and symbol table
absence is sound evidence only for exported functions.

The accompanying dataset, containing the corpus catalogue with hashes, every
decision the pipeline made, the evidence bundles, the hand labelled ground
truth and four figure provenance audits, is archived separately at
https://doi.org/10.5281/zenodo.22093232