Positive-Control-Validated Closed-Loop Evaluation of Vision-Language-Action Models: A Null Result for Parameter-Efficient Fine-Tuning Under Domain Shift

Preprint of the paper "Positive-Control-Validated Closed-Loop Evaluation of Vision-Language-Action Models: A Null Result for Parameter-Efficient Fine-Tuning Under Domain Shift."

OpenVLA-7B was adapted with low-rank adaptation (LoRA) on a pick-and-place task (PickCube-v1, ManiSkill3) and evaluated closed-loop under three conditions (standard, lighting shift, texture shift) across 200/500/1000 demonstrations. Before any policy was judged, the evaluation harness was certified by a scripted positive control that achieved 100% success, so the null result is attributable to the policy rather than a non-functional evaluator. Offline metrics improved and then saturated, yet closed-loop success remained near zero.

The null is robust across three adaptation recipes (attention-only LoRA, all-linear LoRA, and a scoped full fine-tune): across three recipes × three scales × three conditions, only one of 27 cells is non-zero (1/25) and every domain-shift cell is 0/25.

Contents: the clean demonstration datasets, exact training and evaluation configurations, the seed registry, all training/evaluation code (including the positive-control oracle and the adaptation-recipe robustness experiment), and result summaries.