Full Raw Evidence and Pinned Inputs for Early Gradient Spectra Predict Useful Low-Rank Adaptation
Authors/Creators
Description
Full raw experiment releases, pinned GPT-2/WikiText-2 validation inputs, and
Stage4 source sidecar for the project "Early Gradient Spectra Predict Useful
Low-Rank Adaptation."
This data record contains:
- Stage4 raw release archive
- synthetic-transformer raw release archive
- real-LoRA raw release archive
- pinned real-model input archive for GPT-2/WikiText-2 validation
- Stage4 source sidecar at the recorded source commit
- manifests, file-size records, and SHA-256 checksums
The companion software/code release is archived separately.
The central claim is that activation-whitened early-gradient spectra predict
useful LoRA rank in a reduced-rank population model and controlled spiked
matrix simulations. Transformer allocation evidence is bounded: the
synthetic-transformer allocation test is null under its primary condition, and
the GPT-2/WikiText-2 result is restricted to the plan-locked c_attn/c_fc module
suite.
Keywords:
LoRA; low-rank adaptation; PEFT; transformer fine-tuning; early gradient spectra;
activation whitening; effective rank; useful rank; rank allocation; random matrix
theory; GPT-2; WikiText-2; reproducible machine learning
Files
Additional details
Related works
- Is supplement to
- Dataset: https://github.com/GoGoKo699/early-gradient-spectra-lora (Other)
Dates
- Available
-
2026-06-30