Published May 29, 2026
| Version v1
Preprint
Open
Per-Channel Quantile Fullgrid Probing for Small Language Model Observation
Description
We describe a dense activation probing protocol for small to mid-sized language models that operates within a fixed cell budget while maintaining high observation coverage. The protocol combines per-channel quantile binning, an expanded calibration probe corpus (P=1200), and an additive shard merge step that allows heterogeneous compute resources to contribute. Across thirteen runs spanning seven model families and parameter counts from 124M to 7.6B, the protocol reaches at least 90.6 percent coverage on every run, with a median of 94.95 percent and a maximum of 97.05 percent. Per-channel binning combined with a larger probe budget was associated with higher coverage compared to earlier pooled-bin, smaller-corpus configurations on the same models, but the isolated contribution of each factor remains untested in this draft. A controlled 2x2 ablation is the recommended next experiment. The pipeline is organized for reproducibility and can be released alongside the run artifacts. It runs on commodity hardware.
Notes
Files
paper-3-perchannel-quantile.pdf
Additional details
References
- Chen, Ho Yiing (2026). Coverage Failure as a Diagnostic Signal in Neural Model Probing. Zenodo. DOI: 10.5281/zenodo.20448669
- Chen, Ho Yiing (2026). Mercury: An addressable million-cell sensor grid for browser-accessible LLM observability on consumer hardware. Zenodo. DOI: 10.5281/zenodo.20313154
- Chen, Ho Yiing (2026). Mercury-Viewer: A self-contained browser-based interactive 3D visualization of LLM internal sensor maps. Zenodo. DOI: 10.5281/zenodo.20313150
- Chen, Ho Yiing (2026). Mercury MCP v0.1.0 — Cross-Architecture LLM Internal Observation Database for 23 Architectures. Zenodo. DOI: 10.5281/zenodo.20352085
- Chen, Ho Yiing (2026). Cross-Architecture Per-Layer Residual Stream Alignment via Tier-B Fingerprints. Zenodo. DOI: 10.5281/zenodo.20366010
- Chen, Ho Yiing (2026). Tier-A Vocab-Hash Artifact in Cross-Architecture Hot-Dim Studies. Zenodo. DOI: 10.5281/zenodo.20366134
- Chen, Ho Yiing (2026). Internal Specialization Scales with Model Size: Quantitative Evidence from qwen2.5 3B vs 7B. Zenodo. DOI: 10.5281/zenodo.20325676
- Chen, Ho Yiing (2026). Cross-Model Dimensional Preservation in qwen2.5: 49 Hot Coordinates Survive Across Two Independent Fine-Tunes. Zenodo. DOI: 10.5281/zenodo.20346626
- Chen, Ho Yiing (2026). A Reproducible Within-Family SLERP Merge of Llama-3.1-8B-Instruct and DeepSeek-R1-Distill-Llama-8B. Zenodo. DOI: 10.5281/zenodo.20404139
- Carter, S., Armstrong, Z., Schubert, L., Johnson, I., and Olah, C. (2019). Activation Atlas. Distill. DOI: 10.23915/distill.00015
- Goddard, C., Siriwardhana, S., Ehghaghi, M., Meyers, L., Karpukhin, V., Benedict, B., McQuade, M., and Solawetz, J. (2024). Arcee's MergeKit: A Toolkit for Merging Large Language Models. arXiv preprint. DOI: 10.48550/arXiv.2403.13257