Published May 29, 2026 | Version v1

Per-Channel Quantile Fullgrid Probing for Small Language Model Observation

Authors/Creators

  • 1. Independent Researcher

Description

We describe a dense activation probing protocol for small to mid-sized language models that operates within a fixed cell budget while maintaining high observation coverage. The protocol combines per-channel quantile binning, an expanded calibration probe corpus (P=1200), and an additive shard merge step that allows heterogeneous compute resources to contribute. Across thirteen runs spanning seven model families and parameter counts from 124M to 7.6B, the protocol reaches at least 90.6 percent coverage on every run, with a median of 94.95 percent and a maximum of 97.05 percent. Per-channel binning combined with a larger probe budget was associated with higher coverage compared to earlier pooled-bin, smaller-corpus configurations on the same models, but the isolated contribution of each factor remains untested in this draft. A controlled 2x2 ablation is the recommended next experiment. The pipeline is organized for reproducibility and can be released alongside the run artifacts. It runs on commodity hardware.

Notes

© Copyright 2026 Chen, Ho Yiing (ORCID: 0009-0006-6816-9891), Independent researcher, charenix.com. All rights reserved by the author. This work is licensed under the Creative Commons Attribution 4.0 International License.

Files

paper-3-perchannel-quantile.pdf

Files (90.7 kB)

Name Size
md5:809236e7361f6a7c36e20c2fec6d8fcc
69.1 kB Preview Download
md5:55be6aba9a372d6b017364f60dbcc6e9
21.6 kB Download

Additional details

References

  • Chen, Ho Yiing (2026). Coverage Failure as a Diagnostic Signal in Neural Model Probing. Zenodo. DOI: 10.5281/zenodo.20448669
  • Chen, Ho Yiing (2026). Mercury: An addressable million-cell sensor grid for browser-accessible LLM observability on consumer hardware. Zenodo. DOI: 10.5281/zenodo.20313154
  • Chen, Ho Yiing (2026). Mercury-Viewer: A self-contained browser-based interactive 3D visualization of LLM internal sensor maps. Zenodo. DOI: 10.5281/zenodo.20313150
  • Chen, Ho Yiing (2026). Mercury MCP v0.1.0 — Cross-Architecture LLM Internal Observation Database for 23 Architectures. Zenodo. DOI: 10.5281/zenodo.20352085
  • Chen, Ho Yiing (2026). Cross-Architecture Per-Layer Residual Stream Alignment via Tier-B Fingerprints. Zenodo. DOI: 10.5281/zenodo.20366010
  • Chen, Ho Yiing (2026). Tier-A Vocab-Hash Artifact in Cross-Architecture Hot-Dim Studies. Zenodo. DOI: 10.5281/zenodo.20366134
  • Chen, Ho Yiing (2026). Internal Specialization Scales with Model Size: Quantitative Evidence from qwen2.5 3B vs 7B. Zenodo. DOI: 10.5281/zenodo.20325676
  • Chen, Ho Yiing (2026). Cross-Model Dimensional Preservation in qwen2.5: 49 Hot Coordinates Survive Across Two Independent Fine-Tunes. Zenodo. DOI: 10.5281/zenodo.20346626
  • Chen, Ho Yiing (2026). A Reproducible Within-Family SLERP Merge of Llama-3.1-8B-Instruct and DeepSeek-R1-Distill-Llama-8B. Zenodo. DOI: 10.5281/zenodo.20404139
  • Carter, S., Armstrong, Z., Schubert, L., Johnson, I., and Olah, C. (2019). Activation Atlas. Distill. DOI: 10.23915/distill.00015
  • Goddard, C., Siriwardhana, S., Ehghaghi, M., Meyers, L., Karpukhin, V., Benedict, B., McQuade, M., and Solawetz, J. (2024). Arcee's MergeKit: A Toolkit for Merging Large Language Models. arXiv preprint. DOI: 10.48550/arXiv.2403.13257