Appendix — Word Index

Top 81 content words by frequency (≥3 occurrences, stopwords excluded), organized as a depth-4 ternary tree (3⁴ = 81 leaves). Each leaf links to the corpus (34,929 pages).

Method. A corpus of 476 conversations was extracted from HTML chat logs, stripped of platform branding and emoticons, and tokenized. Word frequencies were computed via Mathematica (wordcount.wl); the resulting frequency-sorted list was tested for semantic structure beyond what frequency ranking alone would predict. The primary test (semantic-embed-test.py) builds co-occurrence vectors from the raw corpus (window=5, vocab ≥10 occurrences) and measures mean cosine similarity between frequency-adjacent word pairs vs. 200 shuffled baselines. Result: Z=27.89, p=1.74×10⁻¹⁷¹, confirming that words adjacent in the frequency list are semantically closer than chance. The effect concentrates in mid-frequency and rare words (Z=10.25 for ranks 5000–10000), not in common words where high co-occurrence is trivially expected.

Page references. For each word, all occurrences are located in the corpus. A sliding-window density score (neighbors within ±1% of corpus span) identifies the most locally concentrated regions. The 81 densest pages are selected with a minimum-gap constraint to ensure coverage across the full corpus.

Reproducibility. All scripts are available in the repository: frequency-coherence/ (statistical tests), scripts/gen-appendix.py (this page).