PER-BENCHMARK SPEARMAN CORRELATIONS
======================================================================================
measure                                               20news              agnews
--------------------------------------------------------------------------------------
local weakness proxy                    +0.770282  p=1.54e-10+0.636307  p=1.17e-06
validation accuracy                     +0.950208  p=6.25e-25+0.919041  p=3.19e-20
negative train loss                       -0.109991  p=0.457  +0.108666  p=0.462
negative validation loss                   -0.075520  p=0.61  +0.158493  p=0.282
negative SAM loss r=1e-2                  -0.109991  p=0.457  +0.108666  p=0.462
negative SAM loss r=1e-3                  -0.109991  p=0.457  +0.108666  p=0.462
negative perturbation loss s=1e-2         -0.112220  p=0.448   +0.102044  p=0.49
negative perturbation loss s=1e-3         -0.109991  p=0.457  +0.108666  p=0.462
PAC-Bayes noise score                     -0.113688  p=0.442  +0.105626  p=0.475
negative L2 weight norm                   +0.112818  p=0.445  -0.062855  p=0.671
negative path norm                        +0.112818  p=0.445  -0.062855  p=0.671
mean child margin                         -0.111459  p=0.451  +0.161587  p=0.273
minimum child margin                      -0.155444  p=0.291  +0.072679  p=0.623
negative child logit entropy               -0.129727  p=0.38  +0.083046  p=0.575
parameter count                            -0.097549  p=0.51  -0.092642  p=0.531

DESCRIPTIVE POOLED CORRELATIONS
======================================================================================
local weakness proxy                    +0.704245  p=1.2e-15
validation accuracy                     +0.925725  p=1.79e-41
negative train loss                     -0.013514  p=0.896
negative validation loss                +0.006757  p=0.948
negative SAM loss r=1e-2                -0.013514  p=0.896
negative SAM loss r=1e-3                -0.013514  p=0.896
negative perturbation loss s=1e-2       -0.016865  p=0.87
negative perturbation loss s=1e-3       -0.014036  p=0.892
PAC-Bayes noise score                   -0.016858  p=0.871
negative L2 weight norm                 +0.042969  p=0.678
negative path norm                      +0.043525  p=0.674
mean child margin                       -0.011390  p=0.912
minimum child margin                    -0.072194  p=0.485
negative child logit entropy            -0.043146  p=0.676
parameter count                         -0.087970  p=0.394

MODEL CONFIGURATION RANGES
======================================================================================
20news: n=48, child_n=32, classes=2, sparse_dim=20000, feature_dim=128, widths=[256, 384, 576], lr=[0.001, 0.002, 0.003], batch=[16, 32, 64], weight_decay=[0.0, 1e-05, 0.0001]
agnews: n=48, child_n=96, classes=4, sparse_dim=12000, feature_dim=64, widths=[256, 288], lr=[0.0005, 0.001, 0.0015], batch=[32, 64], weight_decay=[0.0, 1e-05, 0.0001]

SPEARMAN CORRELATIONS WITH 95% PERCENTILE BOOTSTRAP INTERVALS
====================================================================================================
measure                                                        20news                       agnews
----------------------------------------------------------------------------------------------------
local weakness proxy                          +0.770 [+0.630, +0.857]      +0.636 [+0.448, +0.765]
validation accuracy                           +0.950 [+0.887, +0.980]      +0.919 [+0.854, +0.948]
negative train loss                           -0.110 [-0.392, +0.184]      +0.109 [-0.194, +0.388]
negative validation loss                      -0.076 [-0.360, +0.218]      +0.158 [-0.147, +0.443]
negative SAM loss r=1e-2                      -0.110 [-0.385, +0.188]      +0.109 [-0.195, +0.385]
negative SAM loss r=1e-3                      -0.110 [-0.387, +0.188]      +0.109 [-0.195, +0.385]
negative perturbation loss s=1e-2             -0.112 [-0.395, +0.182]      +0.102 [-0.198, +0.381]
negative perturbation loss s=1e-3             -0.110 [-0.388, +0.189]      +0.109 [-0.194, +0.383]
PAC-Bayes noise score                         -0.114 [-0.398, +0.183]      +0.106 [-0.197, +0.385]
negative L2 weight norm                       +0.113 [-0.163, +0.385]      -0.063 [-0.360, +0.260]
negative path norm                            +0.113 [-0.162, +0.379]      -0.063 [-0.357, +0.258]
mean child margin                             -0.111 [-0.388, +0.185]      +0.162 [-0.139, +0.435]
minimum child margin                          -0.155 [-0.451, +0.170]      +0.073 [-0.212, +0.352]
negative child logit entropy                  -0.130 [-0.404, +0.165]      +0.083 [-0.224, +0.364]
parameter count                               -0.098 [-0.371, +0.184]      -0.093 [-0.364, +0.183]

POINT-ESTIMATE SIGN COMPARISON
====================================================================================================
changed sign (12): negative train loss, negative validation loss, negative SAM loss r=1e-2, negative SAM loss r=1e-3, negative perturbation loss s=1e-2, negative perturbation loss s=1e-3, PAC-Bayes noise score, negative L2 weight norm, negative path norm, mean child margin, minimum child margin, negative child logit entropy
held sign (3): local weakness proxy, validation accuracy, parameter count

EXPLORATORY PARTIAL SPEARMAN ASSOCIATION
====================================================================================================
20news: rho(proxy, audit accuracy | validation accuracy)=+0.103485, p=0.483957
agnews: rho(proxy, audit accuracy | validation accuracy)=+0.470521, p=0.000738636
