Frozen-backbone JEPA-style probes on HAM10000: preliminary results from a nine-experiment ablation, with an unpartitioned contamination caveat

Background & Methodology

Self-supervised representations promise robustness to nuisance variation; whether that robustness transfers across novel nuisance families is rarely tested. We probe this on a leakage-controlled HAM10000 longitudinal proxy with three disjoint synthetic nuisance families --- partitioned along the deterministic-augmentation axis. We report a preliminary nine-experiment arc, training a JEPA-style latent predictor on stable pairs from two of three families and evaluating on the third unseen family.

Results: Baseline & General Models

Across four frozen-natural-image configurations spanning two pretraining paradigms within the ViT-B class (DINOv2 ViT-B/14, OpenAI CLIP ViT-B/16), two predictor scaffolds (linear with identity warm-start, 2-layer identity-residual MLP), and two optimisers (SGD, Adam), test AUROC on the held-out family lands in 0.25 - 0.29. This is below random and below the strongest oriented baseline (pixel L2 at 0.580).

A frozen general-medical backbone (BiomedCLIP, PMC-15M scientific-article figures, with no documented raw HAM10000/ISIC archive ingestion) lifts test AUROC by only +0.04 to 0.329 +/- 0.012 across 5 seeds.

Results: Domain-Specific Models

A frozen dermoscopy-pretrained backbone (DermLIP, Derm1M) lifts test AUROC to 0.944 +/- 0.003 across 5 seeds. This is +0.363 above the strongest baseline and the only above-random result on the held-out third family across the six runs that evaluate on it (EXP-004 through EXP-008).

Domain Swing Analysis

Holding architecture (ViT-B/16) and predictor (linear) constant, varying only the pretraining-data domain produces a +0.66 end-to-end AUROC swing (web to dermoscopy) that is concentrated in the dermoscopy-specific step (+0.62 of +0.66, with the residual +0.04 at the web to general-medical step).

Important Disclaimer

We explicitly do not claim that this swing is dermoscopy-domain transfer: DermLIP's Derm1M corpus almost certainly contains HAM10000, and the experiments reported here cannot separate dermoscopy-domain transfer from HAM10000 image-level overlap.

Contributions & Future Work

We release the methodology, the configurations, and the run archive, and sketch a candidate partition experiment (a non-HAM10000 dermoscopy SSL pretrain) as a natural next step we may consider after gathering community feedback on the methodology and the preliminary results. The contribution is the proxy-task design, the failure characterisation, and a clearly scoped open question --- not a conclusion about whether frozen-backbone JEPA generalises on dermoscopy.