DeepDrift: Zero-Training Hidden-State Monitoring for Robustness in Vision, Language, and Generative Models
Description
We introduce DeepDrift, a unified internal monitoring framework for deep neural networks based on Semantic Velocity — the ℓ2‑norm of consecutive hidden‑state differences.
Key results:
• LLM hallucination detection: AUC 0.891, lead time 7.2 tokens
• RL failure prediction: AUC 0.985 (DQN), AUC 1.000 (PPO+noise), lead time 168 steps
• ViT semantic OOD (CIFAR‑100 → SVHN): AUROC 0.817 [0.788, 0.846]
• Diffusion memorization: 3× earlier than validation loss divergence
• External benchmark on CLIP, DINOv2, ConvNeXt confirms generalization
The method requires zero gradient‑based training, operates as a plug‑in PyTorch monitor with <1.5% overhead, and is available as open‑source software (pip install deepdrift). All experiments are reproducible with provided scripts.
GitHub: https://github.com/Eutonics/DeepDrift
PyPI: pip install deepdrift
Raw experiment metrics are included as JSON files: - vit_ood_metrics.json: AUROC, velocity profiles, bootstrap CIs for ViT-B/16 (CIFAR-100 → SVHN) - rl_cartpole_metrics.json: AUC, Cohen's d, lead time, phase portrait data for CartPole (PPO, DQN, PPO+noise)
Files
DeepDrift Zero-Training Hidden-State Monitoring.pdf
Additional details
Dates
- Available
-
2026-02-12
Software
- Repository URL
- https://github.com/Eutonics/DeepDrift
- Programming language
- Python