StillMe: A Practical Framework for Building Transparent, Validated Retrieval-Augmented Generation Systems

StillMe is an open-source, transparency-first framework for building validated Retrieval-Augmented
Generation (RAG) systems designed to address three critical issues in modern AI: black-box opacity,
hallucination, and knowledge cutoff limitations.

Rather than modifying or retraining Large Language Models (LLMs), StillMe demonstrates that
commercial LLMs can be transformed into ethical, auditable systems by surrounding them with a
transparent architecture consisting of:

• Continuous Learning: Automated ingestion from trusted sources (RSS feeds, arXiv, CrossRef,
  Wikipedia) every 4 hours, overcoming traditional LLM knowledge cutoff limitations.

• Multi-layer Validation Chain: Six validators (citation, evidence overlap, numeric consistency,
  uncertainty, ethics, fallback) that enforce grounding, intellectual humility, and safety while reducing
  hallucinations.

• Full System Transparency: Every response includes citations, validation results, and an audit trail.
  All learning sources, metrics, and validation outcomes are fully visible through dashboards and APIs.

We evaluate StillMe on the TruthfulQA benchmark. On a 50-question subset, StillMe achieves 56%
accuracy (vs. 52% for GPT-4 under identical conditions), 100% citation rate, and a 70.6% transparency
score. On an extended 634-question evaluation, StillMe achieves 99.7% citation coverage and 70.9%
transparency score, demonstrating the robustness of the validation chain across challenging subsets.

Unlike traditional approaches focusing on LLM interpretability, StillMe focuses on “system-level
transparency”: making every step of the RAG pipeline visible and auditable. This makes StillMe a
practical, deployable alternative to closed commercial AI systems.

The full codebase, dashboards, evaluation scripts, and deployment instructions are fully open-source:
https://github.com/anhmtk/StillMe-Learning-AI-System-RAG-Foundation