Toward Human-Inspired RAG: Hierarchical Vector Compression and Topic-Guided Retrieval

H-RAG: Hierarchical Vector Compression and Topic-Guided Retrieval introduces a human-inspired approach to Retrieval-Augmented Generation by replacing flat vector search with a hierarchical retrieval structure. Instead of comparing a query against all document chunks at once, H-RAG organizes embeddings into topic trees, where root nodes represent broad domains, internal nodes represent compressed semantic concepts, and leaf nodes contain the original document chunks.

The method combines semantic vector compression using saliency-weighted mean pooling with top-down topic-tree traversal, allowing retrieval to move from high-level topics toward precise passages. This design aims to reduce unnecessary similarity comparisons, improve ranking quality, and better support multi-domain retrieval.

A pilot evaluation on a retrieval-focused subset of SQuAD 2.0 compares H-RAG against Flat RAG, Sentence-Window retrieval, Parent-Child retrieval, and a RAPTOR-like baseline. The results show that H-RAG achieves the best MRR, Recall@1, and NDCG@5, while using significantly fewer similarity comparisons than flat and sentence-window retrieval approaches. The paper presents H-RAG as a preliminary but promising step toward more scalable, structured, and human-like retrieval systems.