Gaze-based Personal Memory: Leveraging Eye Tracking to Improve Relevance in Text Retrieval Systems

g-Rel-RANKER is a benchmark dataset for studying gaze-based boosting strategies in text retrieval and personal memory systems.  It contains two complementary components: (1) an extension of the public g-Rel-READER corpus with additional background passages and refined relevance annotations, and (2) a user study corpus collected with a dedicated search prototype.  Both components provide aligned text passages, gaze data, and passage-level relevance labels, enabling systematic evaluation of whether and how eye tracking signals can improve retrieval of information that users have actually read before.

The g-Rel-READER-based part comprises news-style documents originally used for relevance judgement experiments, enriched with passage segmentations, screen coordinates for each word, gaze fixations from 24 participants, and curated relevance labels for question–passage pairs.  To approximate realistic search scenarios, this core is complemented with 2.6M background passages sampled from English Wikipedia, of which a pooled subset is labeled for relevance using a combination of LLM-based and expert judgement.  This structure yields a mixed corpus containing (a) relevant and non-relevant passages with gaze (previously read “personal memory”) and (b) relevant and non-relevant passages without gaze (unseen or merely visible content).

The user study component captures a more natural visual-memory setting: nine participants read selected Wikipedia passages (German language), while gaze was recorded and passages were indexed in an Elasticsearch-based prototype.  Afterwards, participants issued both self-formulated and predefined questions; for each query, they rated retrieved passages for relevance, which is again complemented with automatic LLM judgements.  This second component mirrors the structure of g-Rel-RANKER (text, gaze, relevance) and allows testing the transfer of methods from controlled benchmark conditions to practical search scenarios.

Across both components, g-Rel-RANKER is designed to evaluate a family of gaze-based boosting functions on top of standard BM25 retrieval, including boolean gaze filters, distance-weighted gaze-point counts, reading intensity, and reading-velocity-based boosts.  The dataset and accompanying benchmark code support common IR metrics (MRR, MAP, Hits@K) and separate analyses of viewed vs. non-viewed passages, making it possible to quantify trade-offs between prioritizing previously read content and maintaining recall for unseen but relevant material.  Overall, g-Rel-RANKER provides a reproducible resource to assess the feasibility of using gaze data as an implicit personalization signal in information retrieval and personal memory systems.