AOL4FOLTR

AOL4FOLTR is the first learning-to-rank (LTR) dataset designed specifically for evaluating federated online learning-to-rank (FOLTR) algorithms.

Including user identifiers and timestamps, this dataset allows for the simulation of real user behavior with heterogeneous data and in asynchronous federated learning settings.

The dataset consists of two files

• letor.txt.gz (55G uncompressed)
• metadata.csv

letor.txt contains the query-document pairs for all query logs in standard LETOR format. Each query-document pair holds a binary label derived from user clicks, and is further represented by a 103-dimensional vector. We document the features in our code repository.

The query logs are cross-referenced (by qid) in metadata.csv, where contextual information is provided. This includes the user, timestamp, raw query, the target document ID, and a list of 20 candidate documents.

The document IDs and user IDs directly map to the AOL-IA dataset; the query IDs do not. For access to the raw document contents, please refer to this dataset.