TraVLR: Now You See It, Now You Don't! A Bimodal Dataset for Evaluating Visio-Linguisic Reasoning

TraVLR dataset for evaluation of visio-linguistic reasoning and cross-modal transfer. Four reasoning tasks are supported: spatiality, cardinality, numerical comparison and quantifiers. For some of the datasets, we provide two versions which differ only in their training sets: one with 8k training examples, and another with 32k training examples. The validation sets, in-distribution and out-of-distribution test sets are identical.