BLM-AgrF: A New French Benchmark to Investigate Generalization of Agreement in Neural Networks

BLM-AgrF is a French dataset for learning the underlying rules of subject-verb agreement in sentences, developed in the BLM framework, a new task inspired by visual IQ tests known as Raven's Progressive Matrices.  In this task, an instance consists of sequences of sentences with specific attributes. To predict the correct answer as the next element of the sequence, a model must correctly detect the generative model used to produce the dataset.