VQA-Introspect with Logical Relations

This is a version of the VQA-Introspect dataset by Selvaraju et al., but with added annotations about logical relations for binary QA pairs. Relations have been predicted using a fine-tuned BERT, which was pre-trained for NLI and fine-tuned on a sub-set of VQA-Introspect.

In general, entries have the following fields:
img_id: This is the image name without extension (images come from COCO)
question_id: Question identifier as int
sent: String version of the question
question_type: Type of question (how it starts)
answer_type: Type of answer
label: Answers using soft scores (as required by LXMERT)
role: Question role (main, sub or unk)
Questions with role='sub' also have a field named parent, which indicates the ID of the QA pair it is related to, and a field named rel, which contains the relation to the parent.

Total samples:
Train: 215862 (sub: 160085, main: 55777)
Val: 69668 (sub: 49882, main: 19786)

Images must be downloaded separately from the COCO Dataset website.

If you use this dataset, please cite:

@inproceedings{tascon2023logical, title={Logical Implications for Visual Question Answering Consistency},
author={Tascon-Morales, Sergio and M{\'a}rquez-Neila, Pablo and Sznitman, Raphael}, 
booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
pages={6725--6735}, 
year={2023} 
}

as well as the original publication where VQA-Introspect was presented:

@inproceedings{selvaraju2020squinting,
  title={Squinting at vqa models: Introspecting vqa models with sub-questions},
  author={Selvaraju, Ramprasaath R and Tendulkar, Purva and Parikh, Devi and Horvitz, Eric and Ribeiro, Marco Tulio and Nushi, Besmira and Kamar, Ece},
  booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  pages={10003--10011},
  year={2020}
}

Note: We respect the original license terms of the VQA-Introspect dataset and make manifest that the liability warranty described in those terms (§ 4.2 and 4.3) apply to this dataset too.