Vis Quelle: Visual Question-based Elementary Learning Companion a system to Facilitate Learning Word-Object Associations

We present an automated, visual question answering based companion – Vis Quelle - to facilitate elementary learning of word-object associations. In particular, we attempt to harness the power of machine learning models for object recognition and the understanding of combined processing of images and text data from visual-question answering to provide variety and nuance in the images associated with letters or words presented to the elementary learner. We incorporate elements such as gamification to motivate the learner by recording scores, errors, etc., to track the learner’s progress. Translation is also provided to reinforce word-object associations in the user’s native tongue, if the learner is using Vis Quelle to learn a second language. Keywords: Visual question answering; object recognition; question generation; question answering; word-object association.