Affective State Recognition Based On Eye Gaze Analysis Using Two-Stream Convolutional Networks

In this paper, we propose a novel technique that combines the concept of spatially targeted optical flow with image processing, for affect state recognition, concerning a wide variety of learner types, including children with autism and mainstream children. We exploit the advantages of deep Neural Networks on image classification, by adopting a two-stream CNN approach for the recognition task, based on gaze analysis. As there is not a publicly available dataset to contain such a variety of learner types, a dataset was created in order to evaluate the performance of our algorithm. We validate our approach using this dataset, by optimising a mean-square error loss function, obtaining promising results for this challenging task.