AffectAI: An Emotion-Aware conversational AI for Human-Computer Interaction

In the present digital environment, conversational agents are becoming more and more common but they usually do not know how to interpret and react to human emotions, contributing to robot-like communication. The current study introduces AffectAI, an effective conversational system that will close this emotional intelligence gap. The holistic AffectAI system envisions a multimodal system, with the text sentiment analysis, speech emotion recognition, and facial expression detection to identify the affective state of a user. The conversational engine uses a dialogue manager which is a reinforcement learning agent that is capable of dynamically adjusting its tone, empathy level, and response content depending on the emotion that has been detected, and the context of the dialogue. The paper describes the design, development, and testing of the underlying visual element of this system, a real-time facial emotion recognition module. Based on a Convolutional Neural Network (CNN) trained using TensorFlow and optimized to execute on the Android platform through TensorFlow Lite, this module is implemented on the Android platform. Experimental analyses prove that this element can show a higher accuracy in emotion detection and user interaction, creating a strong paradigm of building more empathetic, context-aware, and human-like conversational agents to work in the mental health, education, and customer service areas.