LIP READING AND PREDICTION SYSTEM BASED ON DEEP LEARNING

— Speech perception is characterized as a multimodal process, which means it elicits several meanings. Understanding a message can be aided by, and in some cases even made necessary by, lip reading, which overlays visual cues on top of auditory signals. Lip-reading is a crucial field with many uses, including biometrics, speech recognition in noisy environments, silent dictation, and enhanced hearing aids. It is a challenging research project in the area of computer vision, whose major goal is to watch the movement of human lips in a video and recognize the textual content that goes with it. Yet, due to the constraints of lip changes and the depth of linguistic information, the complexity of lip identification has increased, which has slowed the growth of study themes in lip language. Nowadays, deep learning has advanced in several sectors, giving us the confidence to perform the task of lip recognition. Lip learning based on deep learning often entails extracting features and comprehending images using a network model, as opposed to classical lip recognition that recognizes lip characteristics. The design of the network framework for data gathering, processing, and data recognition for lip reading is the main topic of this discussion. In this research, we created a reliable and accurate method for lip reading. We first isolate the mouth region and segment it, after which we extract various aspects from the lip image, such as the Hog, Surf, and Haar features. Lastly, we use Gated Recurrent Units to train our deep learning model (GRU).