Multimodal Sentiment Analysis: A Systematic review of History, Datasets, Multimodal Fusion Methods, Applications, Challenges and Future Directions

Abstract

Sentiment analysis (SA), a buzzword in the fields of artificial intelligence (AI) and natural language processing (NLP), is gaining popularity. Due to numerous SA applications, there is an increasing need to automate the procedure of analysing the user's feelings concerning any products or services. Multimodal Sentiment Analysis (MSA), a branch of sentiment analysis that uses many modalities, is a rapidly growing topic of study as more and more opinions are expressed through videos rather than just text. Recent advances in machine learning are used by MSA to advance. At each stage of the MSA, the most recent developments in machine learning and deep learning are used, including sentiment polarity recognition, multimodal features extraction, and multimodal fusion with reduced error rates and increased speed. This research paper categorises several recent developments in MSA designs into 10 categories and focuses mostly on the primary taxonomy and recently published Multimodal Fusion architectures. The 10 categories are: early fusion, late fusion, hybrid, model-level fusion, tensor fusion, hierarchical, bi-modal, attention-based, quantum-based, and word-level fusion. The primary contribution of this manuscript is a study of the advantages and disadvantages of various architectural developments in MSA fusion. It also talks about future scope, uses in other industries, and research shortages.