Context-Aware Multimodal Voice Assistant for Autonomous Daily Navigation of Visually Impaired Users Using Edge Intelligence

The use of assistive navigation by the visually impaired needs solutions that are real-time, context-sensitive, and dependable; nevertheless, currently, the voice-based assistants used have a high latency rate, little contextual knowledge, and rely on the cloud. The paper presents a context-based multimodal voice assistant, which uses edge intelligence in the context of au- tonomous day-to-day navigation. The model combines YOLOv8, CNN with MobileNetV3 to detect objects in real-time and a Bi-LSTM-based voice intent recognition model. A Multimodal Fusion Transformer (MFT) is used to combine audio, visual and contextual features to make adaptive decisions and sensor fusion of GPS, accelerator, and gyroscope data is used to achieve contextual awareness. This system uses TensorFlow Lite to deploy the deployed system, which supports low-latency (less than 120 ms) and offline usage, and requires the directional feedback of Spatial Audio Rendering. The model has an accuracy of 97.2 percent, precision of 95.8 percent and F1-score of 96.4 percent, which is better than traditional assistive systems, as well as a 30-percent lower response latency. Experimentation has shown higher efficiency and safety of navigation by the user in the real world. The given system indicates that multimodal learning combined with edge computing can greatly improve the consistency, responsiveness, and usability of assistive technologies by people with visual impairment.

Assistive technology, Context-Aware Systems, Multimodal Fusion, Edge Computing, voice assistants, visually impaired navigation, object detection, sensor fusion, real-time systems, human–computer interaction.