VOICE CONTROLLED ROBOTIC ARM WITH OBJECT DETECTION AS A FEEDBACK SYSTEM, FOR VISUALLY IMPAIRED INDIVIDUALS

The purpose of this thesis is to introduce a robot arm as an assistive technology oriented to the aid of individuals with visual impairment. The robot arm will sit at the center of a custom printed container to allow it storage of several items that will be retrieved when commanded to do so. The microcontroller (MCU) for the robot arm is an Arduino based MCU, ArbotiX, and is paired with a Raspberry Pi 4b for command translation, given that the commands for the device are voice commands, and will also serve as a processing unit for feedback systems. The voice command translation is based on Google Speech-to-Text API, and the two feedback systems to be explored are based on a weight sensor and object recognition under the YOLOv7 model. Experimentation has been conducted to visualize the efficacy of the robot arm in fulfilling its intended purpose and to measure the effectiveness of the implemented system for command delivery to the robot arm, and both feedback systems.

The purpose of this robot arm is to aid individuals with visual impairment by retrieving, under voice command, spice bottles from its container and delivering it to the user of this device; thus, facilitating what would be a challenging task to a blind person, such as retrieving spices for cooking purposes.