Sign Language Recognition: Can depth cameras be used to correct Mediapipe errors?

Mediapipe is increasingly deployed in Sign Language Recognition (SLR) pipelines. This paper highlights errors in Mediapipe that adversely affect the performance of these models. The use of depth cameras has been intuitively proposed for the correction of depth estimation errors. We propose a Mediapipe input pre-processing technique using depth sensor data specifically tailored for SLR systems. However, our experimental results show that such techniques are problematic owing to the inherent distortion in depth sensor output.