Anomaly- Aware Gesture Recognition for Sign Language Translation Using Deep Learning

Abstract

Gesture-based word suggestion and
sentence formation using sign language
is an innovative approach to bridge the
communication gap between the hearingimpaired
and the general population. In
this work, MediaPipe is employed to
extract real-time hand landmarks and
gestures from video streams, ensuring
efficient and lightweight pose detection.
These extracted gesture features are then
fed into deep learning models, such as
CNNs and RNNs, to recognize signs with
high accuracy. The recognized gestures
are mapped to corresponding words,
which are further processed through a
language model for context-aware word
suggestions. This enables the system to
not only recognize isolated signs but also
form meaningful sentences dynamically.
The framework provides suggestions to
handle incomplete or ambiguous signs,
improving usability. By integrating
gesture