A Novel CNN Based Hand Gesture Identification and Recognition to Text and Voice

Abstract

Conversing to a person with hearing disability is always a major challenge. Sign language has indelibly become the 
ultimate panacea and is a very powerful tool for individuals with hearing and speech disability to communicate their 
feelings and opinions to the world. It makes the integration process between them and others smooth and less 
complex. However, the invention of sign language alone, is not enough. There are many strings attached to this 
boon. The sign gestures often get mixed and confused for someone who has never learnt it or knows it in a different 
language. However, this communication gap which has existed for years can now be narrowed with the introduction 
of various techniques to automate the detection of sign gestures. In this paper, we introduce a Sign Language 
recognition using American Sign Language. In this study, the user must be able to capture images of the hand 
gesture using web camera and the system shall predict and display the name of the captured image.We use the 
HSV color algorithm to detect the hand gesture and set the background to black. The images undergoa series of 
processing steps which include various Computer vision techniques such as the conversion to grayscale, dilation 
and mask operation. And the region of interest which, in our case is the hand gesture is segmented. The features 
extracted are the binary pixels of the images. We make use of Convolutional Neural Network (CNN) for training 
and to classify the images. We are able to recognize 10 American Sign gesture alphabets with high accuracy. Our 
model has achieved a remarkable accuracy of above 90%. 

 Sign Language, ASL, Hearing disability, Convolutional Neural Network (CNN), Computer Vision, 
Machine Learning, Gesture recognition, Sign language recognition, Hue Saturation Value algorithm.