A GLOVE THAT CONVERTS SIGN LANGUAGE INTO TEXT AND SPEECH

Abstract

Sign language is one of the world’s languages, but instead of using speech or voice, it communicates by hand signals and other body parts. Speech and hearing-impaired people need to be able to communicate in sign language. A speech impaired person is someone who is unable to talk due to a reluctance or in ability to speak. A person who has been hearing-impaired since birth or loss may develop later. Malaysian Sign Language (MSL) will be used by Malaysians speech and hearing-impaired population to communicate, but it will be an obstacle as they need to converse with normal people. Currently, sign language translators are mostly focused on general sign language and American Sign Language (ASL). This project aimed to develop a sign language translator system based on MSL with Node MCU ESP8266, flex sensors and MPU6050 accelerometer gyroscope sensor module that would enable speech and hearing-impaired people to communicate with regular people who are unfamiliar with sign language. It used a data glove approach method that can convert a hand gesture into words. In this context, the flex sensors defined the sensor values based on finger bending, meanwhile, the accelerometer gyro sensor module defined the hand position. This project implements IoT technology to display the meaning of sign language gestures on a smartphone. A Node MCU ESP8266 module was also used as a source IoT platform by integrating with the Blynk application in the smartphone to display the output of the system.The integration of the NodeMCU ESP8266 makes the system capable of transferring data via Wi-Fi. The data captured from the sensors is processed, and the gesture’s meaning is displayed on a smartphone screen via the Blynk app, offering a real-time translation of the sign into text. This IoT functionality is crucial as it allows the system to be portable, user friendly, and flexible for various use cases, especially in everyday situations where a smartphone can easily be used to read the translation.