SIGN LANGUAGE RECOGNITION TO TEXT AND VOICE USING CNN

Abstract

People with hearing impairments are found worldwide therefore, the development of effective local level sign language recognition (SLR) tools is essential. We conducted a comprehensive review of automated sign language recognition based on machine/deep learning methods and techniques published between 2014 and 2021 and concluded that the current methods require conceptual classification to interpret all available data correctly. Thus, we turned our attention to elements that are common to almost all sign language recognition methodologies. This paper discusses their relative strengths and weaknesses, and we propose a general framework for researchers. This study also indicates that input modalities bear great significance in this field; it appears that recognition based on a combination of data sources, including vision based and sensor based channels, is superior to a unimodal analysis. In addition, recent advances have allowed researchers to move from simple recognition of sign language characters and words towards the capacity to translate continuous sign language communication with minimal delay. Many of the presented models are relatively effective for a range of tasks, but none currently possess the necessary generalization potential for commercial deployment. However, the pace of research is encouraging, and further progress is expected if specific difficulties are resolved. The prevalence of hearing impairments across the globe underscores the necessity for effective local level sign language recognition (SLR) tools that cater to diverse linguistic and cultural contexts. This paper presents a thorough review of the advancements in automated sign language recognition methodologies based on machine and deep learning techniques, focusing on literature published between 2014 and 2021. Our analysis highlights that current methodologies often lack a unified conceptual framework to accurately interpret and utilize the vast array of available data, pointing to the need for a more structured approach to SLR. Recent advancements in the field have progressed from basic recognition of individual sign language characters and words to more complex systems capable of translating continuous sign language communication with minimal delay. This progression is largely attributed to improvements in machine learning models and data processing techniques, which have enabled more fluid and natural interaction between users and recognition systems.