Sign Translation Via Natural Language Processing (original) (raw)

Sign Language to Text-Speech Translator Using Machine Learning

International Journal of Emerging Trends in Engineering Research, 2021

Communication with deaf and dumb people is quite difficult task for others. So, through sign language can communicate with deaf and mute persons but it is difficult for normal people to understand the sign language hence it creates a huge gap between them and it's uneasy to exchange their ideas, thoughts with others. This gap has existed for years in order to minimize this, new technologies should be emerged. Therefore, an interpreter is necessary which acts as a bridge between deaf-mute and others. This paper proposed system which is a sign language translator. The system used American Sign Language (ASL) dataset which is pre-processed based on threshold and intensity. This system recognizes sign language alphabet and by joining the letters it creates a sentence then it converts the text to speech. As the system is based on hand, hand gesture is used in sign language recognition system, for that the efficient hand tracking technique which is given by media pipe cross platform is used and it exactly detects the hand after that by using the ANN architecture the model has trained and which classifies the images. The system has achieved 74% accuracy and recognize almost all the letters. The system which also converts sign text to speech so that it will also helpful for blind people.

AI-Enabled Real-Time Sign Language Translator

Advances in Intelligent Systems and Computing, 2020

Even in recent times with the advancement in technology, there exists a hindrance in seamless communication with the hearing and speech-impaired section of the society. Inclusive communication is instrumental for a society to function as a whole. It is not only essential for exchanging ideas, but also for progress and innovation. A lack of means for spontaneous communication should not stand in the way of socializing, employment or productivity. We propose an android application that interprets American Sign Language into English language using convolutional neural network with an aim to provide real-time translation to facilitate seamless communication. Although there is a presence of computer-based translation application for sign language recognition, the availability of such applications over an android platform is relatively few in number. The proposed sign language translator finds its applicability in gesture-controlled applications like human-computer interaction, providing control action for various home appliances and electronic gadgets that are triggered by gestures when given as an input. The proposed work is aimed to transform hearing and speech abnormality to normality, thus eliminating their dependencies.

IJERT-Automatic Translate Real-Time Voice to Sign Language Conversion for Deaf and Dumb People

International Journal of Engineering Research and Technology (IJERT), 2021

https://www.ijert.org/automatic-translate-real-time-voice-to-sign-language-conversion-for-deaf-and-dumb-people https://www.ijert.org/research/automatic-translate-real-time-voice-to-sign-language-conversion-for-deaf-and-dumb-people-IJERTCONV9IS05037.pdf Sign Language Recognition is one of the most growing fields of research area. Many new techniques have been developed recently in this area. The Sign Language is mainly used for communication of deaf-dumb people. In this paper, we propose design and initial implementation of a robust system which can automatically translates voice into text and text to sign language animations. Sign Language Translation Systems could significantly improve deaf lives especially in communications, exchange of information and employment of machine for translation conversations from one language to another has. Therefore, considering these points, it seems necessary to study the speech recognition. Usually, the voice recognition algorithms address three major challenges. The first is extracting feature form speech and the second is when limited sound gallery are available for recognition, and the final challenge is to improve speaker dependent to speaker independent voice recognition. Extracting feature form speech is an important stage in our method. Different procedures are available for extracting feature form speech. One of the commonest of which used in speech recognition systems is Mel-Frequency Cepstral Coefficients (MFCCs). The algorithm starts with preprocessing and signal conditioning. Next extracting feature form speech using Cepstral coefficients will be done. Then the result of this process sends to segmentation part. Finally recognition part recognizes the words and then converting word recognized to facial animation. The project is still in progress and some new interesting methods are described in the current report. The system will perform the recognition process through matching the parameter set of the input speech with the stored templates to finally display the sign language in caption of video on the screen of computer/mobile etc. So, Deaf and Dumb people or students easily learn the subject through the online YouTube video.

An Open CV Framework of Automated Sign language to Text translator for Speech and Hearing Impaired

Generally hearing impaired people use sign language for communication, but they find difficulty in communicating with others who don " t understand sign language. This project aims to lower this barrier in communication. It is based on the need of developing an electronic device that can translate sign language into text in order to make the communication take place between the mute communities and the general public as possible. Computer recognition of sign language is an important research problem for enabling communication with hearing impaired people. This project introduces an efficient and fast algorithm for identification of the number of fingers opened in a gesture representing text of the Binary Sign Language. The system does not require the hand to be perfectly aligned to the camera and any specific back ground for camera. The project uses image processing system to identify, especially English alphabetic sign language used by the deaf people to communicate. The basic objective of this project is to develop a computer based intelligent system that will enable the hearing impaired significantly to communicate with others using their natural hand gestures. The idea consisted of designing and building up an intelligent system using image processing, machine learning and artificial intelligence concepts to take visual inputs of sign language " s hand gestures and generate easily recognizable form of outputs. Hence the objective of this project is to develop an intelligent system which can act as a translator between the sign language and the spoken language dynamically and can make the communication between people with hearing impairment and normal people both effective and efficient. The system is we are implementing for Binary sign language but it can detect any sign language with prior image processing.

SIGN LANGUAGE INTERFACE SYSTEM FOR HEARING IMPAIRED PEOPLE

IRJET, 2022

Communicating with people is a fundamental form of interaction, and being able to do so with people who have hearing impairments has proven to be a significant challenge. As a result, we are using Sign language as a critical tool for bridging the communication gap between hearing and deaf people. Communicating through sign language is difficult, and people with hearing impairments struggle to share their feelings and opinions with people who do not speak that language. To address this issue, we require a versatile and robust product. We proposed a system that aims to translate hand signals into their corresponding words as accurately as possible using data collected and another research. This can be accomplished through the use of various machine learning algorithms and techniques for classifying data into words. For our proposed system, we are using the Convolutional neural networks (CNN) algorithm. The process entails being able to capture multiple images of various hand gestures using a web cam in this analysis, and the system being able to predict and display the corresponding letters/words of the captured images of hand gestures that have previously been trained with Audio. Once an image is captured, it goes through a series of procedures that include computer vision techniques such as gray-scale conversion, mask operation, and dilation. The main goal and purpose of this project will be to remove the barrier between deaf and dumb people and normal people.

Novel AI Powered Sign Language Translator

2022

Although we see many advancements in voice-enabled technologies, there is an acute disparity between speech-based human-machine interface technologies (e.g. speech to text) and those designed for the speech and hearing impaired. This paper proposes a novel camera vision powered translator for American Sign Language for real-time use. It incorporates technologies such as Mediapipe pose estimation and Dynamic Time Warping for two separate sign language translators - one for alphanumeric characters and the other for words and phrases. This novel method does not require intensive training, and has demonstrated an accuracy of over 90 percent on a test data-set.

An App based Solution for Sign Language to Text Conversion

international journal for research in applied science and engineering technology ijraset, 2020

Sign languages are a way in which languages use the visual-manual means to convey a message to the other person. It usually has a predefined sign with each symbol representing a letter. Series of signs are used in order to convey a sentence. The main aim of this paper is to develop a mobile application-based solution that takes sign language gestures as input to a trained deep learning model built using 2D Convolutional Neural Networks and converts it to text and voice outputs in real-time for improved and finer communication. The basic idea behind this is that sign language is not one that is commonly learnt by all and hence people who are familiar with sign language only are able to communicate with the mute. This solution aims to bridge that knowledge gap between people irrespective of their familiarity with sign language. After implementing our solution, it is found that the model predicts gestures with an accuracy rate of 78%. Once translated, the text was also converted to audio output using Text-to-speech library(tts). This app-based approach comes in handy as most people today use smartphones and hence the application can reach out to more users.

Virtual Assistant With Sign Language

International Journal of Scientific Research in Science, Engineering and Technology, 2023

This topic is all about developing a successful Sign language recognition and translation system which is helpful for people with certain disabilities who can’t use present voice enabled Virtual Assistants. Communication is important with people as it helps us understand them and gain knowledge. In case of Deaf & mute people, this becomes a problem. It is solved by creating a virtual assistant built to recognize hand gestures and translate them in normal language to perform tasks ordered by the user. This certainly helps other normal people to successfully communicate with deaf & mute people Without worrying about any misunderstanding at the listener’s end. There are more than 300 sign languages used by various cultural groups worldwide. In this article, we provide a technique for generating a sign language dataset using a camera, followed by the training of a TensorFlow model using transfer learning to produce a real-time sign language recognition system. Despite the small amount of the dataset, the system still performs well.

A Real-Time Automatic Translation of Text to Sign Language

Computers, Materials & Continua, 2022

Communication is a basic need of every human being; by this, they can learn, express their feelings and exchange their ideas, but deaf people cannot listen and speak. For communication, they use various hands gestures, also known as Sign Language (SL), which they learn from special schools. As normal people have not taken SL classes; therefore, they are unable to perform signs of daily routine sentences (e.g., what are the specifications of this mobile phone?). A technological solution can facilitate in overcoming this communication gap by which normal people can communicate with deaf people. This paper presents an architecture for an application named Sign4PSL that translates the sentences to Pakistan Sign Language (PSL) for deaf people with visual representation using virtual signing character. This research aims to develop a generic independent application that is lightweight and reusable on any platform, including web and mobile, with an ability to perform offline text translation. The Sign4PSL relies on a knowledge base that stores both corpus of PSL Words and their coded form in the notation system. Sign4PSL takes English language text as an input, performs the translation to PSL through sign language notation and displays gestures to the user using virtual character. The system is tested on deaf students at a special school. The results have shown that the students were able to understand the story presented to them appropriately.

Design Of Sign Language Interpreter For Specially Challenged Community

European Journal of Molecular & Clinical Medicine, 2020

A special type of language is used in deaf community across the globe a form of non-verbal sign language. The language interpretation is very difficult because of not derived from the common source of origin. The conversion of hand gesture into auditory speech uses a device called Deaf-Mute language interpreter. Hand movement with particular shape and angle is considered as a sign language. In the meantime, gesture also adds the facial expression. Sign is the static shape or position of the hand to indicate communication. This work focus to provide a glove based smart system to translate deaf-mute communication. The objective is to develop a smart glove that can able to interpret 10 characters from the American Sign Language (ASL).The smart glove is designed in such a way that, LDR sensors are attached to fingers to gather the position of the finger to identify the characters. The design uses MSP430 microcontroller to meet the low power constraints for portable smart glove. The translated character is sent to computer through ZigBee wireless protocol and then pronounces the letter through the text to voice module. The whole translation mechanism takes minimum time to read the finger position and to produce a voice message.