Sign Language Recognition System using TensorFlow Object Detection API
- URL: http://arxiv.org/abs/2201.01486v1
- Date: Wed, 5 Jan 2022 07:13:03 GMT
- Title: Sign Language Recognition System using TensorFlow Object Detection API
- Authors: Sharvani Srivastava, Amisha Gangwar, Richa Mishra, Sudhakar Singh
- Abstract summary: In this paper, we propose a method to create an Indian Sign Language dataset using a webcam and then using transfer learning, train a model to create a real-time Sign Language Recognition system.
The system achieves a good level of accuracy even with a limited size dataset.
- Score: 0.0
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Communication is defined as the act of sharing or exchanging information,
ideas or feelings. To establish communication between two people, both of them
are required to have knowledge and understanding of a common language. But in
the case of deaf and dumb people, the means of communication are different.
Deaf is the inability to hear and dumb is the inability to speak. They
communicate using sign language among themselves and with normal people but
normal people do not take seriously the importance of sign language. Not
everyone possesses the knowledge and understanding of sign language which makes
communication difficult between a normal person and a deaf and dumb person. To
overcome this barrier, one can build a model based on machine learning. A model
can be trained to recognize different gestures of sign language and translate
them into English. This will help a lot of people in communicating and
conversing with deaf and dumb people. The existing Indian Sing Language
Recognition systems are designed using machine learning algorithms with single
and double-handed gestures but they are not real-time. In this paper, we
propose a method to create an Indian Sign Language dataset using a webcam and
then using transfer learning, train a TensorFlow model to create a real-time
Sign Language Recognition system. The system achieves a good level of accuracy
even with a limited size dataset.
Related papers
- Continuous Sign Language Recognition System using Deep Learning with MediaPipe Holistic [1.9874264019909988]
Sign languages are the language of hearing-impaired people who use visuals for communication.
Approximately 300 sign languages are being practiced worldwide such as American Sign Language (ASL), Chinese Sign Language (CSL), Indian Sign Language (ISL)
arXiv Detail & Related papers (2024-11-07T08:19:39Z) - Scaling up Multimodal Pre-training for Sign Language Understanding [96.17753464544604]
Sign language serves as the primary meaning of communication for the deaf-mute community.
To facilitate communication between the deaf-mute and hearing people, a series of sign language understanding (SLU) tasks have been studied.
These tasks investigate sign language topics from diverse perspectives and raise challenges in learning effective representation of sign language videos.
arXiv Detail & Related papers (2024-08-16T06:04:25Z) - Lip Reading for Low-resource Languages by Learning and Combining General
Speech Knowledge and Language-specific Knowledge [57.38948190611797]
This paper proposes a novel lip reading framework, especially for low-resource languages.
Since low-resource languages do not have enough video-text paired data to train the model, it is regarded as challenging to develop lip reading models for low-resource languages.
arXiv Detail & Related papers (2023-08-18T05:19:03Z) - A Comparative Analysis of Techniques and Algorithms for Recognising Sign
Language [0.9311364633437358]
Sign language is frequently used as the primary form of communication by people with hearing loss.
It is necessary to create human-computer interface systems that can offer hearing-impaired people a social platform.
Most commercial sign language translation systems are sensor-based, pricey, and challenging to use.
arXiv Detail & Related papers (2023-05-05T10:52:18Z) - Image-based Indian Sign Language Recognition: A Practical Review using
Deep Neural Networks [0.0]
This model is to develop a real-time word-level sign language recognition system that would translate sign language to text.
For this analysis, the user must be able to take pictures of hand movements using a web camera.
Our model is trained using a convolutional neural network (CNN), which is then utilized to recognize the images.
arXiv Detail & Related papers (2023-04-28T09:27:04Z) - Indian Sign Language Recognition Using Mediapipe Holistic [0.0]
We will create a robust system for sign language recognition in order to convert Indian Sign Language to text or speech.
The creation of a text-to-sign language paradigm is essential since it will enhance the sign language-dependent deaf and hard-of-hearing population's communication skills.
arXiv Detail & Related papers (2023-04-20T12:25:47Z) - Multilingual Communication System with Deaf Individuals Utilizing
Natural and Visual Languages [12.369283590206628]
We propose a novel multilingual communication system, namely MUGCAT, to improve the communication efficiency of sign language users.
By converting recognized specific hand gestures into expressive pictures, our MUGCAT system significantly helps deaf people convey their thoughts.
arXiv Detail & Related papers (2022-12-01T06:43:44Z) - Skeleton Based Sign Language Recognition Using Whole-body Keypoints [71.97020373520922]
Sign language is used by deaf or speech impaired people to communicate.
Skeleton-based recognition is becoming popular that it can be further ensembled with RGB-D based method to achieve state-of-the-art performance.
Inspired by the recent development of whole-body pose estimation citejin 2020whole, we propose recognizing sign language based on the whole-body key points and features.
arXiv Detail & Related papers (2021-03-16T03:38:17Z) - Acoustics Based Intent Recognition Using Discovered Phonetic Units for
Low Resource Languages [51.0542215642794]
We propose a novel acoustics based intent recognition system that uses discovered phonetic units for intent classification.
We present results for two languages families - Indic languages and Romance languages, for two different intent recognition tasks.
arXiv Detail & Related papers (2020-11-07T00:35:31Z) - Vokenization: Improving Language Understanding with Contextualized,
Visual-Grounded Supervision [110.66085917826648]
We develop a technique that extrapolates multimodal alignments to language-only data by contextually mapping language tokens to their related images.
"vokenization" is trained on relatively small image captioning datasets and we then apply it to generate vokens for large language corpora.
Trained with these contextually generated vokens, our visually-supervised language models show consistent improvements over self-supervised alternatives on multiple pure-language tasks.
arXiv Detail & Related papers (2020-10-14T02:11:51Z) - Meta-Transfer Learning for Code-Switched Speech Recognition [72.84247387728999]
We propose a new learning method, meta-transfer learning, to transfer learn on a code-switched speech recognition system in a low-resource setting.
Our model learns to recognize individual languages, and transfer them so as to better recognize mixed-language speech by conditioning the optimization on the code-switching data.
arXiv Detail & Related papers (2020-04-29T14:27:19Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.