Indian Sign Language Recognition Using Mediapipe Holistic

• URL: http://arxiv.org/abs/2304.10256v1
• Date: Thu, 20 Apr 2023 12:25:47 GMT
• Title: Indian Sign Language Recognition Using Mediapipe Holistic
• Authors: Dr. Velmathi G, Kaushal Goyal
• Abstract summary: We will create a robust system for sign language recognition in order to convert Indian Sign Language to text or speech. The creation of a text-to-sign language paradigm is essential since it will enhance the sign language-dependent deaf and hard-of-hearing population's communication skills.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Deaf individuals confront significant communication obstacles on a daily basis. Their inability to hear makes it difficult for them to communicate with those who do not understand sign language. Moreover, it presents difficulties in educational, occupational, and social contexts. By providing alternative communication channels, technology can play a crucial role in overcoming these obstacles. One such technology that can facilitate communication between deaf and hearing individuals is sign language recognition. We will create a robust system for sign language recognition in order to convert Indian Sign Language to text or speech. We will evaluate the proposed system and compare CNN and LSTM models. Since there are both static and gesture sign languages, a robust model is required to distinguish between them. In this study, we discovered that a CNN model captures letters and characters for recognition of static sign language better than an LSTM model, but it outperforms CNN by monitoring hands, faces, and pose in gesture sign language phrases and sentences. The creation of a text-to-sign language paradigm is essential since it will enhance the sign language-dependent deaf and hard-of-hearing population's communication skills. Even though the sign-to-text translation is just one side of communication, not all deaf or hard-of-hearing people are proficient in reading or writing text. Some may have difficulty comprehending written language due to educational or literacy issues. Therefore, a text-to-sign language paradigm would allow them to comprehend text-based information and participate in a variety of social, educational, and professional settings. Keywords: deaf and hard-of-hearing, DHH, Indian sign language, CNN, LSTM, static and gesture sign languages, text-to-sign language model, MediaPipe Holistic, sign language recognition, SLR, SLT

Related papers

• Continuous Sign Language Recognition System using Deep Learning with MediaPipe Holistic [1.9874264019909988]
  Sign languages are the language of hearing-impaired people who use visuals for communication. Approximately 300 sign languages are being practiced worldwide such as American Sign Language (ASL), Chinese Sign Language (CSL), Indian Sign Language (ISL)
  arXiv  Detail & Related papers  (2024-11-07T08:19:39Z)
• Scaling up Multimodal Pre-training for Sign Language Understanding [96.17753464544604]
  Sign language serves as the primary meaning of communication for the deaf-mute community. To facilitate communication between the deaf-mute and hearing people, a series of sign language understanding (SLU) tasks have been studied. These tasks investigate sign language topics from diverse perspectives and raise challenges in learning effective representation of sign language videos.
  arXiv  Detail & Related papers  (2024-08-16T06:04:25Z)
• MS2SL: Multimodal Spoken Data-Driven Continuous Sign Language Production [93.32354378820648]
  We propose a unified framework for continuous sign language production, easing communication between sign and non-sign language users. A sequence diffusion model, utilizing embeddings extracted from text or speech, is crafted to generate sign predictions step by step. Experiments on How2Sign and PHOENIX14T datasets demonstrate that our model achieves competitive performance in sign language production.
  arXiv  Detail & Related papers  (2024-07-04T13:53:50Z)
• Neural Sign Actors: A diffusion model for 3D sign language production from text [51.81647203840081]
  Sign Languages (SL) serve as the primary mode of communication for the Deaf and Hard of Hearing communities. This work makes an important step towards realistic neural sign avatars, bridging the communication gap between Deaf and hearing communities.
  arXiv  Detail & Related papers  (2023-12-05T12:04:34Z)
• A Comparative Analysis of Techniques and Algorithms for Recognising Sign Language [0.9311364633437358]
  Sign language is frequently used as the primary form of communication by people with hearing loss. It is necessary to create human-computer interface systems that can offer hearing-impaired people a social platform. Most commercial sign language translation systems are sensor-based, pricey, and challenging to use.
  arXiv  Detail & Related papers  (2023-05-05T10:52:18Z)
• Image-based Indian Sign Language Recognition: A Practical Review using Deep Neural Networks [0.0]
  This model is to develop a real-time word-level sign language recognition system that would translate sign language to text. For this analysis, the user must be able to take pictures of hand movements using a web camera. Our model is trained using a convolutional neural network (CNN), which is then utilized to recognize the images.
  arXiv  Detail & Related papers  (2023-04-28T09:27:04Z)
• Multilingual Communication System with Deaf Individuals Utilizing Natural and Visual Languages [12.369283590206628]
  We propose a novel multilingual communication system, namely MUGCAT, to improve the communication efficiency of sign language users. By converting recognized specific hand gestures into expressive pictures, our MUGCAT system significantly helps deaf people convey their thoughts.
  arXiv  Detail & Related papers  (2022-12-01T06:43:44Z)
• All You Need In Sign Language Production [50.3955314892191]
  Sign language recognition and production need to cope with some critical challenges. We present an introduction to the Deaf culture, Deaf centers, psychological perspective of sign language. Also, the backbone architectures and methods in SLP are briefly introduced and the proposed taxonomy on SLP is presented.
  arXiv  Detail & Related papers  (2022-01-05T13:45:09Z)
• Sign Language Recognition System using TensorFlow Object Detection API [0.0]
  In this paper, we propose a method to create an Indian Sign Language dataset using a webcam and then using transfer learning, train a model to create a real-time Sign Language Recognition system. The system achieves a good level of accuracy even with a limited size dataset.
  arXiv  Detail & Related papers  (2022-01-05T07:13:03Z)
• Sign Language Production: A Review [51.07720650677784]
  Sign Language is the dominant yet non-primary form of communication language used in the deaf and hearing-impaired community. To make an easy and mutual communication between the hearing-impaired and the hearing communities, building a robust system capable of translating the spoken language into sign language is fundamental. To this end, sign language recognition and production are two necessary parts for making such a two-way system.
  arXiv  Detail & Related papers  (2021-03-29T19:38:22Z)
• Skeleton Based Sign Language Recognition Using Whole-body Keypoints [71.97020373520922]
  Sign language is used by deaf or speech impaired people to communicate. Skeleton-based recognition is becoming popular that it can be further ensembled with RGB-D based method to achieve state-of-the-art performance. Inspired by the recent development of whole-body pose estimation citejin 2020whole, we propose recognizing sign language based on the whole-body key points and features.
  arXiv  Detail & Related papers  (2021-03-16T03:38:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.