Classification in Japanese Sign Language Based on Dynamic Facial Expressions

• URL: http://arxiv.org/abs/2411.06347v2
• Date: Tue, 24 Jun 2025 04:10:50 GMT
• Title: Classification in Japanese Sign Language Based on Dynamic Facial Expressions
• Abstract summary: Research on Japanese Sign Language (JSL) recognition is limited due to a lack of datasets.<n>In JSL, sentence types such as affirmative statements and questions are distinguished by facial expressions.<n>Our proposed method utilizes a neural network to analyze facial features and classify sentence types.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Sign language is a visual language expressed through hand movements and non-manual markers. Non-manual markers include facial expressions and head movements. These expressions vary across different nations. Therefore, specialized analysis methods for each sign language are necessary. However, research on Japanese Sign Language (JSL) recognition is limited due to a lack of datasets. The development of recognition models that consider both manual and non-manual features of JSL is crucial for precise and smooth communication with deaf individuals. In JSL, sentence types such as affirmative statements and questions are distinguished by facial expressions. In this paper, we propose a JSL recognition method that focuses on facial expressions. Our proposed method utilizes a neural network to analyze facial features and classify sentence types. Through the experiments, we confirm our method's effectiveness by achieving a classification accuracy of 96.05%.

Related papers

• Emotion Recognition in Signers [5.040463208115642]
  We use eJSL, a new benchmark dataset for emotion recognition in Japanese Sign Language signers, and BOBSL, a large British Sign Language dataset with subtitles.<n>In eJSL, two signers expressed 78 distinct utterances with each of seven different emotional states, resulting in 1,092 video clips.<n>We empirically demonstrate that 1) textual emotion recognition in spoken language mitigates data scarcity in sign language, 2) temporal segment selection has a significant impact, and 3) incorporating hand motion enhances emotion recognition in signers.
  arXiv  Detail & Related papers  (2025-12-17T12:26:21Z)
• The Importance of Facial Features in Vision-based Sign Language Recognition: Eyes, Mouth or Full Face? [5.92177182003275]
  We show that the mouth is the most important non-manual facial feature, significantly improving accuracy.<n>Our findings highlight the necessity of incorporating facial features in automatic sign language recognition.
  arXiv  Detail & Related papers  (2025-07-28T14:36:46Z)
• Signs as Tokens: A Retrieval-Enhanced Multilingual Sign Language Generator [55.94334001112357]
  We introduce a multilingual sign language model, Signs as Tokens (SOKE), which can generate 3D sign avatars autoregressively from text inputs.<n>We propose a retrieval-enhanced SLG approach, which incorporates external sign dictionaries to provide accurate word-level signs.
  arXiv  Detail & Related papers  (2024-11-26T18:28:09Z)
• Empowering Sign Language Communication: Integrating Sentiment and Semantics for Facial Expression Synthesis [0.7223509567556217]
  This paper introduces a new method focused in synthesizing facial expressions for sign language. Our goal is to improve sign language production by integrating sentiment information in facial expression generation.
  arXiv  Detail & Related papers  (2024-08-27T15:55:18Z)
• Scaling up Multimodal Pre-training for Sign Language Understanding [96.17753464544604]
  Sign language serves as the primary meaning of communication for the deaf-mute community. To facilitate communication between the deaf-mute and hearing people, a series of sign language understanding (SLU) tasks have been studied. These tasks investigate sign language topics from diverse perspectives and raise challenges in learning effective representation of sign language videos.
  arXiv  Detail & Related papers  (2024-08-16T06:04:25Z)
• Sign Language Recognition Based On Facial Expression and Hand Skeleton [2.5879170041667523]
  We propose a sign language recognition network that integrates skeleton features of hands and facial expression. By incorporating facial expression information, the accuracy and robustness of sign language recognition are improved.
  arXiv  Detail & Related papers  (2024-07-02T13:02:51Z)
• Neural Sign Actors: A diffusion model for 3D sign language production from text [51.81647203840081]
  Sign Languages (SL) serve as the primary mode of communication for the Deaf and Hard of Hearing communities. This work makes an important step towards realistic neural sign avatars, bridging the communication gap between Deaf and hearing communities.
  arXiv  Detail & Related papers  (2023-12-05T12:04:34Z)
• Sign Languague Recognition without frame-sequencing constraints: A proof of concept on the Argentinian Sign Language [42.27617228521691]
  This paper presents a general probabilistic model for sign classification that combines sub-classifiers based on different types of features. The proposed model achieved an accuracy rate of 97% on an Argentinian Sign Language dataset.
  arXiv  Detail & Related papers  (2023-10-26T14:47:11Z)
• Self-Evolution Learning for Discriminative Language Model Pretraining [103.57103957631067]
  Self-Evolution learning (SE) is a simple and effective token masking and learning method. SE focuses on learning the informative yet under-explored tokens and adaptively regularizes the training by introducing a novel Token-specific Label Smoothing approach.
  arXiv  Detail & Related papers  (2023-05-24T16:00:54Z)
• CIAO! A Contrastive Adaptation Mechanism for Non-Universal Facial Expression Recognition [80.07590100872548]
  We propose Contrastive Inhibitory Adaptati On (CIAO), a mechanism that adapts the last layer of facial encoders to depict specific affective characteristics on different datasets. CIAO presents an improvement in facial expression recognition performance over six different datasets with very unique affective representations.
  arXiv  Detail & Related papers  (2022-08-10T15:46:05Z)
• Skeleton Based Sign Language Recognition Using Whole-body Keypoints [71.97020373520922]
  Sign language is used by deaf or speech impaired people to communicate. Skeleton-based recognition is becoming popular that it can be further ensembled with RGB-D based method to achieve state-of-the-art performance. Inspired by the recent development of whole-body pose estimation citejin 2020whole, we propose recognizing sign language based on the whole-body key points and features.
  arXiv  Detail & Related papers  (2021-03-16T03:38:17Z)
• Transferring Cross-domain Knowledge for Video Sign Language Recognition [103.9216648495958]
  Word-level sign language recognition (WSLR) is a fundamental task in sign language interpretation. We propose a novel method that learns domain-invariant visual concepts and fertilizes WSLR models by transferring knowledge of subtitled news sign to them.
  arXiv  Detail & Related papers  (2020-03-08T03:05:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.