Enhancing Mathematics Learning for Hard-of-Hearing Students Through Real-Time Palestinian Sign Language Recognition: A New Dataset

• URL: http://arxiv.org/abs/2505.17055v1
• Date: Fri, 16 May 2025 19:14:25 GMT
• Title: Enhancing Mathematics Learning for Hard-of-Hearing Students Through Real-Time Palestinian Sign Language Recognition: A New Dataset
• Authors: Fidaa khandaqji, Huthaifa I. Ashqar, Abdelrahem Atawnih,
• Abstract summary: The study aims to enhance mathematics education accessibility for hard-of-hearing students by developing an accurate Palestinian sign language PSL recognition system.<n>To leverage state-of-the-art-computer vision techniques, a Vision Transformer ViTModel was fine-tuned for gesture classification.<n>The model achieved an accuracy of 97.59%, demonstrating its effectiveness in recognizing mathematical signs with high precision and reliability.
• Score: 1.1137087573421256
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The study aims to enhance mathematics education accessibility for hard-of-hearing students by developing an accurate Palestinian sign language PSL recognition system using advanced artificial intelligence techniques. Due to the scarcity of digital resources for PSL, a custom dataset comprising 41 mathematical gesture classes was created, and recorded by PSL experts to ensure linguistic accuracy and domain specificity. To leverage state-of-the-art-computer vision techniques, a Vision Transformer ViTModel was fine-tuned for gesture classification. The model achieved an accuracy of 97.59%, demonstrating its effectiveness in recognizing mathematical signs with high precision and reliability. This study highlights the role of deep learning in developing intelligent educational tools that bridge the learning gap for hard-of-hearing students by providing AI-driven interactive solutions to enhance mathematical comprehension. This work represents a significant step toward innovative and inclusive frosting digital integration in specialized learning environments. The dataset is hosted on Hugging Face at https://huggingface.co/datasets/fidaakh/STEM_data.

Related papers

• Indian Sign Language Detection for Real-Time Translation using Machine Learning [0.1747623282473278]
  We propose a robust, real-time ISL detection & translation system built upon a Convolutional Neural Network (CNN)<n>Our model is trained on a comprehensive ISL dataset & demonstrates exceptional performance, achieving a classification accuracy of 99.95%.<n>For real-time implementation, the framework integrates MediaPipe for precise hand tracking & motion detection, enabling seamless translation of dynamic gestures.
  arXiv  Detail & Related papers  (2025-07-27T21:15:46Z)
• Deep Learning and Machine Learning -- Natural Language Processing: From Theory to Application [17.367710635990083]
  We focus on natural language processing (NLP) and the role of large language models (LLMs)<n>This paper discusses advanced data preprocessing techniques and the use of frameworks like Hugging Face for implementing transformer-based models.<n>It highlights challenges such as handling multilingual data, reducing bias, and ensuring model robustness.
  arXiv  Detail & Related papers  (2024-10-30T09:35:35Z)
• Deep Neural Network-Based Sign Language Recognition: A Comprehensive Approach Using Transfer Learning with Explainability [0.0]
  We suggest a novel solution that uses a deep neural network to fully automate sign language recognition. This methodology integrates sophisticated preprocessing methodologies to optimise the overall performance. Our model's ability to provide informational clarity was assessed using the SHAP (SHapley Additive exPlanations) method.
  arXiv  Detail & Related papers  (2024-09-11T17:17:44Z)
• Sign language recognition based on deep learning and low-cost handcrafted descriptors [0.0]
  It is important to consider as many linguistic parameters as possible in gesture execution to avoid ambiguity between words. It is essential to ensure that the chosen technology is realistic, avoiding expensive, intrusive, or low-mobility sensors. We propose an efficient sign language recognition system that utilizes low-cost sensors and techniques.
  arXiv  Detail & Related papers  (2024-08-14T00:56:51Z)
• Self-Supervised Representation Learning with Spatial-Temporal Consistency for Sign Language Recognition [96.62264528407863]
  We propose a self-supervised contrastive learning framework to excavate rich context via spatial-temporal consistency. Inspired by the complementary property of motion and joint modalities, we first introduce first-order motion information into sign language modeling. Our method is evaluated with extensive experiments on four public benchmarks, and achieves new state-of-the-art performance with a notable margin.
  arXiv  Detail & Related papers  (2024-06-15T04:50:19Z)
• Enhancing Deep Knowledge Tracing via Diffusion Models for Personalized Adaptive Learning [1.2248793682283963]
  This study aims to tackle data shortage issues in student learning records to enhance DKT performance for personalized adaptive learning (PAL) It employs TabDDPM, a diffusion model, to generate synthetic educational records to augment training data for enhancing DKT. The experimental results demonstrate that the AI-generated data by TabDDPM significantly improves DKT performance.
  arXiv  Detail & Related papers  (2024-04-25T00:23:20Z)
• SignVTCL: Multi-Modal Continuous Sign Language Recognition Enhanced by Visual-Textual Contrastive Learning [51.800031281177105]
  SignVTCL is a continuous sign language recognition framework enhanced by visual-textual contrastive learning. It integrates multi-modal data (video, keypoints, and optical flow) simultaneously to train a unified visual backbone. It achieves state-of-the-art results compared with previous methods.
  arXiv  Detail & Related papers  (2024-01-22T11:04:55Z)
• Recognizing Unseen Objects via Multimodal Intensive Knowledge Graph Propagation [68.13453771001522]
  We propose a multimodal intensive ZSL framework that matches regions of images with corresponding semantic embeddings. We conduct extensive experiments and evaluate our model on large-scale real-world data.
  arXiv  Detail & Related papers  (2023-06-14T13:07:48Z)
• VidLanKD: Improving Language Understanding via Video-Distilled Knowledge Transfer [76.3906723777229]
  We present VidLanKD, a video-language knowledge distillation method for improving language understanding. We train a multi-modal teacher model on a video-text dataset, and then transfer its knowledge to a student language model with a text dataset. In our experiments, VidLanKD achieves consistent improvements over text-only language models and vokenization models.
  arXiv  Detail & Related papers  (2021-07-06T15:41:32Z)
• Skeleton Based Sign Language Recognition Using Whole-body Keypoints [71.97020373520922]
  Sign language is used by deaf or speech impaired people to communicate. Skeleton-based recognition is becoming popular that it can be further ensembled with RGB-D based method to achieve state-of-the-art performance. Inspired by the recent development of whole-body pose estimation citejin 2020whole, we propose recognizing sign language based on the whole-body key points and features.
  arXiv  Detail & Related papers  (2021-03-16T03:38:17Z)
• Synbols: Probing Learning Algorithms with Synthetic Datasets [112.45883250213272]
  Synbols is a tool for rapidly generating new datasets with a rich composition of latent features rendered in low resolution images. Our tool's high-level interface provides a language for rapidly generating new distributions on the latent features. To showcase the versatility of Synbols, we use it to dissect the limitations and flaws in standard learning algorithms in various learning setups.
  arXiv  Detail & Related papers  (2020-09-14T13:03:27Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.