Isharah: A Large-Scale Multi-Scene Dataset for Continuous Sign Language Recognition

• URL: http://arxiv.org/abs/2506.03615v1
• Date: Wed, 04 Jun 2025 06:48:21 GMT
• Title: Isharah: A Large-Scale Multi-Scene Dataset for Continuous Sign Language Recognition
• Authors: Sarah Alyami, Hamzah Luqman, Sadam Al-Azani, Maad Alowaifeer, Yazeed Alharbi, Yaser Alonaizan,
• Abstract summary: We present Isharah, a large multi-scene dataset for continuous SLR.<n>The dataset consists of 30,000 video clips performed by 18 deaf and professional signers.<n>This paper also introduces multiple sign language understanding benchmarks, including signer-independent and unseen-sentence CSLR.
• Score: 5.772467856207932
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Current benchmarks for sign language recognition (SLR) focus mainly on isolated SLR, while there are limited datasets for continuous SLR (CSLR), which recognizes sequences of signs in a video. Additionally, existing CSLR datasets are collected in controlled settings, which restricts their effectiveness in building robust real-world CSLR systems. To address these limitations, we present Isharah, a large multi-scene dataset for CSLR. It is the first dataset of its type and size that has been collected in an unconstrained environment using signers' smartphone cameras. This setup resulted in high variations of recording settings, camera distances, angles, and resolutions. This variation helps with developing sign language understanding models capable of handling the variability and complexity of real-world scenarios. The dataset consists of 30,000 video clips performed by 18 deaf and professional signers. Additionally, the dataset is linguistically rich as it provides a gloss-level annotation for all dataset's videos, making it useful for developing CSLR and sign language translation (SLT) systems. This paper also introduces multiple sign language understanding benchmarks, including signer-independent and unseen-sentence CSLR, along with gloss-based and gloss-free SLT. The Isharah dataset is available on https://snalyami.github.io/Isharah_CSLR/.

Related papers

• Logos as a Well-Tempered Pre-train for Sign Language Recognition [75.42794328290088]
  This paper presents Logos, a novel Russian Sign Language (RSL) dataset.<n>It is shown that a model, pre-trained on the Logos dataset can be used as a universal encoder for other language SLR tasks.<n>We show that explicitly labeling visually similar signs improves trained model quality as a visual encoder for downstream tasks.
  arXiv  Detail & Related papers  (2025-05-15T16:31:49Z)
• SSLR: A Semi-Supervised Learning Method for Isolated Sign Language Recognition [2.409285779772107]
  Sign language recognition systems aim to recognize sign gestures and translate them into spoken language.<n>One of the main challenges in SLR is the scarcity of annotated datasets.<n>We propose a semi-supervised learning approach for SLR, employing a pseudo-label method to annotate unlabeled samples.
  arXiv  Detail & Related papers  (2025-04-23T11:59:52Z)
• SCOPE: Sign Language Contextual Processing with Embedding from LLMs [49.5629738637893]
  Sign languages, used by around 70 million Deaf individuals globally, are visual languages that convey visual and contextual information. Current methods in vision-based sign language recognition ( SLR) and translation (SLT) struggle with dialogue scenes due to limited dataset diversity and the neglect of contextually relevant information. We introduce SCOPE, a novel context-aware vision-based SLR and SLT framework.
  arXiv  Detail & Related papers  (2024-09-02T08:56:12Z)
• A Tale of Two Languages: Large-Vocabulary Continuous Sign Language Recognition from Spoken Language Supervision [74.972172804514]
  We introduce a multi-task Transformer model, CSLR2, that is able to ingest a signing sequence and output in a joint embedding space between signed language and spoken language text. New dataset annotations provide continuous sign-level annotations for six hours of test videos, and will be made publicly available. Our model significantly outperforms the previous state of the art on both tasks.
  arXiv  Detail & Related papers  (2024-05-16T17:19:06Z)
• SignVTCL: Multi-Modal Continuous Sign Language Recognition Enhanced by Visual-Textual Contrastive Learning [51.800031281177105]
  SignVTCL is a continuous sign language recognition framework enhanced by visual-textual contrastive learning. It integrates multi-modal data (video, keypoints, and optical flow) simultaneously to train a unified visual backbone. It achieves state-of-the-art results compared with previous methods.
  arXiv  Detail & Related papers  (2024-01-22T11:04:55Z)
• Towards the extraction of robust sign embeddings for low resource sign language recognition [7.969704867355098]
  We show that keypoint-based embeddings can transfer between sign languages and achieve competitive performance. We furthermore achieve better performance using fine-tuned transferred embeddings than models trained only on the target sign language.
  arXiv  Detail & Related papers  (2023-06-30T11:21:40Z)
• Two-Stream Network for Sign Language Recognition and Translation [38.43767031555092]
  We introduce a dual visual encoder containing two separate streams to model both the raw videos and the keypoint sequences. The resulting model is called TwoStream- SLR, which is competent for sign language recognition. TwoStream-SLT is extended to a sign language translation model, TwoStream-SLT, by simply attaching an extra translation network.
  arXiv  Detail & Related papers  (2022-11-02T17:59:58Z)
• Signing Outside the Studio: Benchmarking Background Robustness for Continuous Sign Language Recognition [79.23777980180755]
  We propose a pipeline to automatically generate a benchmark dataset utilizing existing Continuous Sign Language Recognition benchmarks. Our newly constructed benchmark dataset consists of diverse scenes to simulate a real-world environment. In this regard, we also propose a simple yet effective training scheme including (1) background randomization and (2) feature disentanglement for CSLR models.
  arXiv  Detail & Related papers  (2022-11-01T13:27:44Z)
• Multi-Modal Zero-Shot Sign Language Recognition [51.07720650677784]
  We propose a multi-modal Zero-Shot Sign Language Recognition model. A Transformer-based model along with a C3D model is used for hand detection and deep features extraction. A semantic space is used to map the visual features to the lingual embedding of the class labels.
  arXiv  Detail & Related papers  (2021-09-02T09:10:39Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.