RAIDAL: Redundancy-Aware Information Density Active Learning for CTC-Based Continuous Sign Language Recognition

• URL: http://arxiv.org/abs/2609.06843v2
• Date: Wed, 09 Sep 2026 22:38:29 GMT
• Title: RAIDAL: Redundancy-Aware Information Density Active Learning for CTC-Based Continuous Sign Language Recognition
• Abstract summary: We introduce RAIDAL, which repurposes the CTC decoder to restrict representation-based scoring to decoder-aligned gloss regions.<n> RAIDAL achieves its strongest data-efficiency gains over competing baselines in largevocabulary, budget-limited settings.
• Score: 7.186818726913181
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Continuous sign language recognition (CSLR) is a key technology for accessibility, yet its development remains limited by the high cost of annotating continuous video streams. Active learning offers a path toward mitigating this cost, but standard acquisition functions are not designed for weakly aligned sign language videos, where sign executions are interleaved with rest poses, irregular pauses, sign-like motion, and temporally redundant frames. This temporal redundancy can undermine sample selection, as acquisition scores may be influenced by timesteps from regions that are not associated with the decoded gloss sequence, distorting the video's estimated informativeness. In this work, we show that modern CSLR models already contain a mechanism for identifying gloss-level temporal evidence: the CTC decoder. Although typically used only during inference, its alignment peaks indicate where the model localizes each predicted gloss in the feature sequence, providing a source of temporal structure for active learning acquisition functions at zero additional labeling cost. Thus, we introduce RAIDAL (Redundancy-Aware Information Density Active Learning), which repurposes the CTC decoder to restrict representation-based scoring to decoder-aligned gloss regions, rather than exposing the acquisition function to the entire unfiltered video. Across three datasets and two architectures, RAIDAL achieves its strongest data-efficiency gains over competing baselines in large-vocabulary, budget-limited settings, while remaining competitive in the smaller-vocabulary, large-budget setting. The code used in this work is publicly available at github.com/verlab/RAIDAL.

Related papers

• SMART: MLLM-guided Temporal Alignment for Unifying Sign Language Recognition and Spotting [1.3287545480967504]
  We propose an MLLM-guided temporal alignment framework for joint sign recognition and spotting.<n> SMART uses motion descriptions as auxiliary semantic cues and performs stable videotext alignment under small-batch training.<n>Experiments on four sign language benchmarks demonstrate the effectiveness of SMART across both recognition and spotting tasks.
  arXiv  Detail & Related papers  (2026-08-26T08:06:10Z)
• LiteFrame: Efficient Vision Encoders Unlock Frame Scaling in Video LLMs [90.77662862634509]
  LiteFrame is a strong, yet highly efficient backbone for Video Large Language Models.<n>LiteFrame provides a 35% reduction in end-to-end latency while processing 8$times$ more frames.<n>Our results demonstrate a potential path to unlocking longer-form video understanding under fixed compute budgets.
  arXiv  Detail & Related papers  (2026-05-17T05:02:52Z)
• FRAME: Pre-Training Video Feature Representations via Anticipation and Memory [55.046881477209695]
  FRAME is a self-supervised video frame encoder tailored for dense video understanding.<n>It learns to predict current and future DINO patch features from past and present RGB frames.<n>It consistently outperforms image encoders and existing self-supervised video models.
  arXiv  Detail & Related papers  (2025-06-05T19:44:47Z)
• STORM: Token-Efficient Long Video Understanding for Multimodal LLMs [116.4479155699528]
  STORM is a novel architecture incorporating a dedicated temporal encoder between the image encoder and the Video-LLMs.<n>We show that STORM achieves state-of-the-art results across various long video understanding benchmarks.
  arXiv  Detail & Related papers  (2025-03-06T06:17:38Z)
• DRCap: Decoding CLAP Latents with Retrieval-Augmented Generation for Zero-shot Audio Captioning [13.601154787754046]
  DRCap is a data-efficient and flexible zero-shot audio captioning system.<n>It requires text-only data for training and can quickly adapt to new domains without additional fine-tuning.
  arXiv  Detail & Related papers  (2024-10-12T10:21:00Z)
• SEDS: Semantically Enhanced Dual-Stream Encoder for Sign Language Retrieval [82.51117533271517]
  Previous works typically only encode RGB videos to obtain high-level semantic features. Existing RGB-based sign retrieval works suffer from the huge memory cost of dense visual data embedding in end-to-end training. We propose a novel sign language representation framework called Semantically Enhanced Dual-Stream.
  arXiv  Detail & Related papers  (2024-07-23T11:31:11Z)
• Continuous Sign Language Recognition Using Intra-inter Gloss Attention [0.0]
  In this study, we introduce a novel module in sign language recognition studies, called intra-inter gloss attention module. In the intra-gloss attention module, the video is divided into equally sized chunks and a self-attention mechanism is applied within each chunk. Experimental results on the PHOENIX-2014 benchmark dataset demonstrate that our method can effectively extract sign language features in an end-to-end manner.
  arXiv  Detail & Related papers  (2024-06-26T13:21:08Z)
• Leveraging Temporal Contextualization for Video Action Recognition [47.8361303269338]
  We propose a framework for video understanding called Temporally Contextualized CLIP (TC-CLIP) We introduce Temporal Contextualization (TC), a layer-wise temporal information infusion mechanism for videos. The Video-Prompting (VP) module processes context tokens to generate informative prompts in the text modality.
  arXiv  Detail & Related papers  (2024-04-15T06:24:56Z)
• SignVTCL: Multi-Modal Continuous Sign Language Recognition Enhanced by Visual-Textual Contrastive Learning [51.800031281177105]
  SignVTCL is a continuous sign language recognition framework enhanced by visual-textual contrastive learning. It integrates multi-modal data (video, keypoints, and optical flow) simultaneously to train a unified visual backbone. It achieves state-of-the-art results compared with previous methods.
  arXiv  Detail & Related papers  (2024-01-22T11:04:55Z)
• TVTSv2: Learning Out-of-the-box Spatiotemporal Visual Representations at Scale [59.01246141215051]
  We analyze the factor that leads to degradation from the perspective of language supervision. We propose a tunable-free pre-training strategy to retain the generalization ability of the text encoder. We produce a series of models, dubbed TVTSv2, with up to one billion parameters.
  arXiv  Detail & Related papers  (2023-05-23T15:44:56Z)
• Variational Stacked Local Attention Networks for Diverse Video Captioning [2.492343817244558]
  Variational Stacked Local Attention Network exploits low-rank bilinear pooling for self-attentive feature interaction. We evaluate VSLAN on MSVD and MSR-VTT datasets in terms of syntax and diversity.
  arXiv  Detail & Related papers  (2022-01-04T05:14:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.