FlashSign: Pose-Free Guidance for Efficient Sign Language Video Generation

• URL: http://arxiv.org/abs/2603.27915v1
• Date: Mon, 30 Mar 2026 00:06:26 GMT
• Title: FlashSign: Pose-Free Guidance for Efficient Sign Language Video Generation
• Abstract summary: We propose a pose-free framework for real-time sign language video generation.<n>Our method eliminates the need for intermediate pose representations by directly mapping natural language text to sign language.<n>Our contributions open new avenues for real-time, high-quality, pose-free sign language synthesis.
• Score: 39.618003889108685
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Sign language plays a crucial role in bridging communication gaps between the deaf and hard-of-hearing communities. However, existing sign language video generation models often rely on complex intermediate representations, which limits their flexibility and efficiency. In this work, we propose a novel pose-free framework for real-time sign language video generation. Our method eliminates the need for intermediate pose representations by directly mapping natural language text to sign language videos using a diffusion-based approach. We introduce two key innovations: (1) a pose-free generative model based on the a state-of-the-art diffusion backbone, which learns implicit text-to-gesture alignments without pose estimation, and (2) a Trainable Sliding Tile Attention (T-STA) mechanism that accelerates inference by exploiting spatio-temporal locality patterns. Unlike previous training-free sparsity approaches, T-STA integrates trainable sparsity into both training and inference, ensuring consistency and eliminating the train-test gap. This approach significantly reduces computational overhead while maintaining high generation quality, making real-time deployment feasible. Our method increases video generation speed by 3.07x without compromising video quality. Our contributions open new avenues for real-time, high-quality, pose-free sign language synthesis, with potential applications in inclusive communication tools for diverse communities. Code: https://github.com/AIGeeksGroup/FlashSign.

Related papers

• MIBURI: Towards Expressive Interactive Gesture Synthesis [62.45332399212876]
  Embodied Conversational Agents (ECAs) aim to emulate human face-to-face interaction through speech, gestures, and facial expressions.<n>Existing solutions for ECAs produce rigid, low-diversity motions that are unsuitable for human-like interaction.<n>We present MIBURI, the first online, causal framework for generating expressive full-body gestures and facial expressions synchronized with real-time spoken dialogue.
  arXiv  Detail & Related papers  (2026-03-03T18:59:51Z)
• Text2Sign Diffusion: A Generative Approach for Gloss-Free Sign Language Production [32.99299619724994]
  Sign language production aims to translate spoken language sentences into a sequence of pose frames in a sign language.<n>Existing methods rely on gloss, a symbolic representation of sign language words or phrases.<n>We present a novel diffusion-based generative approach - Text2Sign Diffusion (Text2SignDiff) for gloss-free SLP.
  arXiv  Detail & Related papers  (2025-09-13T15:05:19Z)
• Beyond Gloss: A Hand-Centric Framework for Gloss-Free Sign Language Translation [18.544615425772903]
  Sign Language Translation (SLT) is a challenging task that requires bridging the modality gap between visual and linguistic information.<n>We introduce textbfBeyondGloss, a novel gloss-free SLT framework that leverages thetemporal-aware reasoning capabilities of VideoLLMs.<n>Beyondtexttext achieves state-of-the-art performance on the PhoenixT14 and CSL-Daily benchmarks, demonstrating the effectiveness of the proposed framework.
  arXiv  Detail & Related papers  (2025-07-31T14:06:07Z)
• SignAligner: Harmonizing Complementary Pose Modalities for Coherent Sign Language Generation [41.240893601941536]
  We introduce ENIX14T+, an extended version of the widely-used RWTH-ENIXPHO-Weather 2014T dataset, featuring three new sign representations: Pose, Hamer and Smplerx.<n>We also propose a novel method, SignAligner, for realistic sign language generation, consisting of three stages: text-driven pose modalities co-generation, online collaborative correction of multimodality, and realistic sign video synthesis.
  arXiv  Detail & Related papers  (2025-06-13T09:44:42Z)
• MS2SL: Multimodal Spoken Data-Driven Continuous Sign Language Production [93.32354378820648]
  We propose a unified framework for continuous sign language production, easing communication between sign and non-sign language users. A sequence diffusion model, utilizing embeddings extracted from text or speech, is crafted to generate sign predictions step by step. Experiments on How2Sign and PHOENIX14T datasets demonstrate that our model achieves competitive performance in sign language production.
  arXiv  Detail & Related papers  (2024-07-04T13:53:50Z)
• Self-Supervised Representation Learning with Spatial-Temporal Consistency for Sign Language Recognition [96.62264528407863]
  We propose a self-supervised contrastive learning framework to excavate rich context via spatial-temporal consistency. Inspired by the complementary property of motion and joint modalities, we first introduce first-order motion information into sign language modeling. Our method is evaluated with extensive experiments on four public benchmarks, and achieves new state-of-the-art performance with a notable margin.
  arXiv  Detail & Related papers  (2024-06-15T04:50:19Z)
• Expedited Training of Visual Conditioned Language Generation via Redundancy Reduction [61.16125290912494]
  $textEVL_textGen$ is a framework designed for the pre-training of visually conditioned language generation models. We show that our approach accelerates the training of vision-language models by a factor of 5 without a noticeable impact on overall performance.
  arXiv  Detail & Related papers  (2023-10-05T03:40:06Z)
• EC^2: Emergent Communication for Embodied Control [72.99894347257268]
  Embodied control requires agents to leverage multi-modal pre-training to quickly learn how to act in new environments. We propose Emergent Communication for Embodied Control (EC2), a novel scheme to pre-train video-language representations for few-shot embodied control. EC2 is shown to consistently outperform previous contrastive learning methods for both videos and texts as task inputs.
  arXiv  Detail & Related papers  (2023-04-19T06:36:02Z)
• Pose-based Sign Language Recognition using GCN and BERT [0.0]
  Word-level sign language recognition (WSLR) is the first important step towards understanding and interpreting sign language. recognizing signs from videos is a challenging task as the meaning of a word depends on a combination of subtle body motions, hand configurations, and other movements. Recent pose-based architectures for W SLR either model both the spatial and temporal dependencies among the poses in different frames simultaneously or only model the temporal information without fully utilizing the spatial information. We tackle the problem of W SLR using a novel pose-based approach, which captures spatial and temporal information separately and performs late fusion.
  arXiv  Detail & Related papers  (2020-12-01T19:10:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.