Context-Aware Autoregressive Diffusion for Gloss-Wise Sign Language Production

• URL: http://arxiv.org/abs/2606.21234v1
• Date: Fri, 19 Jun 2026 08:57:45 GMT
• Title: Context-Aware Autoregressive Diffusion for Gloss-Wise Sign Language Production
• Abstract summary: Context-aware Gloss-wise AutoRegressive Diffusion model (GARD)<n>GARD is a gloss-wise diffusion framework that models coarticulation by conditioning on both semantic (linguistic) and kinematic (motion) contexts.<n> experiments on Phoenix-T and CSL-Daily datasets demonstrate that GARD achieves superior performance over existing SLP methods in terms of both linguistic accuracy and motion similarity.
• Score: 19.908237295817177
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: To generate natural and accurate sentence-level sign language, synthesizing the "gloss", the fundamental semantic unit, is essential. However, most current sign-language production (SLP) methods generate entire sequences at once. While this end-to-end approach is often efficient, it is prone to temporal drift and hand motion blur as sentences get longer, and fails to accurately control individual glosses. In this paper, we propose the Context-aware Gloss-wise AutoRegressive Diffusion model (GARD), a gloss-wise diffusion framework that models coarticulation by conditioning on both semantic (linguistic) and kinematic (motion) contexts. To ensure natural continuity between gloss motions, GARD introduces two additional strategies: i) Inter-Gloss Transition Guidance, which applies gradient-based guidance to kinematically align inter-gloss boundaries and ensure seamless pose consistency. ii) Global Motion Harmonizer, refining the entire gloss motion sequence based on the boundary poses adjusted by Inter-Gloss Transition Guidance. Extensive experiments on Phoenix-T and CSL-Daily datasets demonstrate that GARD achieves superior performance over existing SLP methods in terms of both linguistic accuracy and motion similarity.

Related papers

• ViPo-MLLM: Visual-Pose Multimodal LLM for Gloss-Free Sign Language Translation [23.358041920240208]
  Existing methods often use single-modality or weakly fused features, limiting performance.<n>We propose Vi-Po-MLLM, a framework that integrates intra-temporal and dynamics cross-modal attention.<n>The proposed model was evaluated on the ENIXENIX14T and CSL-Daily datasets and achieved new state-the-art results.
  arXiv  Detail & Related papers  (2026-07-04T01:17:48Z)
• SiGnature: Explicit Motion Diffusion for Stylized Semantic Gesture [31.872713283587604]
  SiGnature is a framework for Stylized and Semantic Gesture generation.<n>It reconciles precise semantic control with high-fidelity style preservation.<n>It offers superior semantic motion control while maintaining smooth and natural co-speech gesture generation.
  arXiv  Detail & Related papers  (2026-06-14T16:18:26Z)
• SignDPO: Multi-level Direct Preference Optimisation for Skeleton-based Gloss-free Sign Language Translation [52.752934028506274]
  We present SignDPO, a novel multi-level Direct Preference optimisation framework.<n>We show that SignDPO consistently outperforms state-of-the-art gloss-free methods.<n>Our results suggest that multi-level preference alignment is a powerful paradigm for bridging the gap between high-entropy skeletal trajectories and discrete linguistic semantics.
  arXiv  Detail & Related papers  (2026-04-20T09:59:18Z)
• MIBURI: Towards Expressive Interactive Gesture Synthesis [62.45332399212876]
  Embodied Conversational Agents (ECAs) aim to emulate human face-to-face interaction through speech, gestures, and facial expressions.<n>Existing solutions for ECAs produce rigid, low-diversity motions that are unsuitable for human-like interaction.<n>We present MIBURI, the first online, causal framework for generating expressive full-body gestures and facial expressions synchronized with real-time spoken dialogue.
  arXiv  Detail & Related papers  (2026-03-03T18:59:51Z)
• Semantic Routing: Exploring Multi-Layer LLM Feature Weighting for Diffusion Transformers [31.67315012315044]
  We introduce a unified normalized convex fusion framework equipped with lightweight gates to systematically organize multi-layer LLM hidden states.<n>Experiments establish Depth-wise Semantic Routing as the superior conditioning strategy.<n>We find that purely time-wise fusion can paradoxically degrade visual generation fidelity.
  arXiv  Detail & Related papers  (2026-02-03T13:30:13Z)
• AutoSign: Direct Pose-to-Text Translation for Continuous Sign Language Recognition [0.0]
  Continuously recognizing sign gestures and converting them to glosses plays a key role in bridging the gap between hearing and hearing-impaired communities.<n>We propose AutoSign, an autoregressive decoder-only transformer that directly translates pose sequences to natural language text.<n>By eliminating the multi-stage pipeline, AutoSign achieves substantial improvements on the Isharah-1000 dataset.
  arXiv  Detail & Related papers  (2025-07-26T07:28:33Z)
• GLOS: Sign Language Generation with Temporally Aligned Gloss-Level Conditioning [60.86278956347739]
  GLOS is a sign language generation framework with temporally aligned gloss-level conditioning.<n>Our method generates signs with correct lexical order and high semantic accuracy, outperforming prior methods on CSL-Daily and Phoenix-2014T.
  arXiv  Detail & Related papers  (2025-06-09T06:09:03Z)
• Multimodal LLM-Guided Semantic Correction in Text-to-Image Diffusion [52.315729095824906]
  MLLM Semantic-Corrected Ping-Pong-Ahead Diffusion (PPAD) is a novel framework that introduces a Multimodal Large Language Model (MLLM) as a semantic observer during inference.<n>It performs real-time analysis on intermediate generations, identifies latent semantic inconsistencies, and translates feedback into controllable signals that actively guide the remaining denoising steps.<n>Extensive experiments demonstrate PPAD's significant improvements.
  arXiv  Detail & Related papers  (2025-05-26T14:42:35Z)
• Bridging Sign and Spoken Languages: Pseudo Gloss Generation for Sign Language Translation [48.20483623444857]
  Sign Language Translation aims to map sign language videos to spoken language text.<n>A common approach relies on gloss annotations as an intermediate representation.<n>We propose a gloss-free pseudo gloss generation framework that eliminates the need for human-annotated glosses.
  arXiv  Detail & Related papers  (2025-05-21T12:19:55Z)
• Collaborative Temporal Consistency Learning for Point-supervised Natural Language Video Localization [129.43937834515688]
  We propose a new COllaborative Temporal consistEncy Learning (COTEL) framework to strengthen the video-language alignment.<n>Specifically, we first design a frame- and a segment-level Temporal Consistency Learning (TCL) module that models semantic alignment across frame saliencies and sentence-moment pairs.
  arXiv  Detail & Related papers  (2025-03-22T05:04:12Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.