Physics-Informed Diffusion for Biomechanically Plausible 3D Sign Language Generation

• URL: http://arxiv.org/abs/2607.14836v1
• Date: Thu, 16 Jul 2026 11:01:33 GMT
• Title: Physics-Informed Diffusion for Biomechanically Plausible 3D Sign Language Generation
• Abstract summary: PIDiffSign is a physics-informed diffusion model for gloss-to-pose translation.<n>It incorporates anatomical constraints into both the architecture and training objective.<n>Experiments show consistent improvements over a strong diffusion baseline in pose accuracy, joint-angle correctness, distributional realism, and back-translation quality.
• Score: 7.824804954393769
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Sign language production, which generates continuous 3D skeletal motion from spoken language input, must simultaneously satisfy two constraints: semantic fidelity, so that a deaf viewer can recognize the intended sequence of glosses, and biomechanical plausibility, so that the generated skeleton respects anatomical constraints. Existing approaches optimize semantic reconstruction through coordinate-based objectives that treat the skeleton as an unstructured vector, thus allowing for bone length drift, joint angle violations, and temporarily locked fingers. We introduce PIDiffSign, a physics-informed diffusion model for gloss-to-pose translation that incorporates anatomical constraints into both the architecture and training objective. The model uses a Transformer encoder-decoder, where the decoder is conditioned on the diffusion time step through adaptive zero-initialized layer normalization and cross-attends to gloss representations. A differentiable geometry module enforces bone length consistency and biologically valid joint angles throughout generation. Training combines anthropomorphic, kinematic, angular, and finger-joint constraints with a contrastive gloss-pose alignment loss and classifier-free guidance for semantically conditioned sampling. Experiments on the PHOENIX14T and CSL-Daily benchmarks show consistent improvements over a strong diffusion baseline in pose accuracy, joint-angle correctness, distributional realism, and back-translation quality. These results demonstrate that physics-informed diffusion improves both motion realism and semantic fidelity for sign language generation.

Related papers

• DeSeG: Decoupling Semantic Intent and Geometric Constraints for Physically Plausible Human-Scene Interaction [50.06148080975928]
  We propose DeSeG, a hierarchical framework that explicitly decouples semantic intent from geometric constraints.<n>DeSeG achieves state-of-the-art performance, reducing mean scene penetration by 47% and improving semantic alignment by 29% over the SOTA baselines.
  arXiv  Detail & Related papers  (2026-07-07T03:21:08Z)
• DeGenseGS: Geometrically and Semantically Decoupled Surgical Scene Understanding in 4D Gaussian Splatting [17.162560675768642]
  Existing solutions integrate Vision-Language Models into deformable fields via a rigid coupling scheme that tightly binds semantic features to geometric warping.<n>We propose DeGenseGS, Geometric Understandingally and Semantically Decoupled Scene in 4D Splatting, a novel framework that independently models semantic evolution and geometric deformation.<n>Our framework yields enhanced geometric robustness and robust semantic-anatomic alignment, enabling spatially continuous segmentation despite drastic tissue deformation and topological transitions.
  arXiv  Detail & Related papers  (2026-07-06T07:56:17Z)
• PhyScene3D: Physically Consistent Interactive 3D Tabletop Scene Generation [75.9028457575434]
  Generating 3D tabletop scenes is a fundamental yet underexplored problem for interactive and generalist robotic learning.<n>Here, we introduce PhyScene3D, a framework that reformulates generation as a Human-Mimetic Constructive Process.<n>Experiments demonstrate that PhyScene3D outperforms state-of-the-art approaches in both semantic accuracy and physical validity.
  arXiv  Detail & Related papers  (2026-06-01T03:56:32Z)
• MotionDreamer: Universal Skeletal Motion Generation for 3D Rigged Shapes [52.45449797329548]
  MotionDreamer is a diffusion-based framework for category-agnostic skeletal animation generation from 2D video guidance.<n>We propose a structural-semantic injection mechanism to bridge the gap between 2D visual motion cues and heterogeneous 3D skeletal structures.<n>Our approach significantly outperforms existing methods, setting a new state-of-the-art benchmark for robust and efficient 4D asset generation.
  arXiv  Detail & Related papers  (2026-06-01T00:42:31Z)
• LA-Sign: Looped Transformers with Geometry-aware Alignment for Skeleton-based Sign Language Recognition [47.469519895247366]
  LA-Sign is a looped transformer framework with geometry-aware alignment for sign language recognition.<n>We show that LA-Sign achieves state-of-the-art results while using fewer unique layers.
  arXiv  Detail & Related papers  (2026-03-30T22:49:42Z)
• Retrieval-Augmented Anatomical Guidance for Text-to-CT Generation [1.5532758127091075]
  We propose a retrieval-augmented approach for Text-to-CT generation that integrates semantic and anatomical information under a realistic inference setting.<n>Experiments on the CT-RATE dataset show that retrieval-augmented generation improves image fidelity and clinical consistency compared to text-only baselines.<n>This work introduces a principled and scalable mechanism to bridge semantic conditioning and anatomical plausibility in volumetric medical image synthesis.
  arXiv  Detail & Related papers  (2026-03-09T12:27:17Z)
• AffordGrasp: Cross-Modal Diffusion for Affordance-Aware Grasp Synthesis [50.793806818677716]
  AffordGrasp produces physically stable and semantically faithful human grasps with high precision.<n>AffordGrasp integrates an affordance-aware latent representation of hand poses with a dual-conditioning diffusion process.<n>We evaluate AffordGrasp across four instruction-augmented benchmarks derived from HO-3D, OakInk, GRAB, and AffordPose.
  arXiv  Detail & Related papers  (2026-03-09T06:56:35Z)
• StdGEN++: A Comprehensive System for Semantic-Decomposed 3D Character Generation [57.06461272772509]
  StdGEN++ is a novel and comprehensive system for generating high-fidelity, semantically decomposed 3D characters from diverse inputs.<n>It achieves state-of-the-art performance, significantly outperforming existing methods in geometric accuracy and semantic disentanglement.<n>The resulting structural independence unlocks advanced downstream capabilities, including non-destructive editing, physics-compliant animation, and gaze tracking.
  arXiv  Detail & Related papers  (2026-01-12T15:41:27Z)
• ArtiLatent: Realistic Articulated 3D Object Generation via Structured Latents [31.495577251319315]
  ArtiLatent is a generative framework that synthesizes human-made 3D objects with fine-grained geometry, accurate articulation, and realistic appearance.
  arXiv  Detail & Related papers  (2025-10-24T13:08:15Z)
• Towards Skeletal and Signer Noise Reduction in Sign Language Production via Quaternion-Based Pose Encoding and Contrastive Learning [7.740338361213371]
  We propose two enhancements to the standard Progressive Transformers (PT) architecture.<n>First, we encode poses using bone rotations in quaternion space and train with a geodesic loss to improve the accuracy and clarity of angular joint movements.<n>Second, we introduce a contrastive loss to structure decoder embeddings by semantic similarity, using either gloss overlap or SBERT-based sentence similarity.
  arXiv  Detail & Related papers  (2025-08-20T09:52:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.