Sign Language Video Synthesis via Loss-Guided Multi-Expert GANs

• URL: http://arxiv.org/abs/2608.13368v2
• Date: Mon, 17 Aug 2026 16:04:00 GMT
• Title: Sign Language Video Synthesis via Loss-Guided Multi-Expert GANs
• Abstract summary: This report presents a framework for sign language video synthesis using a loss-guided multi-expert Generative Adversarial Network (GAN)<n>Three specialized discriminators -- global, hand, and head -- each guide a corresponding expert branch in the generator toward a distinct visual region.<n>The system was showcased at the 2025 Hong Kong Frontier Technology Summit.
• Score: 5.455881292140741
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: This preliminary technical report presents a framework for sign language video synthesis using a loss-guided multi-expert Generative Adversarial Network (GAN) to enhance communication for individuals with hearing impairments. Three specialized discriminators -- global, hand, and head -- each guide a corresponding expert branch in the generator toward a distinct visual region, enabling implicit feature specialization without explicit diversity losses. To stabilize this multi-discriminator system, whose early-phase training otherwise exhibits chaotic dynamics, we introduce a United Loss consensus mechanism that regularizes each discriminator toward the ensemble average at a 10% weight. Each branch further adopts a dual-pathway convolutional-transformer design with learnable AdaptiveFeatureFusion, balancing the stability of convolutions against the detail of windowed self-attention. The generator is trained using an alternating three-mode schedule (discriminator, holistic generation, branch-specialized generation). On a custom 156GB dataset with a filtered test set that removes easy and repetitive samples, our 0.2B-parameter variant achieves 29.8 PSNR (0.959 SSIM) and the 1.3B-parameter variant achieves 30.7 PSNR (0.965 SSIM), with inference VRAM footprints of 1.5 GB and 8 GB respectively, enabling deployment on consumer-grade hardware. Full ablation studies remain ongoing due to the 2-3 month training cycle on a single GPU. The system was showcased at the 2025 Hong Kong Frontier Technology Summit.

Related papers

• Multi-Modal Semantic Expansion with Constrained LLM Reranking for Conversational Music Recommendation [2.8110777003480654]
  We present Team Semiintelligencn's solution for the ACM RecSys 2026 TalkPlayData Challenge.<n>Our submitted system employs a three-stage pipeline: (1) multi-modal retrieval constructing decay-weighted centroids across seven dense embedding spaces.<n>We optimize RRF weights on a 500-session development split via differential evolution, improving MRR by +19.5%.
  arXiv  Detail & Related papers  (2026-08-24T16:48:50Z)
• DiffusionGemma Technical Report [72.10570774996219]
  We introduce DiffusionGemma, a language model that uses discrete diffusion to generate text at exceptionally high speed.<n>Rather than decoding one token at a time, DiffusionGemma iteratively refines blocks of 256 tokens in parallel.<n>Our compute-efficient two-stage training pipeline uses fewer than 10% of the starting AR model's total training token budget.
  arXiv  Detail & Related papers  (2026-07-31T16:11:46Z)
• Scaling and Stabilizing Large-Scale Embedding-Based Retrieval [16.34687313227663]
  We present a unified pipeline deployed at Walmart that addresses both signal quality and model evolution.<n>We introduce a Warm-Start Distillation technique that transfers domain-specific expertise from the legacy model to the new backbone.<n>The proposed pipeline is deployed in live production, delivering a +7.34% improvement in NDCG@5 and a +0.50% lift in gross revenue.
  arXiv  Detail & Related papers  (2026-07-11T03:24:26Z)
• Spectrally Regularized Latent Flow Matching for Turbulence Generation [0.0]
  We introduce a latent flow matching framework with a spectrally regularized compression stage.<n>On a 2562 DNS dataset at Re_f approx 2250, replacing an MSE-trained VAE with a zone-weighted log-spectral objective raises deep-dissipation retained spectral power.<n>The improved latent representation also yields a substantially better sampling cost-fidelity tradeoff.
  arXiv  Detail & Related papers  (2026-06-10T06:09:51Z)
• KC-3DGS: Kurtosis-Constrained Gaussian Splatting for High-Fidelity View Synthesis [28.10448224660243]
  3D Gaussian Splatting (3DGS) enables real-time novel view synthesis by representing scenes as collections of anisotropic Gaussians optimized via differentiableization.<n>Standard pixel-space losses (L1, SSIM) constrain only aggregate reconstruction error, permitting the optimization to redistribute error across frequency scales.<n>We propose KC-3DGS, which augments 3DGS training with wavelet-domain supervision based on natural image statistics.
  arXiv  Detail & Related papers  (2026-06-02T04:04:24Z)
• Compact Latent Manifold Translation: A Parameter-Efficient Foundation Model for Cross-Modal and Cross-Frequency Physiological Signal Synthesis [4.672369796753925]
  Existing foundation models rely on continuous latent spaces, which frequently suffer from severe modality entanglement.<n>We propose Compact Latent Translation (CLMT), a highly parameter-efficient (0.09B) unified framework that bridges these gaps.<n>By learning a universal discrete language for biological signals with a fraction of the computational footprint, our approach sets a new trajectory for edge-deployable, multi-fidelity medical foundation models.
  arXiv  Detail & Related papers  (2026-05-13T09:31:36Z)
• A Unified Foundation Model for All-in-One Multi-Modal Remote Sensing Image Restoration and Fusion with Language Prompting [3.2569765796445034]
  We present Language-conditioned Large-scale Remote Sensing restoration model (LLaRS)<n>LLaRS is the first unified foundation model for multi-modal and multi-task remote sensing low-level vision.<n> Experiments show LLaRS consistently outperforms seven competitive models.
  arXiv  Detail & Related papers  (2026-04-07T09:33:42Z)
• DeepGen 1.0: A Lightweight Unified Multimodal Model for Advancing Image Generation and Editing [67.77471070868852]
  DeepGen 1.0 is a lightweight 5B unified model for image generation and editing.<n>It is trained on only 50M samples, surpassing the 80B HunyuanImage by 28% on WISE and the 27B Qwen-Image-Edit by 37% on UniREditBench.<n>By open-sourcing our training code, weights, and datasets, we provide an efficient, high-performance alternative to democratize unified multimodal research.
  arXiv  Detail & Related papers  (2026-02-12T17:44:24Z)
• SSVP: Synergistic Semantic-Visual Prompting for Industrial Zero-Shot Anomaly Detection [55.54007781679915]
  We propose Synergistic Semantic-Visual Prompting (SSVP), that efficiently fuses diverse visual encodings to elevate model's fine-grained perception.<n>SSVP achieves state-of-the-art performance with 93.0% Image-AUROC and 92.2% Pixel-AUROC on MVTec-AD, significantly outperforming existing zero-shot approaches.
  arXiv  Detail & Related papers  (2026-01-14T04:42:19Z)
• From Feature Interaction to Feature Generation: A Generative Paradigm of CTR Prediction Models [81.43473418572567]
  Click-Through Rate (CTR) prediction is a core task in recommendation systems.<n>We propose a novel generative framework to address embedding dimensional collapse and information redundancy.<n>We show that SFG consistently mitigates embedding collapse and reduces information redundancy, while yielding substantial performance gains.
  arXiv  Detail & Related papers  (2025-12-16T03:17:18Z)
• Conditioning Trick for Training Stable GANs [70.15099665710336]
  We propose a conditioning trick, called difference departure from normality, applied on the generator network in response to instability issues during GAN training. We force the generator to get closer to the departure from normality function of real samples computed in the spectral domain of Schur decomposition.
  arXiv  Detail & Related papers  (2020-10-12T16:50:22Z)
• When Relation Networks meet GANs: Relation GANs with Triplet Loss [110.7572918636599]
  Training stability is still a lingering concern of generative adversarial networks (GANs) In this paper, we explore a relation network architecture for the discriminator and design a triplet loss which performs better generalization and stability. Experiments on benchmark datasets show that the proposed relation discriminator and new loss can provide significant improvement on variable vision tasks.
  arXiv  Detail & Related papers  (2020-02-24T11:35:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.