A Lightweight Dual-Mode Optimization for Generative Face Video Coding

• URL: http://arxiv.org/abs/2508.13547v1
• Date: Tue, 19 Aug 2025 06:09:28 GMT
• Title: A Lightweight Dual-Mode Optimization for Generative Face Video Coding
• Authors: Zihan Zhang, Shanzhi Yin, Bolin Chen, Ru-Ling Liao, Shiqi Wang, Yan Ye,
• Abstract summary: Generative Face Video Coding (GFVC) achieves superior rate-distortion performance by leveraging the strong inference capabilities of deep generative models.<n>We propose a lightweight GFVC framework that introduces dual-mode optimization to reduce complexity whilst preserving reconstruction quality.<n> Experimental results demonstrate that the proposed lightweight dual-mode optimization for GFVC can achieve 90.4% parameter reduction and 88.9% saving compared to the baseline.
• Score: 26.308480665852052
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Generative Face Video Coding (GFVC) achieves superior rate-distortion performance by leveraging the strong inference capabilities of deep generative models. However, its practical deployment is hindered by large model parameters and high computational costs. To address this, we propose a lightweight GFVC framework that introduces dual-mode optimization - combining architectural redesign and operational refinement - to reduce complexity whilst preserving reconstruction quality. Architecturally, we replace traditional 3 x 3 convolutions with slimmer and more efficient layers, reducing complexity without compromising feature expressiveness. Operationally, we develop a two-stage adaptive channel pruning strategy: (1) soft pruning during training identifies redundant channels via learnable thresholds, and (2) hard pruning permanently eliminates these channels post-training using a derived mask. This dual-phase approach ensures both training stability and inference efficiency. Experimental results demonstrate that the proposed lightweight dual-mode optimization for GFVC can achieve 90.4% parameter reduction and 88.9% computation saving compared to the baseline, whilst achieving superior performance compared to state-of-the-art video coding standard Versatile Video Coding (VVC) in terms of perceptual-level quality metrics. As such, the proposed method is expected to enable efficient GFVC deployment in resource-constrained environments such as mobile edge devices.

Related papers

• VAE-REPA: Variational Autoencoder Representation Alignment for Efficient Diffusion Training [53.09658039757408]
  This paper proposes textbfnamex, a lightweight intrinsic guidance framework for efficient diffusion training.<n>name aligns the intermediate latent features of diffusion transformers with VAE features via a lightweight projection layer, supervised by a feature alignment loss.<n>Experiments demonstrate that name improves both generation quality and training convergence speed compared to vanilla diffusion transformers.
  arXiv  Detail & Related papers  (2026-01-25T13:22:38Z)
• Content Adaptive based Motion Alignment Framework for Learned Video Compression [72.13599533975413]
  This paper proposes a content adaptive based motion alignment framework.<n>We first introduce a two-stage flow-guided deformable warping mechanism that refines motion compensation with coarse-to-fine offset prediction and mask modulation.<n>Second, we propose a multi-reference quality aware strategy that adjusts distortion weights based on reference quality, and applies it to hierarchical training to reduce error propagation.<n>Third, we integrate a training-free module that downsamples frames by motion magnitude and resolution to obtain smooth motion estimation.
  arXiv  Detail & Related papers  (2025-12-15T02:51:47Z)
• A Multi-Stage Optimization Framework for Deploying Learned Image Compression on FPGAs [7.577235739757108]
  Deep learning-based image compression (LIC) has achieved state-of-the-art rate-distortion (RD) performance, yet deploying these models on resource-constrained FPGAs remains a major challenge.<n>This work presents a complete, multi-stage optimization framework to bridge the gap between high-performance floating-point models and efficient, hardware-friendly integer-based implementations.
  arXiv  Detail & Related papers  (2025-11-21T10:55:44Z)
• Boosting Neural Video Representation via Online Structural Reparameterization [16.30945567392327]
  We propose Online-RepNeRV, a NVR framework based on online structural re parameterization.<n>Our method achieves an average PSNR gain of 0.37-2.7 dB over baseline methods, while maintaining comparable training time and decoding speed.
  arXiv  Detail & Related papers  (2025-11-14T08:44:31Z)
• Rethinking Autoregressive Models for Lossless Image Compression via Hierarchical Parallelism and Progressive Adaptation [75.58269386927076]
  Autoregressive (AR) models are often dismissed as impractical due to prohibitive computational cost.<n>This work re-thinks this paradigm, introducing a framework built on hierarchical parallelism and progressive adaptation.<n> Experiments on diverse datasets (natural, satellite, medical) validate that our method achieves new state-of-the-art compression.
  arXiv  Detail & Related papers  (2025-11-14T06:27:58Z)
• Boosting Fine-Grained Urban Flow Inference via Lightweight Architecture and Focalized Optimization [35.11698882937702]
  We propose a unified solution that synergizes architectural efficiency with adaptive optimization.<n>PLGF is a lightweight yet powerful architecture that employs a Progressive Local-Global Fusion strategy.<n>DualFocal Loss is a novel function that integrates dual-space supervision with a difficulty-aware focusing mechanism.
  arXiv  Detail & Related papers  (2025-11-10T13:38:26Z)
• VividFace: High-Quality and Efficient One-Step Diffusion For Video Face Enhancement [51.83206132052461]
  Video Face Enhancement (VFE) seeks to reconstruct high-quality facial regions from degraded video sequences.<n>Current methods that rely on video super-resolution and generative frameworks face three fundamental challenges.<n>We propose VividFace, a novel and efficient one-step diffusion framework for video face enhancement.
  arXiv  Detail & Related papers  (2025-09-28T02:39:48Z)
• SimpleGVR: A Simple Baseline for Latent-Cascaded Video Super-Resolution [55.14432034345353]
  We study key design principles for latter cascaded video super-resolution models, which are underexplored currently.<n>First, we propose two strategies to generate training pairs that better mimic the output characteristics of the base model, ensuring alignment between the VSR model and its upstream generator.<n>Second, we provide critical insights into VSR model behavior through systematic analysis of (1) timestep sampling strategies, (2) noise augmentation effects on low-resolution (LR) inputs.
  arXiv  Detail & Related papers  (2025-06-24T17:57:26Z)
• FCA2: Frame Compression-Aware Autoencoder for Modular and Fast Compressed Video Super-Resolution [68.77813885751308]
  State-of-the-art (SOTA) compressed video super-resolution (CVSR) models face persistent challenges, including prolonged inference time, complex training pipelines, and reliance on auxiliary information.<n>We propose an efficient and scalable solution inspired by the structural and statistical similarities between hyperspectral images (HSI) and video data.<n>Our approach introduces a compression-driven dimensionality reduction strategy that reduces computational complexity, accelerates inference, and enhances the extraction of temporal information across frames.
  arXiv  Detail & Related papers  (2025-06-13T07:59:52Z)
• FPSAttention: Training-Aware FP8 and Sparsity Co-Design for Fast Video Diffusion [44.206702976963676]
  We introduce FPSAttention, a novel training-aware co-design of FP8 quantization and sparsity for video generation.<n>Our approach features three key innovations: 1) A unified 3D tile-wise granularity that simultaneously supports both quantization and sparsity; 2) A denoising step-aware strategy that adapts to the noise schedule, addressing the strong correlation between quantization/sparsity errors and denoising steps; 3) A native, hardware-friendly kernel that leverages FlashAttention and is implemented with Hopper optimized architecture features for highly efficient execution.
  arXiv  Detail & Related papers  (2025-06-05T05:30:30Z)
• Turbo2K: Towards Ultra-Efficient and High-Quality 2K Video Synthesis [50.77548592888096]
  Demand for 2K video synthesis is rising with increasing consumer expectations for ultra-clear visuals.<n>Turbo2K is an efficient framework for generating detail-rich 2K videos.
  arXiv  Detail & Related papers  (2025-04-20T03:30:59Z)
• Ultra-Resolution Adaptation with Ease [62.56434979517156]
  We propose a set of key guidelines for ultra-resolution adaptation termed emphURAE.<n>We show that tuning minor components of the weight matrices outperforms widely-used low-rank adapters when synthetic data are unavailable.<n>Experiments validate that URAE achieves comparable 2K-generation performance to state-of-the-art closed-source models like FLUX1.1 [Pro] Ultra with only 3K samples and 2K iterations.
  arXiv  Detail & Related papers  (2025-03-20T16:44:43Z)
• One-Step Diffusion Model for Image Motion-Deblurring [85.76149042561507]
  We propose a one-step diffusion model for deblurring (OSDD), a novel framework that reduces the denoising process to a single step.<n>To tackle fidelity loss in diffusion models, we introduce an enhanced variational autoencoder (eVAE), which improves structural restoration.<n>Our method achieves strong performance on both full and no-reference metrics.
  arXiv  Detail & Related papers  (2025-03-09T09:39:57Z)
• Faster Image2Video Generation: A Closer Look at CLIP Image Embedding's Impact on Spatio-Temporal Cross-Attentions [27.111140222002653]
  This paper investigates the role of CLIP image embeddings within the Stable Video Diffusion (SVD) framework. We introduce the VCUT, a training-free approach optimized for efficiency within the SVD architecture. The implementation of VCUT leads to a reduction of up to 322T Multiple-Accumulate Operations (MACs) per video and a decrease in model parameters by up to 50M, achieving a 20% reduction in latency compared to the baseline.
  arXiv  Detail & Related papers  (2024-07-27T08:21:14Z)
• LADDER: An Efficient Framework for Video Frame Interpolation [12.039193291203492]
  Video Frame Interpolation (VFI) is a crucial technique in various applications such as slow-motion generation, frame rate conversion, video frame restoration etc. This paper introduces an efficient video frame framework that aims to strike a favorable balance between efficiency and quality.
  arXiv  Detail & Related papers  (2024-04-17T06:47:17Z)
• Channel-wise Feature Decorrelation for Enhanced Learned Image Compression [16.638869231028437]
  The emerging Learned Compression (LC) replaces the traditional modules with Deep Neural Networks (DNN), which are trained end-to-end for rate-distortion performance. This paper proposes to improve compression by fully exploiting the existing DNN capacity. Three strategies are proposed and evaluated, which optimize (1) the transformation network, (2) the context model, and (3) both networks.
  arXiv  Detail & Related papers  (2024-03-16T14:30:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.