LibraGen: Playing a Balance Game in Subject-Driven Video Generation

• URL: http://arxiv.org/abs/2603.13506v2
• Date: Tue, 17 Mar 2026 07:01:19 GMT
• Title: LibraGen: Playing a Balance Game in Subject-Driven Video Generation
• Abstract summary: We propose a novel framework that views extending foundation models for S2V generation as a balance game between intrinsic VGFM strengths and S2V capability.<n>We construct a hybrid pipeline that combines automated and manual data filtering to improve overall data quality.<n> Experimental results demonstrate that LibraGen outperforms both open-source and commercial S2V models using only thousand-scale training data.
• Score: 49.4880360924921
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: With the advancement of video generation foundation models (VGFMs), customized generation, particularly subject-to-video (S2V), has attracted growing attention. However, a key challenge lies in balancing the intrinsic priors of a VGFM, such as motion coherence, visual aesthetics, and prompt alignment, with its newly derived S2V capability. Existing methods often neglect this balance by enhancing one aspect at the expense of others. To address this, we propose LibraGen, a novel framework that views extending foundation models for S2V generation as a balance game between intrinsic VGFM strengths and S2V capability. Specifically, guided by the core philosophy of "Raising the Fulcrum, Tuning to Balance," we identify data quality as the fulcrum and advocate a quality-over-quantity approach. We construct a hybrid pipeline that combines automated and manual data filtering to improve overall data quality. To further harmonize the VGFM's native capabilities with its S2V extension, we introduce a Tune-to-Balance post-training paradigm. During supervised fine-tuning, both cross-pair and in-pair data are incorporated, and model merging is employed to achieve an effective trade-off. Subsequently, two tailored direct preference optimization (DPO) pipelines, namely Consis-DPO and Real-Fake DPO, are designed and merged to consolidate this balance. During inference, we introduce a time-dependent dynamic classifier-free guidance scheme to enable flexible and fine-grained control. Experimental results demonstrate that LibraGen outperforms both open-source and commercial S2V models using only thousand-scale training data.

Related papers

• GFPL: Generative Federated Prototype Learning for Resource-Constrained and Data-Imbalanced Vision Task [43.723840781330914]
  Federated learning (FL) facilitates the secure utilization of decentralized images.<n>FL faces two critical challenges in real-world deployment: ineffective knowledge fusion and prohibitive communication overhead.<n>We propose a novel Generative Federated Prototype Learning framework to address these issues.
  arXiv  Detail & Related papers  (2026-02-25T12:57:45Z)
• DeepGen 1.0: A Lightweight Unified Multimodal Model for Advancing Image Generation and Editing [67.77471070868852]
  DeepGen 1.0 is a lightweight 5B unified model for image generation and editing.<n>It is trained on only 50M samples, surpassing the 80B HunyuanImage by 28% on WISE and the 27B Qwen-Image-Edit by 37% on UniREditBench.<n>By open-sourcing our training code, weights, and datasets, we provide an efficient, high-performance alternative to democratize unified multimodal research.
  arXiv  Detail & Related papers  (2026-02-12T17:44:24Z)
• DiffusionDriveV2: Reinforcement Learning-Constrained Truncated Diffusion Modeling in End-to-End Autonomous Driving [65.7087560656003]
  Generative diffusion models for end-to-end autonomous driving often suffer from mode collapse.<n>We propose DiffusionDriveV2, which leverages reinforcement learning to constrain low-quality modes and explore for superior trajectories.<n>This significantly enhances the overall output quality while preserving the inherent multimodality of its core Gaussian Mixture Model.
  arXiv  Detail & Related papers  (2025-12-08T17:29:52Z)
• Dual-Stream Diffusion for World-Model Augmented Vision-Language-Action Model [62.889356203346985]
  We propose DUal-STream diffusion (DUST), a world-model augmented VLA framework that handles the modality conflict.<n>DUST achieves up to 6% gains over a standard VLA baseline and implicit world-modeling methods.<n>On real-world tasks with the Franka Research 3, DUST outperforms baselines in success rate by 13%.
  arXiv  Detail & Related papers  (2025-10-31T16:32:12Z)
• Adaptive Divergence Regularized Policy Optimization for Fine-tuning Generative Models [31.470613363668672]
  Adaptive Divergence Regularized Policy Optimization automatically adjusts regularization strength based on advantage estimates.<n>Our implementation with Wasserstein-2 regularization for flow matching generative models achieves remarkable results on text-to-image generation.<n> ADRPO generalizes to KL-regularized fine-tuning of both text-only LLMs and multi-modal reasoning models.
  arXiv  Detail & Related papers  (2025-10-20T19:46:02Z)
• Learning from Heterogeneity: Generalizing Dynamic Facial Expression Recognition via Distributionally Robust Optimization [23.328511708942045]
  Heterogeneity-aware Distributional Framework (HDF) designed to enhance time-frequency modeling and mitigate imbalance caused by hard samples.<n>Time-Frequency Distributional Attention Module (DAM) captures both temporal consistency and frequency robustness.<n> adaptive optimization module Distribution-aware Scaling Module (DSM) introduced to dynamically balance classification and contrastive losses.
  arXiv  Detail & Related papers  (2025-07-21T16:21:47Z)
• DanceGRPO: Unleashing GRPO on Visual Generation [42.567425922760144]
  Reinforcement Learning (RL) has emerged as a promising approach for fine-tuning generative models.<n>Existing methods like DDPO and DPOK face fundamental limitations when scaling to large and diverse prompt sets.<n>This paper presents DanceGRPO, a framework that addresses these limitations through an innovative adaptation of Group Relative Policy Optimization.
  arXiv  Detail & Related papers  (2025-05-12T17:59:34Z)
• NVComposer: Boosting Generative Novel View Synthesis with Multiple Sparse and Unposed Images [50.36605863731669]
  NVComposer is a novel approach that eliminates the need for explicit external alignment.<n> NVComposer achieves state-of-the-art performance in generative multi-view NVS tasks.<n>Our approach shows substantial improvements in synthesis quality as the number of unposed input views increases.
  arXiv  Detail & Related papers  (2024-12-04T17:58:03Z)
• T2V-Turbo-v2: Enhancing Video Generation Model Post-Training through Data, Reward, and Conditional Guidance Design [84.03286690283747]
  Our proposed method, T2V-Turbo-v2, introduces a significant advancement by integrating various supervision signals.<n>We highlight the crucial importance of tailoring datasets to specific learning objectives.<n>We demonstrate the potential of this approach by extracting motion guidance from the training datasets and incorporating it into the ODE solver.
  arXiv  Detail & Related papers  (2024-10-08T04:30:06Z)
• iiANET: Inception Inspired Attention Hybrid Network for efficient Long-Range Dependency [0.5497663232622965]
  We present iiANET, an efficient hybrid visual backbone designed to improve the modeling of long-range dependen-cies.<n>The core innovation of iiANET is the iiABlock, a unified building block that in-tegrates global r-MHSA (Multi-Head Self-Attention) and convolutional layers in paral-lel.
  arXiv  Detail & Related papers  (2024-07-10T12:39:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.