Improving Visual Representation Alignment Generation with GRPO

• URL: http://arxiv.org/abs/2606.00583v1
• Date: Sat, 30 May 2026 07:21:40 GMT
• Title: Improving Visual Representation Alignment Generation with GRPO
• Abstract summary: diffusion transformers have demonstrated strong image synthesis capabilities but remain inefficient to train due to weak alignment.<n>We propose VRPO, a reinforcement-based optimization strategy that replaces REPA's static alignment loss with a generative representation policy optimization objective.<n>Our VRPO-driven training seamlessly integrates into diffusion transformers, introducing negligible cost and preserving full compatibility with SiT and DiT architectures.
• Score: 51.071351994330605
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Recent diffusion transformers have demonstrated strong image synthesis capabilities but remain inefficient to train due to weak alignment between generative and discriminative representations. While representation alignment frameworks such as REPA improve convergence by aligning noisy denoising features with pretrained visual encoders, their externally supervised alignment loss is static and lacks adaptivity during training and inference. Existing methods rely on fixed cosine alignment or contrastive objectives, which cannot dynamically balance representation consistency and generation quality, resulting in limited discriminative benefit and failing to optimize alignment in a task-adaptive manner. To address this, we propose VRPO, a reinforcement-based optimization strategy that replaces REPA's static alignment loss with a generative representation policy optimization objective. Instead of enforcing a fixed similarity constraint, VRPO treats representation alignment as a reward-guided process: the model receives adaptive rewards based on generation fidelity, perceptual quality, and semantic coherence between the diffusion features and pretrained visual embeddings. This formulation enables the generator to continuously refine its internal representations toward semantically meaningful directions while improving image quality. Our VRPO-driven training seamlessly integrates into diffusion transformers, introducing negligible computation cost and preserving full compatibility with SiT and DiT architectures. Extensive experiments on ImageNet-256x256 demonstrate that our VRPO-Alignment substantially enhances both convergence and fidelity, achieving up to +1.8 FID improvement and 2.3x faster training compared to REPA under identical compute budgets.

Related papers

• Balancing Emotional Alignment and Semantic Consistency in Image Generation via Reinforcement Learning with Valence-Arousal Anchoring [55.57242007016976]
  Continuous emotion control in text-to-image generation requires a model to improve affective alignment without changing the objects, layout, or scene described by the prompt.<n>Existing supervised emotion-injection methods often optimize feature-space proxies and may therefore exhibit emotion-semantic drift.<n>We address this problem with a flow-matching image-generation framework that combines continuous valence-arousal conditioning, Group Relative Policy Optimization, and a neutral semantic anchor.
  arXiv  Detail & Related papers  (2026-09-11T13:24:35Z)
• MoASE++: Mixture of Activation Sparsity Experts with Domain-Adaptive On-policy Distillation for Continual Test Time Adaptation [65.068801413044]
  Continual test-time adaptation adapts a source-pretrained model to non-stationary, unlabeled target streams.<n>We introduce MoASE, a mixture-of-experts that disentangles domain-agnostic structure from domain-specific texture.
  arXiv  Detail & Related papers  (2026-05-18T01:52:12Z)
• Beyond Point-Wise Matching: Structural Representation Alignment for Accelerating Diffusion Transformers [93.3976834364707]
  We propose sREPA, a structural REPresentation Alignment framework to enforce consistency in the relational geometry of feature maps.<n>By encouraging the model to internalize holistic spatial layouts and structural correlations from pre-trained features, sREPA achieves faster and more stable convergence.
  arXiv  Detail & Related papers  (2026-05-16T12:01:04Z)
• AHPA: Adaptive Hierarchical Prior Alignment for Diffusion Transformers [24.937985157569823]
  We argue that such timestep-agnostic alignment is suboptimal because the useful granularity of representation supervision changes systematically with the signal-to-noise ratio.<n>In high-noise regimes, diffusion models benefit more from coarse semantic and layout-level anchoring, whereas in low-noise regimes, the training signal should emphasize spatially detailed and structurally faithful refinement.<n>This non-stationary alignment behavior creates a representational mismatch for static single-level supervisors.
  arXiv  Detail & Related papers  (2026-05-05T03:07:29Z)
• Bidirectional Reward-Guided Diffusion for Real-World Image Super-Resolution [79.35296000454694]
  Diffusion-based super-resolution can synthesize rich details, but models trained on synthetic paired data often fail on real-world LR images.<n>We propose Bird-SR, a reward-guided diffusion framework that formulates super-resolution as trajectory-level preference optimization.<n>Experiments on real-world SR benchmarks demonstrate that Bird-SR consistently outperforms state-of-the-art methods in perceptual quality.
  arXiv  Detail & Related papers  (2026-02-05T19:21:45Z)
• Training-Free Representation Guidance for Diffusion Models with a Representation Alignment Projector [14.027059904924135]
  We introduce a representation alignment projector that injects representations predicted by a projector into intermediate sampling steps.<n>Experiments on SiTs and REPAs show notable improvements in class-conditional ImageNet synthesis.<n>The proposed method outperforms representative guidance when applied to SiT models.
  arXiv  Detail & Related papers  (2026-01-30T02:29:54Z)
• A Constrained Optimization Perspective of Unrolled Transformers [77.12297732942095]
  We introduce a constrained optimization framework for training transformers that behave like optimization descent algorithms.<n>We observe constrained transformers achieve stronger to perturbations robustness and maintain higher out-of-distribution generalization.
  arXiv  Detail & Related papers  (2026-01-24T02:12:39Z)
• Align & Invert: Solving Inverse Problems with Diffusion and Flow-based Models via Representational Alignment [13.028121107802127]
  In inverse problems, pretrained generative models are employed as priors.<n>We propose applying representation alignment (REPA) between diffusion or flow-based models and a pretrained self-supervised visual encoder.<n>We show that aligning model representations with approximate target features can substantially enhance reconstruction fidelity and perceptual realism.
  arXiv  Detail & Related papers  (2025-11-21T00:37:04Z)
• SARA: Structural and Adversarial Representation Alignment for Training-efficient Diffusion Models [12.26595705520937]
  We introduce SARA, a hierarchical alignment framework that enforces multi-level representation constraints.<n>Experiments on ImageNet-256 show that SARA achieves an FID of 1.36 while converging twice as fast as REPA, surpassing recent state-of-the-art image generation methods.
  arXiv  Detail & Related papers  (2025-03-11T10:17:32Z)
• Efficient Real-world Image Super-Resolution Via Adaptive Directional Gradient Convolution [80.85121353651554]
  We introduce kernel-wise differential operations within the convolutional kernel and develop several learnable directional gradient convolutions. These convolutions are integrated in parallel with a novel linear weighting mechanism to form an Adaptive Directional Gradient Convolution (DGConv) We further devise an Adaptive Information Interaction Block (AIIBlock) to adeptly balance the enhancement of texture and contrast while meticulously investigating the interdependencies, culminating in the creation of a DGPNet for Real-SR through simple stacking.
  arXiv  Detail & Related papers  (2024-05-11T14:21:40Z)
• Group Equivariant Generative Adversarial Networks [7.734726150561089]
  In this work, we explicitly incorporate inductive symmetry priors into the network architectures via group-equivariant convolutional networks. Group-convariants have higher expressive power with fewer samples and lead to better gradient feedback between generator and discriminator.
  arXiv  Detail & Related papers  (2020-05-04T17:38:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.