DivRL: Disentangled Self-Similarity Rewards for Diverse Subject-Driven Generation

• URL: http://arxiv.org/abs/2606.23950v1
• Date: Mon, 22 Jun 2026 21:22:42 GMT
• Title: DivRL: Disentangled Self-Similarity Rewards for Diverse Subject-Driven Generation
• Abstract summary: We propose a post-training framework called DivRL that jointly optimize identity consistency and structural diversity simultaneously.<n>We introduce a Negative Self-Similarity Measure (nSSM) to quantify structural diversity, and Visual Semantic Matching (VSM) to evaluate identity consistency.
• Score: 52.23323703015865
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Subject-driven image generation faces an "Identity-Diversity Paradox", where strong identity preservation often leads to rigid and low-diversity outputs. We propose a post-training framework called DivRL that jointly optimizes identity consistency and structural diversity simultaneously by leveraging disentangled visual features from a robust similarity model. Specifically, we introduce a Negative Self-Similarity Measure (nSSM) to quantify structural diversity, and Visual Semantic Matching (VSM) to evaluate identity consistency. We propose an "Explore-and-Suppress" strategy that treats VSM as a gated constraint: the model freely explores structurally diverse configurations, and only samples that violate the identity threshold are penalized via a quadratic hinge loss. This converts identity preservation from a competing objective into a feasibility constraint, allowing nSSM and VSM to improve jointly. Experiments demonstrate that our method effectively pushes the model to generate both consistent and diverse images and improves structural diversity while maintaining comparable identity consistency through a gated optimization formulation.

Related papers

• Identity-Decoupled Anonymization for Visual Evidence in Multi-modal Retrieval-Augmented Generation [9.39195684989942]
  Identity-Decoupled MRAG is a framework that interposes a generative anonymization module between retrieval and generation.<n>Our approach consists of three components: (i)a disentangled variational encoder that factorizes each face into an identity code and a spatially-structured attribute code, regularized by a mutual-information penalty and a gradient-based independence term; (ii)a manifold-aware rejection sampler that replaces the identity code with a synthetic one guaranteed to be both distinct from the original and realistic; and (iii)a conditional latent diffusion generator that synthesizes the anonymized face from the replacement identity and the
  arXiv  Detail & Related papers  (2026-04-26T07:42:33Z)
• AnyCrowd: Instance-Isolated Identity-Pose Binding for Arbitrary Multi-Character Animation [55.94507360511886]
  We propose AnyCrowd, a Transformer-based video generation framework capable of scaling to an arbitrary number of characters.<n>Specifically, we first introduce an Instance-Isolated Latent Representation (IILR), which encodes character instances independently prior to DiT processing to prevent latent identity entanglement.<n>Building on this disentangled representation, we further propose Tri-Stage Decoupled Attention (TSDA) to bind identities to driving poses by decomposing self-attention into: (i) instance-aware foreground attention, (ii) background-centric interaction, and (iii) global-background coordination.
  arXiv  Detail & Related papers  (2026-03-16T15:25:04Z)
• SIDeR: Semantic Identity Decoupling for Unrestricted Face Privacy [53.75084833636302]
  We propose SIDeR, a Semantic decoupling-driven framework for unrestricted face privacy protection.<n> SIDeR decomposes a facial image into a machine-recognizable identity feature vector and a visually perceptible semantic appearance component.<n>For authorized access, SIDeR can be restored to its original form when the correct password is provided.
  arXiv  Detail & Related papers  (2026-02-04T19:30:48Z)
• Beyond the Dirac Delta: Mitigating Diversity Collapse in Reinforcement Fine-Tuning for Versatile Image Generation [51.305316234962554]
  We propose textbfDRIFT (textbfDivetextbfRsity-textbfIncentivized Reinforcement textbfFine-textbfTuning for Versatile Image Generation), an innovative framework that systematically incentivizes output throughout the on-policy fine-tuning process.<n>DRIFT achieves superior dominance regarding task alignment and generation diversity, yielding a $ 9.08%!sim! 43.46%$ increase in diversity equivalent alignment levels and a $ 59.65
  arXiv  Detail & Related papers  (2026-01-18T13:25:43Z)
• DiverseGRPO: Mitigating Mode Collapse in Image Generation via Diversity-Aware GRPO [50.89703227426486]
  Reinforcement learning (RL) improves image generation quality significantly by comparing the relative performance of images generated within the same group.<n>In the later stages of training, the model tends to produce homogenized outputs, lacking creativity and visual diversity.<n>This issue can be analyzed from both reward modeling and generation dynamics perspectives.
  arXiv  Detail & Related papers  (2025-12-25T05:37:37Z)
• Taming Identity Consistency and Prompt Diversity in Diffusion Models via Latent Concatenation and Masked Conditional Flow Matching [1.9270911143386336]
  Subject-driven image generation aims to synthesize novel depictions of a specific subject across diverse contexts.<n>We propose a LoRA fine-tuned diffusion model employing a latent concatenation strategy.<n>For filtering and quality assessment, we present CHARIS, a fine-grained evaluation framework.
  arXiv  Detail & Related papers  (2025-11-11T10:00:32Z)
• Beyond Inference Intervention: Identity-Decoupled Diffusion for Face Anonymization [55.29071072675132]
  Face anonymization aims to conceal identity information while preserving non-identity attributes.<n>We propose textbfIDsuperscript2Face, a training-centric anonymization framework.<n>We show that IDtextsuperscript2Face outperforms existing methods in visual quality, identity suppression, and utility preservation.
  arXiv  Detail & Related papers  (2025-10-28T09:28:12Z)
• WithAnyone: Towards Controllable and ID Consistent Image Generation [83.55786496542062]
  Identity-consistent generation has become an important focus in text-to-image research.<n>We develop a large-scale paired dataset tailored for multi-person scenarios.<n>We propose a novel training paradigm with a contrastive identity loss that leverages paired data to balance fidelity with diversity.
  arXiv  Detail & Related papers  (2025-10-16T17:59:54Z)
• DisCo: Reinforcement with Diversity Constraints for Multi-Human Generation [60.741022906593685]
  DisCo is the first RL-based framework to directly optimize identity diversity in multi-human generation.<n>DisCo fine-tunes flow-matching models via Group-Relative Policy Optimization.<n>On the DiverseHumans Testset, DisCo achieves 98.6 Unique Face Accuracy and near-perfect Global Identity Spread.
  arXiv  Detail & Related papers  (2025-10-01T19:28:51Z)
• UMO: Scaling Multi-Identity Consistency for Image Customization via Matching Reward [15.094319754425468]
  We present UMO, a framework designed to maintain high-fidelity identity preservation and alleviate identity confusion with scalability.<n>With "multi-to-multi matching" paradigm, UMO reformulates multi-identity generation as a global assignment optimization problem.<n>We develop a scalable customization dataset with multi-reference images, consisting of both synthesised and real parts.
  arXiv  Detail & Related papers  (2025-09-08T15:54:55Z)
• From Poses to Identity: Training-Free Person Re-Identification via Feature Centralization [9.614305363044737]
  Person re-identification (ReID) aims to extract accurate identity representation features.<n>We propose a Training-Free Feature Centralization ReID framework (Pose2ID) to reduce individual noise.<n>Our method sets new state-of-the-art results across standard, cross-modality, and occluded ReID tasks.
  arXiv  Detail & Related papers  (2025-03-02T15:31:48Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.