Scaling Multi-Reference Image Generation with Dynamic Reward Optimization

• URL: http://arxiv.org/abs/2606.26947v1
• Date: Thu, 25 Jun 2026 12:21:13 GMT
• Title: Scaling Multi-Reference Image Generation with Dynamic Reward Optimization
• Abstract summary: We propose DyRef, a two-stage training framework for complex MRIG tasks.<n>In the first stage, supervised fine-tuning equips the model with the basic capability to handle complex MRIG tasks.<n>In the second stage, we introduce Difficulty-aware Advantage Reweighting (DAR) and Discriminative Reward Scaling (DRS)
• Score: 25.072640574425417
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: While personalized image generation has achieved remarkable progress, multi-reference image generation (MRIG) remains a challenging task. Most existing benchmarks fail to adequately evaluate complex MRIG scenarios, hindering further progress in this area. To better assess model performance on complex MRIG tasks, we introduce OmniRef-Bench, a benchmark that covers complex combinations of reference image types and a large number of reference images. Evaluations on OmniRef-Bench show that mainstream open-source models struggle in complex MRIG scenarios, and their performance deteriorates significantly as the number of mixed-type reference images increases. To address this issue, we propose DyRef, a two-stage training framework. In the first stage, supervised fine-tuning equips the model with the basic capability to handle complex MRIG tasks. In the second stage, we introduce Difficulty-aware Advantage Reweighting (DAR) and Discriminative Reward Scaling (DRS). DAR dynamically adjusts the optimization objective to improve performance when handling a large number of mixed-type reference images. DRS enlarges intra-group reward differences for more effective policy optimization. Experiments demonstrate that DyRef significantly improves the performance of open-source models on OmniRef-Bench and single-image editing benchmarks, demonstrating the effectiveness and generalization capability of our approach.

Related papers

• Tuning Real-World Image Restoration at Inference: A Test-Time Scaling Paradigm for Flow Matching Models [26.002982722748726]
  ResFlow-Tuner is an image restoration framework based on the state-of-the-art flow matching model, FLUX.1-dev.<n>We propose a training-free test-time scaling paradigm tailored for image restoration.
  arXiv  Detail & Related papers  (2026-03-23T14:33:43Z)
• MICON-Bench: Benchmarking and Enhancing Multi-Image Context Image Generation in Unified Multimodal Models [89.89575486159795]
  We introduce textbfMICON-Bench, a benchmark for multi-image context generation.<n>We propose an MLLM-driven Evaluation-by-Checkpoint framework for automatic verification of semantic and visual consistency.<n>We also present textbfDynamic Attention Rebalancing (DAR), a training-free, plug-and-play mechanism that dynamically adjusts attention during inference to enhance coherence and reduce hallucinations.
  arXiv  Detail & Related papers  (2026-02-23T04:32:52Z)
• PS-ReID: Advancing Person Re-Identification and Precise Segmentation with Multimodal Retrieval [38.530536338075684]
  Person re-identification (ReID) plays a critical role in applications such as security surveillance and criminal investigations.<n>We propose bf PS-ReID, a multimodal model that combines image and text inputs to enhance ReID performance.<n> Experimental results demonstrate that PS-ReID significantly outperforms unimodal query-based models in both ReID and segmentation tasks.
  arXiv  Detail & Related papers  (2025-03-27T15:14:03Z)
• Bridging the Domain Gap: A Simple Domain Matching Method for Reference-based Image Super-Resolution in Remote Sensing [8.36527949191506]
  Recently, reference-based image super-resolution (RefSR) has shown excellent performance in image super-resolution (SR) tasks. We introduce a Domain Matching (DM) module that can be seamlessly integrated with existing RefSR models. Our analysis reveals that their domain gaps often occur in different satellites, and our model effectively addresses these challenges.
  arXiv  Detail & Related papers  (2024-01-29T08:10:00Z)
• DRM-IR: Task-Adaptive Deep Unfolding Network for All-In-One Image Restoration [5.573836220587265]
  This work proposes an efficient Dynamic Reference Modeling paradigm (DRM-IR) DRM-IR consists of task-adaptive degradation modeling and model-based image restoring. Experiments on multiple benchmark datasets show that our DRM-IR achieves state-of-the-art in All-In-One IR.
  arXiv  Detail & Related papers  (2023-07-15T02:42:19Z)
• Reference-based Image and Video Super-Resolution via C2-Matching [100.0808130445653]
  We propose C2-Matching, which performs explicit robust matching crossing transformation and resolution. C2-Matching significantly outperforms state of the arts on the standard CUFED5 benchmark. We also extend C2-Matching to Reference-based Video Super-Resolution task, where an image taken in a similar scene serves as the HR reference image.
  arXiv  Detail & Related papers  (2022-12-19T16:15:02Z)
• RRSR:Reciprocal Reference-based Image Super-Resolution with Progressive Feature Alignment and Selection [66.08293086254851]
  We propose a reciprocal learning framework to reinforce the learning of a RefSR network. The newly proposed module aligns reference-input images at multi-scale feature spaces and performs reference-aware feature selection. We empirically show that multiple recent state-of-the-art RefSR models can be consistently improved with our reciprocal learning paradigm.
  arXiv  Detail & Related papers  (2022-11-08T12:39:35Z)
• MASA-SR: Matching Acceleration and Spatial Adaptation for Reference-Based Image Super-Resolution [74.24676600271253]
  We propose the MASA network for RefSR, where two novel modules are designed to address these problems. The proposed Match & Extraction Module significantly reduces the computational cost by a coarse-to-fine correspondence matching scheme. The Spatial Adaptation Module learns the difference of distribution between the LR and Ref images, and remaps the distribution of Ref features to that of LR features in a spatially adaptive way.
  arXiv  Detail & Related papers  (2021-06-04T07:15:32Z)
• Robust Reference-based Super-Resolution via C2-Matching [77.51610726936657]
  Super-Resolution (Ref-SR) has recently emerged as a promising paradigm to enhance a low-resolution (LR) input image by introducing an additional high-resolution (HR) reference image. Existing Ref-SR methods mostly rely on implicit correspondence matching to borrow HR textures from reference images to compensate for the information loss in input images. We propose C2-Matching, which produces explicit robust matching crossing transformation and resolution.
  arXiv  Detail & Related papers  (2021-06-03T16:40:36Z)
• DDet: Dual-path Dynamic Enhancement Network for Real-World Image Super-Resolution [69.2432352477966]
  Real image super-resolution(Real-SR) focus on the relationship between real-world high-resolution(HR) and low-resolution(LR) image. In this article, we propose a Dual-path Dynamic Enhancement Network(DDet) for Real-SR. Unlike conventional methods which stack up massive convolutional blocks for feature representation, we introduce a content-aware framework to study non-inherently aligned image pair.
  arXiv  Detail & Related papers  (2020-02-25T18:24:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.