Uni-Synergy: Bridging Understanding and Generation for Personalized Reasoning via Co-operative Reinforcement Learning

• URL: http://arxiv.org/abs/2605.10445v1
• Date: Mon, 11 May 2026 12:18:26 GMT
• Title: Uni-Synergy: Bridging Understanding and Generation for Personalized Reasoning via Co-operative Reinforcement Learning
• Abstract summary: We propose Sync-R1, an end-to-end reinforcement learning framework that jointly optimize personalized understanding and generation.<n> Sync-R1 enables personalized comprehension to guide content creation, while the resulting generation quality reciprocally refines understanding.<n>We also introduce Dynamic Group Scaling (DGS), which adaptively filters low-potential trajectories to reduce gradient variance and accelerate convergence.<n> Experimental results demonstrate that Sync-R1 achieves state-of-the-art performance, showcasing superior cross-task reasoning and robust personalization.
• Score: 56.99010101756807
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Unified Multimodal Models (UMMs) excel in general tasks but struggle to bridge the gap between personalized understanding and generation. Prior works largely rely on implicit token-level alignment via supervised fine-tuning, which fails to fully capture the potential synergy between comprehension and creation. In this work, we propose Sync-R1, an end-to-end reinforcement learning framework that jointly optimizes personalized understanding and generation within a single, explicit reasoning loop. Through this unified feedback process, Sync-R1 enables personalized comprehension to guide content creation, while the resulting generation quality reciprocally refines understanding within an integrated reward landscape. To efficiently orchestrate this dual-task synergy, we introduce Sync-GRPO, a reinforcement learning method utilizing an ensemble reward system. Furthermore, we propose Dynamic Group Scaling (DGS), which adaptively filters low-potential trajectories to reduce gradient variance and accelerate convergence. To better reflect real-world complexity, we introduce UnifyBench++, featuring denser textual descriptions and richer user contexts. Experimental results demonstrate that Sync-R1 achieves state-of-the-art performance, showcasing superior cross-task reasoning and robust personalization without requiring complex cold-start procedures. The code and the UnifyBench++ dataset will be released at: https://github.com/arctanxarc/UniCTokens.

Related papers

• Uncovering Understanding-Generation Synergy in Native Unified Multimodal Models: From Representation, Task to System [90.64481646805855]
  unified multimodal models (UMMs) jointly perform visual understanding and generation within a single model.<n>We investigate their relationship at the representation, task, and system levels in a controlled, structurally native setting.<n>We show that the value of UMMs extends beyond a unified interface: appropriate specialization, shared task knowledge, and end-to-end optimization can turn coexistence into synergy.
  arXiv  Detail & Related papers  (2026-09-01T17:59:59Z)
• Accelerating Unified Multimodal Models with Core-Expansion Routing and Unified Computation Scheduling [61.28599393707943]
  Unified multimodal models jointly support understanding and generation, but incur substantial redundant computation across tokens, layers, and generation timesteps.<n>We propose a task-shared core scorer and progress-conditioned generation expansions, optimized through generation decomposition and cross-task core alignment.<n> Experiments on two representative UMM architectures demonstrate consistent quality--efficiency improvements across both tasks, retaining 98.03% of dense understanding performance with a 1.93times$ end-to-end inference speedup.
  arXiv  Detail & Related papers  (2026-08-29T14:27:55Z)
• C^2GR: Coupled Comprehensive Generative Replay for a Continually Learnable Universal Segmentation Model [17.749377881392828]
  Universal segmentation models exhibit significant potential for diverse tasks involving different imaging modalities and segmentation objectives.<n> Task-Incremental Learning provides a privacy-preserving approach to continually evolve a universal model on tasks from sequentially-arriving medical departments.<n>We propose a novel Coupled Comprehensive Generative (C2GR) framework that simultaneously synthesizes image-mask pairs of previous tasks.
  arXiv  Detail & Related papers  (2026-06-22T15:19:03Z)
• Causal Reward World Models: Zero-shot Reward Design for Automated Skill Generation [23.371552518874807]
  Automated Reward Design (ARD) aims to replace manual reward engineering in reinforcement learning with language-driven reward function synthesis.<n>Existing approaches based on large language models (LLMs) remain inherently correlation-driven, relying on iterative environmental feedback to refine reward hypotheses for each specific task.<n>We propose the Causal Reward World Model (CRWM), which explicitly models the causal relationships between candidate reward components and task-targeted physical variables through offline pre-training on multi-task interaction data.
  arXiv  Detail & Related papers  (2026-06-22T12:57:17Z)
• Steering Visual Generation in Unified Multimodal Models with Understanding Supervision [42.765106450407814]
  Unified multimodal models are envisioned to bridge the gap between understanding and generation.<n>We propose Understanding-Oriented Post-Training (UNO), a lightweight framework that treats understanding not only as a distinct task, but also a direct supervisory signal to steer generative representations.
  arXiv  Detail & Related papers  (2026-05-07T07:20:04Z)
• Best of Both Worlds: Multimodal Reasoning and Generation via Unified Discrete Flow Matching [17.68651859356751]
  UniDFlow is a discrete flow-matching framework for multimodal understanding, generation, and editing.<n>UniDFlpw SOTA achieves strong zero-shot generalization to tasks including inpainting, in-context image generation, reference-based editing, and compositional generation.
  arXiv  Detail & Related papers  (2026-02-12T17:59:08Z)
• PIT: A Dynamic Personalized Item Tokenizer for End-to-End Generative Recommendation [10.959841655014387]
  PIT is a dynamic Personalized Item Tokenizer framework for end-to-end generative recommendation.<n>It employs a co-generative architecture that harmonizes collaborative patterns through collaborative signal alignment.<n>Experiments on real-world datasets demonstrate that PIT consistently outperforms competitive baselines.
  arXiv  Detail & Related papers  (2026-02-09T11:28:56Z)
• UniX: Unifying Autoregression and Diffusion for Chest X-Ray Understanding and Generation [98.93314262366681]
  We present UniX, a next-generation unified medical foundation model for chest X-ray understanding and generation.<n>UniX decouples the two tasks into an autoregressive branch for understanding and a diffusion branch for high-fidelity generation.<n>On two representative benchmarks, UniX achieves a 46.1% improvement in understanding performance and a 24.2% gain in generation quality.
  arXiv  Detail & Related papers  (2026-01-16T18:59:58Z)
• HiCoGen: Hierarchical Compositional Text-to-Image Generation in Diffusion Models via Reinforcement Learning [66.99487505369254]
  HiCoGen is built upon a novel Chain of Synthesis paradigm.<n>It decomposes complex prompts into minimal semantic units.<n>It then synthesizes these units iteratively, where the image generated in each step provides crucial visual context for the next.<n>Experiments show our approach significantly outperforms existing methods in both concept coverage and compositional accuracy.
  arXiv  Detail & Related papers  (2025-11-25T06:24:25Z)
• CIR-CoT: Towards Interpretable Composed Image Retrieval via End-to-End Chain-of-Thought Reasoning [93.05917922306196]
  Composed Image Retrieval (CIR) aims to find a target image from a reference image and a modification text.<n>CIR-CoT is the first end-to-end retrieval-oriented MLLM designed to integrate explicit Chain-of-Thought (CoT) reasoning.
  arXiv  Detail & Related papers  (2025-10-09T09:41:45Z)
• COPO: Consistency-Aware Policy Optimization [17.328515578426227]
  Reinforcement learning has significantly enhanced the reasoning capabilities of Large Language Models (LLMs) in complex problem-solving tasks.<n>Recently, the introduction of DeepSeek R1 has inspired a surge of interest in leveraging rule-based rewards as a low-cost alternative for computing advantage functions and guiding policy optimization.<n>We propose a consistency-aware policy optimization framework that introduces a structured global reward based on outcome consistency.
  arXiv  Detail & Related papers  (2025-08-06T07:05:18Z)
• Killing Two Birds with One Stone: Unifying Retrieval and Ranking with a Single Generative Recommendation Model [71.45491434257106]
  Unified Generative Recommendation Framework (UniGRF) is a novel approach that integrates retrieval and ranking into a single generative model.<n>To enhance inter-stage collaboration, UniGRF introduces a ranking-driven enhancer module.<n>UniGRF significantly outperforms existing models on benchmark datasets.
  arXiv  Detail & Related papers  (2025-04-23T06:43:54Z)
• SyMerge: From Non-Interference to Synergistic Merging via Single-Layer Adaptation [28.417947631789783]
  SyMerge is a lightweight framework that jointly optimize one task-specific layer and merging coefficients.<n>SyMerge achieves state-of-the-art results across vision, dense prediction, and NLP benchmarks.
  arXiv  Detail & Related papers  (2024-12-26T07:42:06Z)
• Plug-and-Play Regulators for Image-Text Matching [76.28522712930668]
  Exploiting fine-grained correspondence and visual-semantic alignments has shown great potential in image-text matching. We develop two simple but quite effective regulators which efficiently encode the message output to automatically contextualize and aggregate cross-modal representations. Experiments on MSCOCO and Flickr30K datasets validate that they can bring an impressive and consistent R@1 gain on multiple models.
  arXiv  Detail & Related papers  (2023-03-23T15:42:05Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.