DARS: Dual-Level Credit Assignment RL with Structured Reasoning for Instruction-Based Image Editing

• URL: http://arxiv.org/abs/2608.20161v1
• Date: Thu, 20 Aug 2026 15:16:39 GMT
• Title: DARS: Dual-Level Credit Assignment RL with Structured Reasoning for Instruction-Based Image Editing
• Abstract summary: We present DARS, a reinforcement learning framework for dual-level credit assignment.<n>We show that DARS outperforms a JointRL baseline with the same backbone, data, reward model, and rollout budget.
• Score: 10.682353786796734
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Instruction-based image editing uses a planner-renderer pipeline: a vision-language model (VLM) first converts the instruction into an edit plan, and a diffusion model then executes that plan. Training such systems with only final-image rewards is inefficient because a poor edit does not reveal whether additional optimization should place more emphasis on the planner or the renderer, and even planner-dominant cases remain difficult to localize within a free-form reasoning trace. We present DARS, a reinforcement learning framework for dual-level credit assignment in this two-stage setting. Across modules, multi-plan multi-render rollouts estimate between-plan and within-plan reward variability for soft module routing, while rollout mean rewards provide hardness estimates for an adaptive curriculum. Within the planner, a four-field structured reasoning output enables a prefix-gated reward and token-level advantage reweighting, turning outcome-level feedback into localized supervision. Experiments on five benchmarks show that DARS outperforms a Joint~RL baseline with the same backbone, data, reward model, and rollout budget, with the largest gains on reasoning-intensive edits.

Related papers

• REChart: Reasoning-Efficient Chart Editing with Large Reasoning Models [6.014442477518621]
  We introduce REChart, a two-stage training framework that provides process-level supervision over intermediate reasoning steps.<n>First, we synthesize 200k high-quality reasoning trajectories for supervised fine-tuning from a large image-instruction-code pool.<n>Second, we optimize the model via reinforcement learning with two complementary rewards: a emphfidelity reward evaluating code correctness, visual fidelity, and structural consistency, and an emphefficiency reward that assigns each rollout a random thinking budget.
  arXiv  Detail & Related papers  (2026-08-18T06:26:45Z)
• TRACE: A Unified Rollout Budget Allocation Framework for Efficient Agentic Reinforcement Learning [52.67756371231985]
  Reinforcement learning with verifiable rewards (RLVR) is a promising approach for enhancing reasoning and agentic behavior in large language models.<n>We introduce Tree Rollout Allocation for Contrastive Exploration (TRACE), a unified rollout allocation framework.<n>Technically, TRACE allocates rollout budget to both prompt roots and intermediate prefixes that are most likely to yield mixed terminal rewards.
  arXiv  Detail & Related papers  (2026-06-09T17:16:03Z)
• TIER: Trajectory-Invariant Execution Rewards for Multi-Step Tool Composition [62.56752617853322]
  Outcome-based rewards provide only sparse feedback, while trajectory-supervised rewards depend on annotated reference solutions.<n>We propose TIER: Trajectory-Invariant Execution Rewards, a reward framework that derives supervision directly from function schemas and runtime execution.
  arXiv  Detail & Related papers  (2026-05-16T03:47:26Z)
• DDA-Thinker: Decoupled Dual-Atomic Reinforcement Learning for Reasoning-Driven Image Editing [41.08605870394525]
  We propose a Thinker-centric framework for the independent optimization of a planning module (Thinker) over a fixed generative model (Editor)<n>This framework decomposes feedback into two distinct atomic rewards implemented through verifiable checklists.<n>Experiments on reasoning-driven image editing benchmarks, including RISE-Bench and KRIS-Bench, demonstrate that our approach substantially improves overall performance.
  arXiv  Detail & Related papers  (2026-04-28T10:30:01Z)
• VisFly-Lab: Unified Differentiable Framework for First-Order Reinforcement Learning of Quadrotor Control [11.762246555812554]
  First-order reinforcement learning with differentiable simulation is promising for quadrotor control.<n>But practical progress remains fragmented across task-specific settings.<n>We present a unified differentiable framework for multi-task quadrotor control.
  arXiv  Detail & Related papers  (2026-03-22T08:42:26Z)
• Agentic Planning with Reasoning for Image Styling via Offline RL [66.10749901925941]
  Direct prompt-based editing fails on complex transformations because vague and subjective prompts often require nuanced understanding of what should be changed in the image.<n>We present a tool-based agentic RL post-training framework that addresses this through structured planning with chain-of-thought reasoning.
  arXiv  Detail & Related papers  (2026-03-07T11:14:37Z)
• ThinkRL-Edit: Thinking in Reinforcement Learning for Reasoning-Centric Image Editing [33.888289858260706]
  Reinforcement learning (RL) has been investigated for improving the quality of image editing.<n>RL faces three key challenges: (1) limited reasoning exploration confined to denoising, (2) biased reward fusion, and (3) unstable VLM-based instruction rewards.<n>We propose ThinkRL-Edit, a reasoning-centric RL framework that decouples visual reasoning from image synthesis.
  arXiv  Detail & Related papers  (2026-01-06T23:43:00Z)
• RePlan: Reasoning-guided Region Planning for Complex Instruction-based Image Editing [80.70169829264812]
  RePlan is a plan-then-execute framework that couples a vision-language planner with a diffusion editor.<n>The planner decomposes instructions via step-by-step reasoning and explicitly grounds them to target regions.<n>The editor then applies changes using a training-free attention-region injection mechanism, enabling precise, parallel multi-region edits without iterative inpainting.
  arXiv  Detail & Related papers  (2025-12-18T18:34:23Z)
• Learning a Dense Reasoning Reward Model from Expert Demonstration via Inverse Reinforcement Learning [50.20267980386502]
  We learn a dense, token-level reward model for process supervision directly from expert demonstrations.<n>The learned reasoning reward serves two complementary roles: (i) it provides step-level feedback to optimise a reasoning policy during training; and (ii) it functions at inference as a critic to rerank sampled traces under fixed compute budgets.
  arXiv  Detail & Related papers  (2025-10-02T09:55:26Z)
• DART: Dual Adaptive Refinement Transfer for Open-Vocabulary Multi-Label Recognition [59.203152078315235]
  Open-Vocabulary Multi-Label Recognition (OV-MLR) aims to identify multiple seen and unseen object categories within an image.<n> Vision-Language Pre-training models offer a strong open-vocabulary foundation, but struggle with fine-grained localization under weak supervision.<n>We propose the Dual Adaptive Refinement Transfer (DART) framework to overcome these limitations.
  arXiv  Detail & Related papers  (2025-08-07T17:22:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.