HP-Edit: A Human-Preference Post-Training Framework for Image Editing

• URL: http://arxiv.org/abs/2604.19406v1
• Date: Tue, 21 Apr 2026 12:29:50 GMT
• Title: HP-Edit: A Human-Preference Post-Training Framework for Image Editing
• Abstract summary: We propose HP-Edit, a post-training framework for Human Preference-aligned Editing.<n>We also introduce RealPref-50K, a real-world dataset across eight common tasks and balancing common object editing.
• Score: 56.02114960634816
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Common image editing tasks typically adopt powerful generative diffusion models as the leading paradigm for real-world content editing. Meanwhile, although reinforcement learning (RL) methods such as Diffusion-DPO and Flow-GRPO have further improved generation quality, efficiently applying Reinforcement Learning from Human Feedback (RLHF) to diffusion-based editing remains largely unexplored, due to a lack of scalable human-preference datasets and frameworks tailored to diverse editing needs. To fill this gap, we propose HP-Edit, a post-training framework for Human Preference-aligned Editing, and introduce RealPref-50K, a real-world dataset across eight common tasks and balancing common object editing. Specifically, HP-Edit leverages a small amount of human-preference scoring data and a pretrained visual large language model (VLM) to develop HP-Scorer--an automatic, human preference-aligned evaluator. We then use HP-Scorer both to efficiently build a scalable preference dataset and to serve as the reward function for post-training the editing model. We also introduce RealPref-Bench, a benchmark for evaluating real-world editing performance. Extensive experiments demonstrate that our approach significantly enhances models such as Qwen-Image-Edit-2509, aligning their outputs more closely with human preference.

Related papers

• Leveraging Verifier-Based Reinforcement Learning in Image Editing [41.64251379221566]
  We introduce Edit-R1, a framework that builds a chain-of-thought (CoT) verifier-based reasoning reward model (RRM) for downstream image editing.<n>Our experiments demonstrate that our Edit-RRM surpasses powerful VLMs as an editing-specific reward model.
  arXiv  Detail & Related papers  (2026-04-30T06:54:39Z)
• Trust Your Critic: Robust Reward Modeling and Reinforcement Learning for Faithful Image Editing and Generation [67.26349227500084]
  Current reward models, which act as critics during reinforcement learning, often suffer from hallucinations and assign noisy scores.<n>We present FIRM, a comprehensive framework that develops robust reward models to provide accurate and reliable guidance for faithful image generation and editing.<n> FIRM mitigates hallucinations, establishing a new standard for fidelity and instruction adherence over existing general models.
  arXiv  Detail & Related papers  (2026-03-12T17:57:21Z)
• Personalized Image Editing in Text-to-Image Diffusion Models via Collaborative Direct Preference Optimization [11.306247975771013]
  Collaborative Preference Optimization (C-DPO) is a novel method that aligns image edits with user-specific preferences.<n>Our approach encodes each user as a node in a dynamic preference graph and learns embeddings via a lightweight graph neural network.<n>Our method consistently outperforms baselines in generating edits that are aligned with user preferences.
  arXiv  Detail & Related papers  (2025-11-06T18:59:54Z)
• Learning an Image Editing Model without Image Editing Pairs [83.03646586929638]
  Recent image editing models have achieved impressive results while following natural language editing instructions.<n>They rely on supervised fine-tuning with large datasets of input-target pairs.<n>Current workarounds use synthetic training pairs that leverage the zero-shot capabilities of existing models.<n>We present a new training paradigm that eliminates the need for paired data entirely.
  arXiv  Detail & Related papers  (2025-10-16T17:59:57Z)
• EditScore: Unlocking Online RL for Image Editing via High-Fidelity Reward Modeling [71.8265422228785]
  Reinforcement Learning (RL) offers a promising solution, but its adoption in image editing has been hindered by the lack of a high-fidelity, efficient reward signal.<n>We present a comprehensive methodology to overcome this barrier, centered on the development of a state-of-the-art, specialized reward model.
  arXiv  Detail & Related papers  (2025-09-28T14:28:24Z)
• PartEdit: Fine-Grained Image Editing using Pre-Trained Diffusion Models [80.98455219375862]
  We present the first text-based image editing approach for object parts based on pre-trained diffusion models.<n>Our approach is preferred by users 66-90% of the time in conducted user studies.
  arXiv  Detail & Related papers  (2025-02-06T13:08:43Z)
• PIXELS: Progressive Image Xemplar-based Editing with Latent Surgery [10.594261300488546]
  We introduce a novel framework for progressive exemplar-driven editing with off-the-shelf diffusion models, dubbed PIXELS.<n>PIXELS provides granular control over edits, allowing adjustments at the pixel or region level.<n>We demonstrate that PIXELS delivers high-quality edits efficiently, leading to a notable improvement in quantitative metrics as well as human evaluation.
  arXiv  Detail & Related papers  (2025-01-16T20:26:30Z)
• InstructRL4Pix: Training Diffusion for Image Editing by Reinforcement Learning [31.799923647356458]
  We propose Reinforcement Learning Guided Image Editing Method(InstructRL4Pix) to train a diffusion model to generate images that are guided by the attention maps of the target object. Experimental results show that InstructRL4Pix breaks through the limitations of traditional datasets and uses unsupervised learning to optimize editing goals and achieve accurate image editing based on natural human commands.
  arXiv  Detail & Related papers  (2024-06-14T12:31:48Z)
• HIVE: Harnessing Human Feedback for Instructional Visual Editing [127.29436858998064]
  We present a novel framework to harness human feedback for instructional visual editing (HIVE) Specifically, we collect human feedback on the edited images and learn a reward function to capture the underlying user preferences. We then introduce scalable diffusion model fine-tuning methods that can incorporate human preferences based on the estimated reward.
  arXiv  Detail & Related papers  (2023-03-16T19:47:41Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.