Personalized Image Editing in Text-to-Image Diffusion Models via Collaborative Direct Preference Optimization

• URL: http://arxiv.org/abs/2511.05616v1
• Date: Thu, 06 Nov 2025 18:59:54 GMT
• Title: Personalized Image Editing in Text-to-Image Diffusion Models via Collaborative Direct Preference Optimization
• Authors: Connor Dunlop, Matthew Zheng, Kavana Venkatesh, Pinar Yanardag,
• Abstract summary: Collaborative Preference Optimization (C-DPO) is a novel method that aligns image edits with user-specific preferences.<n>Our approach encodes each user as a node in a dynamic preference graph and learns embeddings via a lightweight graph neural network.<n>Our method consistently outperforms baselines in generating edits that are aligned with user preferences.
• Score: 11.306247975771013
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Text-to-image (T2I) diffusion models have made remarkable strides in generating and editing high-fidelity images from text. Yet, these models remain fundamentally generic, failing to adapt to the nuanced aesthetic preferences of individual users. In this work, we present the first framework for personalized image editing in diffusion models, introducing Collaborative Direct Preference Optimization (C-DPO), a novel method that aligns image edits with user-specific preferences while leveraging collaborative signals from like-minded individuals. Our approach encodes each user as a node in a dynamic preference graph and learns embeddings via a lightweight graph neural network, enabling information sharing across users with overlapping visual tastes. We enhance a diffusion model's editing capabilities by integrating these personalized embeddings into a novel DPO objective, which jointly optimizes for individual alignment and neighborhood coherence. Comprehensive experiments, including user studies and quantitative benchmarks, demonstrate that our method consistently outperforms baselines in generating edits that are aligned with user preferences.

Related papers

• EditInfinity: Image Editing with Binary-Quantized Generative Models [64.05135380710749]
  We investigate the parameter-efficient adaptation of binary-quantized generative models for image editing.<n>Specifically, we propose EditInfinity, which adapts emphInfinity, a binary-quantized generative model, for image editing.<n>We propose an efficient yet effective image inversion mechanism that integrates text prompting rectification and image style preservation.
  arXiv  Detail & Related papers  (2025-10-23T05:06:24Z)
• ImageGem: In-the-wild Generative Image Interaction Dataset for Generative Model Personalization [11.7261367003714]
  ImageGem is a dataset for studying generative models that understand fine-grained individual preferences.<n>Our dataset features real-world interaction data from 57K users, who collectively have built 242K customized LoRAs, written 3M text prompts, and created 5M generated images.
  arXiv  Detail & Related papers  (2025-10-21T09:08:01Z)
• PartEdit: Fine-Grained Image Editing using Pre-Trained Diffusion Models [80.98455219375862]
  We present the first text-based image editing approach for object parts based on pre-trained diffusion models.<n>Our approach is preferred by users 66-90% of the time in conducted user studies.
  arXiv  Detail & Related papers  (2025-02-06T13:08:43Z)
• Personalized Preference Fine-tuning of Diffusion Models [75.22218338096316]
  We introduce PPD, a multi-reward optimization objective that aligns diffusion models with personalized preferences.<n>With PPD, a diffusion model learns the individual preferences of a population of users in a few-shot way.<n>Our approach achieves an average win rate of 76% over Stable Cascade, generating images that more accurately reflect specific user preferences.
  arXiv  Detail & Related papers  (2025-01-11T22:38:41Z)
• DreamSteerer: Enhancing Source Image Conditioned Editability using Personalized Diffusion Models [7.418186319496487]
  Recent text-to-image personalization methods have shown great promise in teaching a diffusion model user-specified concepts. A promising extension is personalized editing, namely to edit an image using personalized concepts. We propose DreamSteerer, a plug-in method for augmenting existing T2I personalization methods.
  arXiv  Detail & Related papers  (2024-10-15T02:50:54Z)
• Powerful and Flexible: Personalized Text-to-Image Generation via Reinforcement Learning [40.06403155373455]
  We propose a novel reinforcement learning framework for personalized text-to-image generation. Our proposed approach outperforms existing state-of-the-art methods by a large margin on visual fidelity while maintaining text-alignment.
  arXiv  Detail & Related papers  (2024-07-09T08:11:53Z)
• JeDi: Joint-Image Diffusion Models for Finetuning-Free Personalized Text-to-Image Generation [49.997839600988875]
  Existing personalization methods rely on finetuning a text-to-image foundation model on a user's custom dataset. We propose Joint-Image Diffusion (jedi), an effective technique for learning a finetuning-free personalization model. Our model achieves state-of-the-art generation quality, both quantitatively and qualitatively, significantly outperforming both the prior finetuning-based and finetuning-free personalization baselines.
  arXiv  Detail & Related papers  (2024-07-08T17:59:02Z)
• FreeCompose: Generic Zero-Shot Image Composition with Diffusion Prior [50.0535198082903]
  We offer a novel approach to image composition, which integrates multiple input images into a single, coherent image. We showcase the potential of utilizing the powerful generative prior inherent in large-scale pre-trained diffusion models to accomplish generic image composition.
  arXiv  Detail & Related papers  (2024-07-06T03:35:43Z)
• Direct Consistency Optimization for Robust Customization of Text-to-Image Diffusion Models [67.68871360210208]
  Text-to-image (T2I) diffusion models, when fine-tuned on a few personal images, can generate visuals with a high degree of consistency.<n>We propose a novel fine-tuning objective, dubbed Direct Consistency Optimization, which controls the deviation between fine-tuning and pretrained models.<n>We show that our approach achieves better prompt fidelity and subject fidelity than those post-optimized for merging regular fine-tuned models.
  arXiv  Detail & Related papers  (2024-02-19T09:52:41Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.