Learning Feature-Preserving Portrait Editing from Generated Pairs

• URL: http://arxiv.org/abs/2407.20455v1
• Date: Mon, 29 Jul 2024 23:19:42 GMT
• Title: Learning Feature-Preserving Portrait Editing from Generated Pairs
• Authors: Bowei Chen, Tiancheng Zhi, Peihao Zhu, Shen Sang, Jing Liu, Linjie Luo,
• Abstract summary: We propose a training-based method leveraging auto-generated paired data to learn desired editing. Our method achieves state-of-the-art quality, quantitatively and qualitatively.
• Score: 11.122956539965761
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Portrait editing is challenging for existing techniques due to difficulties in preserving subject features like identity. In this paper, we propose a training-based method leveraging auto-generated paired data to learn desired editing while ensuring the preservation of unchanged subject features. Specifically, we design a data generation process to create reasonably good training pairs for desired editing at low cost. Based on these pairs, we introduce a Multi-Conditioned Diffusion Model to effectively learn the editing direction and preserve subject features. During inference, our model produces accurate editing mask that can guide the inference process to further preserve detailed subject features. Experiments on costume editing and cartoon expression editing show that our method achieves state-of-the-art quality, quantitatively and qualitatively.

Related papers

• PartEdit: Fine-Grained Image Editing using Pre-Trained Diffusion Models [80.98455219375862]
  We present the first text-based image editing approach for object parts based on pre-trained diffusion models. Our approach is preferred by users 77-90% of the time in conducted user studies.
  arXiv  Detail & Related papers  (2025-02-06T13:08:43Z)
• IP-FaceDiff: Identity-Preserving Facial Video Editing with Diffusion [12.494492016414503]
  Existing models encounter challenges such as poor editing quality, high computational costs and difficulties in preserving facial identity across diverse edits. We propose a novel facial video editing framework that leverages the rich latent space of pre-trained text-to-image (T2I) diffusion models. Our approach significantly reduces editing time by 80%, while maintaining temporal consistency throughout the video sequence.
  arXiv  Detail & Related papers  (2025-01-13T18:08:27Z)
• Edicho: Consistent Image Editing in the Wild [90.42395533938915]
  Edicho steps in with a training-free solution based on diffusion models. It features a fundamental design principle of using explicit image correspondence to direct editing.
  arXiv  Detail & Related papers  (2024-12-30T16:56:44Z)
• UIP2P: Unsupervised Instruction-based Image Editing via Cycle Edit Consistency [69.33072075580483]
  We propose an unsupervised model for instruction-based image editing that eliminates the need for ground-truth edited images during training. Our method addresses these challenges by introducing a novel editing mechanism called Cycle Edit Consistency ( CEC) CEC applies forward and backward edits in one training step and enforces consistency in image and attention spaces.
  arXiv  Detail & Related papers  (2024-12-19T18:59:58Z)
• Pathways on the Image Manifold: Image Editing via Video Generation [11.891831122571995]
  We reformulate image editing as a temporal process, using pretrained video models to create smooth transitions from the original image to the desired edit. Our approach achieves state-of-the-art results on text-based image editing, demonstrating significant improvements in both edit accuracy and image preservation.
  arXiv  Detail & Related papers  (2024-11-25T16:41:45Z)
• AnyEdit: Mastering Unified High-Quality Image Editing for Any Idea [88.79769371584491]
  We present AnyEdit, a comprehensive multi-modal instruction editing dataset. We ensure the diversity and quality of the AnyEdit collection through three aspects: initial data diversity, adaptive editing process, and automated selection of editing results. Experiments on three benchmark datasets show that AnyEdit consistently boosts the performance of diffusion-based editing models.
  arXiv  Detail & Related papers  (2024-11-24T07:02:56Z)
• LIPE: Learning Personalized Identity Prior for Non-rigid Image Editing [20.861672583434718]
  LIPE is a two-stage framework designed to customize the generative model utilizing a limited set of images of the same subject. We present LIPE, a two-stage framework designed to customize the generative model utilizing a limited set of images of the same subject, and subsequently employ the model with learned prior for non-rigid image editing.
  arXiv  Detail & Related papers  (2024-06-25T02:56:16Z)
• Optimisation-Based Multi-Modal Semantic Image Editing [58.496064583110694]
  We propose an inference-time editing optimisation to accommodate multiple editing instruction types. By allowing to adjust the influence of each loss function, we build a flexible editing solution that can be adjusted to user preferences. We evaluate our method using text, pose and scribble edit conditions, and highlight our ability to achieve complex edits.
  arXiv  Detail & Related papers  (2023-11-28T15:31:11Z)
• Emu Edit: Precise Image Editing via Recognition and Generation Tasks [62.95717180730946]
  We present Emu Edit, a multi-task image editing model which sets state-of-the-art results in instruction-based image editing. We train it to multi-task across an unprecedented range of tasks, such as region-based editing, free-form editing, and Computer Vision tasks. We show that Emu Edit can generalize to new tasks, such as image inpainting, super-resolution, and compositions of editing tasks, with just a few labeled examples.
  arXiv  Detail & Related papers  (2023-11-16T18:55:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.