StableDrag: Stable Dragging for Point-based Image Editing
- URL: http://arxiv.org/abs/2403.04437v1
- Date: Thu, 7 Mar 2024 12:11:02 GMT
- Title: StableDrag: Stable Dragging for Point-based Image Editing
- Authors: Yutao Cui, Xiaotong Zhao, Guozhen Zhang, Shengming Cao, Kai Ma and
Limin Wang
- Abstract summary: Point-based image editing has attracted remarkable attention since the emergence of DragGAN.
Recently, DragDiffusion further pushes forward the generative quality via adapting this dragging technique to diffusion models.
We build a stable and precise drag-based editing framework, coined as StableDrag, by designing a discirminative point tracking method and a confidence-based latent enhancement strategy for motion supervision.
- Score: 24.924112878074336
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Point-based image editing has attracted remarkable attention since the
emergence of DragGAN. Recently, DragDiffusion further pushes forward the
generative quality via adapting this dragging technique to diffusion models.
Despite these great success, this dragging scheme exhibits two major drawbacks,
namely inaccurate point tracking and incomplete motion supervision, which may
result in unsatisfactory dragging outcomes. To tackle these issues, we build a
stable and precise drag-based editing framework, coined as StableDrag, by
designing a discirminative point tracking method and a confidence-based latent
enhancement strategy for motion supervision. The former allows us to precisely
locate the updated handle points, thereby boosting the stability of long-range
manipulation, while the latter is responsible for guaranteeing the optimized
latent as high-quality as possible across all the manipulation steps. Thanks to
these unique designs, we instantiate two types of image editing models
including StableDrag-GAN and StableDrag-Diff, which attains more stable
dragging performance, through extensive qualitative experiments and
quantitative assessment on DragBench.
Related papers
- Localize, Understand, Collaborate: Semantic-Aware Dragging via Intention Reasoner [8.310002338000954]
Current methods typically model this problem as automatically learning how to drag'' through point dragging.
We propose LucidDrag, which shifts the focus from how to drag'' to a paradigm of what-then-how''
arXiv Detail & Related papers (2024-06-01T13:10:43Z) - MotionFollower: Editing Video Motion via Lightweight Score-Guided Diffusion [94.66090422753126]
MotionFollower is a lightweight score-guided diffusion model for video motion editing.
It delivers superior motion editing performance and exclusively supports large camera movements and actions.
Compared with MotionEditor, the most advanced motion editing model, MotionFollower achieves an approximately 80% reduction in GPU memory.
arXiv Detail & Related papers (2024-05-30T17:57:30Z) - InstaDrag: Lightning Fast and Accurate Drag-based Image Editing Emerging from Videos [101.59710862476041]
We present InstaDrag, a rapid approach enabling high quality drag-based image editing in 1 second.
Unlike most previous methods, we redefine drag-based editing as a conditional generation task.
Our approach can significantly outperform previous methods in terms of accuracy and consistency.
arXiv Detail & Related papers (2024-05-22T15:14:00Z) - DragGaussian: Enabling Drag-style Manipulation on 3D Gaussian Representation [57.406031264184584]
DragGaussian is a 3D object drag-editing framework based on 3D Gaussian Splatting.
Our contributions include the introduction of a new task, the development of DragGaussian for interactive point-based 3D editing, and comprehensive validation of its effectiveness through qualitative and quantitative experiments.
arXiv Detail & Related papers (2024-05-09T14:34:05Z) - GoodDrag: Towards Good Practices for Drag Editing with Diffusion Models [31.708968272342315]
We introduce GoodDrag, a novel approach to improve the stability and image quality of drag editing.
GoodDrag introduces an AlDD framework that alternates between drag and denoising operations within the diffusion process.
We also propose an information-preserving motion supervision operation that maintains the original features of the starting point for precise manipulation and artifact reduction.
arXiv Detail & Related papers (2024-04-10T17:59:59Z) - FreeDrag: Feature Dragging for Reliable Point-based Image Editing [17.837570645460964]
We propose FreeDrag, a feature dragging methodology designed to free the burden on point tracking.
The FreeDrag incorporates two key designs, i.e., template feature via adaptive updating and line search with backtracking.
Our approach significantly outperforms pre-existing methodologies, offering reliable point-based editing even in various complex scenarios.
arXiv Detail & Related papers (2023-07-10T16:37:46Z) - DragDiffusion: Harnessing Diffusion Models for Interactive Point-based Image Editing [94.24479528298252]
DragGAN is an interactive point-based image editing framework that achieves impressive editing results with pixel-level precision.
By harnessing large-scale pretrained diffusion models, we greatly enhance the applicability of interactive point-based editing on both real and diffusion-generated images.
We present a challenging benchmark dataset called DragBench to evaluate the performance of interactive point-based image editing methods.
arXiv Detail & Related papers (2023-06-26T06:04:09Z) - Drag Your GAN: Interactive Point-based Manipulation on the Generative Image Manifold [79.94300820221996]
DragGAN is a new way of controlling generative adversarial networks (GANs)
DragGAN allows anyone to deform an image with precise control over where pixels go, thus manipulating the pose, shape, expression, and layout of diverse categories such as animals, cars, humans, landscapes, etc.
Both qualitative and quantitative comparisons demonstrate the advantage of DragGAN over prior approaches in the tasks of image manipulation and point tracking.
arXiv Detail & Related papers (2023-05-18T13:41:25Z) - DUT: Learning Video Stabilization by Simply Watching Unstable Videos [86.88635774560017]
We propose a Deep Unsupervised Trajectory-based stabilization framework (DUT)
DUT makes the first attempt to stabilize unstable videos by explicitly estimating and smoothing trajectories in an unsupervised deep learning manner.
Experiment results on public benchmarks show that DUT outperforms representative state-of-the-art methods both qualitatively and quantitatively.
arXiv Detail & Related papers (2020-11-30T06:48:20Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.