Intuitive Shape Editing in Latent Space
- URL: http://arxiv.org/abs/2111.12488v1
- Date: Wed, 24 Nov 2021 13:33:10 GMT
- Title: Intuitive Shape Editing in Latent Space
- Authors: Tim Elsner, Moritz Ibing, Victor Czech, Julius Nehring-Wirxel, Leif
Kobbelt
- Abstract summary: We present an autoencoder-based method that enables intuitive shape editing in latent space by disentangling latent sub-spaces.
We evaluate our method by comparing it to state-of-the-art data-driven shape editing methods.
- Score: 9.034665429931406
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: The use of autoencoders for shape generation and editing suffers from
manipulations in latent space that may lead to unpredictable changes in the
output shape. We present an autoencoder-based method that enables intuitive
shape editing in latent space by disentangling latent sub-spaces to obtain
control points on the surface and style variables that can be manipulated
independently. The key idea is adding a Lipschitz-type constraint to the loss
function, i.e. bounding the change of the output shape proportionally to the
change in latent space, leading to interpretable latent space representations.
The control points on the surface can then be freely moved around, allowing for
intuitive shape editing directly in latent space. We evaluate our method by
comparing it to state-of-the-art data-driven shape editing methods. Besides
shape manipulation, we demonstrate the expressiveness of our control points by
leveraging them for unsupervised part segmentation.
Related papers
- Drag Your Gaussian: Effective Drag-Based Editing with Score Distillation for 3D Gaussian Splatting [55.14822004410817]
We introduce DYG, an effective 3D drag-based editing method for 3D Gaussian Splatting.
It enables precise control over the extent of editing through the input of 3D masks and pairs of control points.
DYG integrates the strengths of the implicit triplane representation to establish the geometric scaffold of the editing results.
arXiv Detail & Related papers (2025-01-30T18:51:54Z) - MagicStick: Controllable Video Editing via Control Handle Transformations [49.29608051543133]
MagicStick is a controllable video editing method that edits the video properties by utilizing the transformation on the extracted internal control signals.
We present experiments on numerous examples within our unified framework.
We also compare with shape-aware text-based editing and handcrafted motion video generation, demonstrating our superior temporal consistency and editing capability than previous works.
arXiv Detail & Related papers (2023-12-05T17:58:06Z) - Shape-aware Text-driven Layered Video Editing [39.56765973770167]
We present a shape-aware, text-driven video editing method to handle shape changes.
We first propagate the deformation field between the input and edited to all frames.
We then leverage a pre-trained text-conditioned diffusion model as guidance for refining shape distortion and completing unseen regions.
arXiv Detail & Related papers (2023-01-30T18:41:58Z) - Spatial Steerability of GANs via Self-Supervision from Discriminator [123.27117057804732]
We propose a self-supervised approach to improve the spatial steerability of GANs without searching for steerable directions in the latent space.
Specifically, we design randomly sampled Gaussian heatmaps to be encoded into the intermediate layers of generative models as spatial inductive bias.
During inference, users can interact with the spatial heatmaps in an intuitive manner, enabling them to edit the output image by adjusting the scene layout, moving, or removing objects.
arXiv Detail & Related papers (2023-01-20T07:36:29Z) - Shape-Guided Diffusion with Inside-Outside Attention [60.557437251084465]
We introduce precise object silhouette as a new form of user control in text-to-image diffusion models.
Our training-free method uses an Inside-Outside Attention mechanism to apply a shape constraint to the cross- and self-attention maps.
arXiv Detail & Related papers (2022-12-01T01:39:28Z) - DeepMLS: Geometry-Aware Control Point Deformation [76.51312491336343]
We introduce DeepMLS, a space-based deformation technique, guided by a set of displaced control points.
We leverage the power of neural networks to inject the underlying shape geometry into the deformation parameters.
Our technique facilitates intuitive piecewise smooth deformations, which are well suited for manufactured objects.
arXiv Detail & Related papers (2022-01-05T23:55:34Z) - StylePart: Image-based Shape Part Manipulation [12.441476696381814]
StylePart is a framework that enables direct shape manipulation of an image by leveraging generative models of both images and 3D shapes.
Our key contribution is a shape-consistent latent mapping function that connects the image generative latent space and the 3D man-made shape attribute latent space.
We demonstrate our approach through various manipulation tasks, including part replacement, part resizing, and viewpoint manipulation.
arXiv Detail & Related papers (2021-11-20T05:30:08Z) - EditVAE: Unsupervised Part-Aware Controllable 3D Point Cloud Shape
Generation [19.817166425038753]
This paper tackles the problem of parts-aware point cloud generation.
A simple modification of the Variational Auto-Encoder yields a joint model of the point cloud itself.
In addition to the flexibility afforded by our disentangled representation, the inductive bias introduced by our joint modelling approach yields the state-of-the-art experimental results on the ShapeNet dataset.
arXiv Detail & Related papers (2021-10-13T12:38:01Z) - Learning to Infer Semantic Parameters for 3D Shape Editing [14.902766305317202]
We learn a deep network that infers the semantic parameters of an input shape and then allows the user to manipulate those parameters.
The network is trained jointly on shapes from an auxiliary synthetic template and unlabeled realistic models.
Experiments with datasets of chairs, airplanes, and human bodies demonstrate that our method produces more natural edits than prior work.
arXiv Detail & Related papers (2020-11-09T20:58:49Z) - Learning to Manipulate Individual Objects in an Image [71.55005356240761]
We describe a method to train a generative model with latent factors that are independent and localized.
This means that perturbing the latent variables affects only local regions of the synthesized image, corresponding to objects.
Unlike other unsupervised generative models, ours enables object-centric manipulation, without requiring object-level annotations.
arXiv Detail & Related papers (2020-04-11T21:50:20Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.