Deformable Generator Networks: Unsupervised Disentanglement of Appearance and Geometry

• URL: http://arxiv.org/abs/1806.06298v4
• Date: Fri, 6 Oct 2023 08:41:14 GMT
• Title: Deformable Generator Networks: Unsupervised Disentanglement of Appearance and Geometry
• Authors: Xianglei Xing, Ruiqi Gao, Tian Han, Song-Chun Zhu, Ying Nian Wu
• Abstract summary: We present a deformable generator model to disentangle the appearance and geometric information for both image and video data. The appearance generator network models the information related to appearance, including color, illumination, identity or category. The geometric generator performs geometric warping, such as rotation and stretching, through generating deformation field.
• Score: 93.02523642523477
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We present a deformable generator model to disentangle the appearance and geometric information for both image and video data in a purely unsupervised manner. The appearance generator network models the information related to appearance, including color, illumination, identity or category, while the geometric generator performs geometric warping, such as rotation and stretching, through generating deformation field which is used to warp the generated appearance to obtain the final image or video sequences. Two generators take independent latent vectors as input to disentangle the appearance and geometric information from image or video sequences. For video data, a nonlinear transition model is introduced to both the appearance and geometric generators to capture the dynamics over time. The proposed scheme is general and can be easily integrated into different generative models. An extensive set of qualitative and quantitative experiments shows that the appearance and geometric information can be well disentangled, and the learned geometric generator can be conveniently transferred to other image datasets to facilitate knowledge transfer tasks.

Related papers

• Geometry Distributions [51.4061133324376]
  We propose a novel geometric data representation that models geometry as distributions. Our approach uses diffusion models with a novel network architecture to learn surface point distributions. We evaluate our representation qualitatively and quantitatively across various object types, demonstrating its effectiveness in achieving high geometric fidelity.
  arXiv  Detail & Related papers  (2024-11-25T04:06:48Z)
• Generative Topology for Shape Synthesis [13.608942872770855]
  We develop a novel framework for shape generation tasks on point clouds. Our model exhibits high quality in reconstruction and generation tasks, affords efficient latent-space, and is orders of magnitude faster than existing methods.
  arXiv  Detail & Related papers  (2024-10-09T17:19:22Z)
• Learning Geometric Invariant Features for Classification of Vector Polygons with Graph Message-passing Neural Network [3.804240190982697]
  We propose a novel graph message-passing neural network (PolyMP) to learn the geometric-invariant features for shape classification of polygons. We show that the proposed graph-based PolyMP network enables the learning of expressive geometric features invariant to geometric transformations of polygons.
  arXiv  Detail & Related papers  (2024-07-05T08:19:36Z)
• Varying Manifolds in Diffusion: From Time-varying Geometries to Visual Saliency [25.632973225129728]
  We study the geometric properties of the diffusion model, whose forward diffusion process and reverse generation process construct a series of distributions on a manifold. We show that the generation rate is highly correlated with intuitive visual properties, such as visual saliency, of the image component. We propose an efficient and differentiable scheme to estimate the generation rate for a given image component over time, giving rise to a generation curve.
  arXiv  Detail & Related papers  (2024-06-07T07:32:41Z)
• A Survey of Geometric Graph Neural Networks: Data Structures, Models and Applications [67.33002207179923]
  This paper presents a survey of data structures, models, and applications related to geometric GNNs. We provide a unified view of existing models from the geometric message passing perspective. We also summarize the applications as well as the related datasets to facilitate later research for methodology development and experimental evaluation.
  arXiv  Detail & Related papers  (2024-03-01T12:13:04Z)
• Curve Your Attention: Mixed-Curvature Transformers for Graph Representation Learning [77.1421343649344]
  We propose a generalization of Transformers towards operating entirely on the product of constant curvature spaces. We also provide a kernelized approach to non-Euclidean attention, which enables our model to run in time and memory cost linear to the number of nodes and edges.
  arXiv  Detail & Related papers  (2023-09-08T02:44:37Z)
• Geo-SIC: Learning Deformable Geometric Shapes in Deep Image Classifiers [8.781861951759948]
  This paper presents Geo-SIC, the first deep learning model to learn deformable shapes in a deformation space for an improved performance of image classification. We introduce a newly designed framework that (i) simultaneously derives features from both image and latent shape spaces with large intra-class variations. We develop a boosted classification network, equipped with an unsupervised learning of geometric shape representations.
  arXiv  Detail & Related papers  (2022-10-25T01:55:17Z)
• Generative Deformable Radiance Fields for Disentangled Image Synthesis of Topology-Varying Objects [52.46838926521572]
  3D-aware generative models have demonstrated their superb performance to generate 3D neural radiance fields (NeRF) from a collection of monocular 2D images. We propose a generative model for synthesizing radiance fields of topology-varying objects with disentangled shape and appearance variations.
  arXiv  Detail & Related papers  (2022-09-09T08:44:06Z)
• Image Generators with Conditionally-Independent Pixel Synthesis [10.792933031825527]
  We present a new architecture for image generators, where the color value at each pixel is computed independently. No spatial convolutions or similar operations that propagate information across pixels are involved during the synthesis. We analyze the modeling capabilities of such generators when trained in an adversarial fashion, and observe the new generators to achieve similar generation quality to state-of-the-art convolutional generators.
  arXiv  Detail & Related papers  (2020-11-27T15:16:11Z)
• Fine-grained Image-to-Image Transformation towards Visual Recognition [102.51124181873101]
  We aim at transforming an image with a fine-grained category to synthesize new images that preserve the identity of the input image. We adopt a model based on generative adversarial networks to disentangle the identity related and unrelated factors of an image. Experiments on the CompCars and Multi-PIE datasets demonstrate that our model preserves the identity of the generated images much better than the state-of-the-art image-to-image transformation models.
  arXiv  Detail & Related papers  (2020-01-12T05:26:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.