Unsupervised Learning of Efficient Geometry-Aware Neural Articulated Representations

• URL: http://arxiv.org/abs/2204.08839v1
• Date: Tue, 19 Apr 2022 12:10:18 GMT
• Title: Unsupervised Learning of Efficient Geometry-Aware Neural Articulated Representations
• Authors: Atsuhiro Noguchi, Xiao Sun, Stephen Lin, Tatsuya Harada
• Abstract summary: We propose an unsupervised method for 3D geometry-aware representation learning of articulated objects. We obviate this need by learning the representations with GAN training. Experiments demonstrate the efficiency of our method and show that GAN-based training enables learning of controllable 3D representations without supervision.
• Score: 89.1388369229542
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We propose an unsupervised method for 3D geometry-aware representation learning of articulated objects. Though photorealistic images of articulated objects can be rendered with explicit pose control through existing 3D neural representations, these methods require ground truth 3D pose and foreground masks for training, which are expensive to obtain. We obviate this need by learning the representations with GAN training. From random poses and latent vectors, the generator is trained to produce realistic images of articulated objects by adversarial training. To avoid a large computational cost for GAN training, we propose an efficient neural representation for articulated objects based on tri-planes and then present a GAN-based framework for its unsupervised training. Experiments demonstrate the efficiency of our method and show that GAN-based training enables learning of controllable 3D representations without supervision.

Related papers

• SUGAR: Pre-training 3D Visual Representations for Robotics [85.55534363501131]
  We introduce a novel 3D pre-training framework for robotics named SUGAR. SUGAR captures semantic, geometric and affordance properties of objects through 3D point clouds. We show that SUGAR's 3D representation outperforms state-of-the-art 2D and 3D representations.
  arXiv  Detail & Related papers  (2024-04-01T21:23:03Z)
• Motion Degeneracy in Self-supervised Learning of Elevation Angle Estimation for 2D Forward-Looking Sonar [4.683630397028384]
  This study aims to realize stable self-supervised learning of elevation angle estimation without pretraining using synthetic images. We first analyze the motion field of 2D forward-looking sonar, which is related to the main supervision signal.
  arXiv  Detail & Related papers  (2023-07-30T08:06:11Z)
• Farm3D: Learning Articulated 3D Animals by Distilling 2D Diffusion [67.71624118802411]
  We present Farm3D, a method for learning category-specific 3D reconstructors for articulated objects. We propose a framework that uses an image generator, such as Stable Diffusion, to generate synthetic training data. Our network can be used for analysis, including monocular reconstruction, or for synthesis, generating articulated assets for real-time applications such as video games.
  arXiv  Detail & Related papers  (2023-04-20T17:59:34Z)
• Visual Reinforcement Learning with Self-Supervised 3D Representations [15.991546692872841]
  We present a unified framework for self-supervised learning of 3D representations for motor control. Our method enjoys improved sample efficiency in simulated manipulation tasks compared to 2D representation learning methods.
  arXiv  Detail & Related papers  (2022-10-13T17:59:55Z)
• De-rendering 3D Objects in the Wild [21.16153549406485]
  We present a weakly supervised method that is able to decompose a single image of an object into shape. For training, the method only relies on a rough initial shape estimate of the training objects to bootstrap the learning process. In our experiments, we show that the method can successfully de-render 2D images into a 3D representation and generalizes to unseen object categories.
  arXiv  Detail & Related papers  (2022-01-06T23:50:09Z)
• Neural Articulated Radiance Field [90.91714894044253]
  We present Neural Articulated Radiance Field (NARF), a novel deformable 3D representation for articulated objects learned from images. Experiments show that the proposed method is efficient and can generalize well to novel poses.
  arXiv  Detail & Related papers  (2021-04-07T13:23:14Z)
• Image GANs meet Differentiable Rendering for Inverse Graphics and Interpretable 3D Neural Rendering [101.56891506498755]
  Differentiable rendering has paved the way to training neural networks to perform "inverse graphics" tasks. We show that our approach significantly outperforms state-of-the-art inverse graphics networks trained on existing datasets.
  arXiv  Detail & Related papers  (2020-10-18T22:29:07Z)
• Chained Representation Cycling: Learning to Estimate 3D Human Pose and Shape by Cycling Between Representations [73.11883464562895]
  We propose a new architecture that facilitates unsupervised, or lightly supervised, learning. We demonstrate the method by learning 3D human pose and shape from un-paired and un-annotated images. While we present results for modeling humans, our formulation is general and can be applied to other vision problems.
  arXiv  Detail & Related papers  (2020-01-06T14:54:00Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.