KeypointDeformer: Unsupervised 3D Keypoint Discovery for Shape Control

• URL: http://arxiv.org/abs/2104.11224v1
• Date: Thu, 22 Apr 2021 17:59:08 GMT
• Title: KeypointDeformer: Unsupervised 3D Keypoint Discovery for Shape Control
• Authors: Tomas Jakab, Richard Tucker, Ameesh Makadia, Jiajun Wu, Noah Snavely, Angjoo Kanazawa
• Abstract summary: KeypointDeformer is an unsupervised method for shape control through automatically discovered 3D keypoints. Our approach produces intuitive and semantically consistent control of shape deformations.
• Score: 64.46042014759671
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We introduce KeypointDeformer, a novel unsupervised method for shape control through automatically discovered 3D keypoints. We cast this as the problem of aligning a source 3D object to a target 3D object from the same object category. Our method analyzes the difference between the shapes of the two objects by comparing their latent representations. This latent representation is in the form of 3D keypoints that are learned in an unsupervised way. The difference between the 3D keypoints of the source and the target objects then informs the shape deformation algorithm that deforms the source object into the target object. The whole model is learned end-to-end and simultaneously discovers 3D keypoints while learning to use them for deforming object shapes. Our approach produces intuitive and semantically consistent control of shape deformations. Moreover, our discovered 3D keypoints are consistent across object category instances despite large shape variations. As our method is unsupervised, it can be readily deployed to new object categories without requiring annotations for 3D keypoints and deformations.

Related papers

• SNAKE: Shape-aware Neural 3D Keypoint Field [62.91169625183118]
  Detecting 3D keypoints from point clouds is important for shape reconstruction. This work investigates the dual question: can shape reconstruction benefit 3D keypoint detection? We propose a novel unsupervised paradigm named SNAKE, which is short for shape-aware neural 3D keypoint field.
  arXiv  Detail & Related papers  (2022-06-03T17:58:43Z)
• End-to-End Learning of Multi-category 3D Pose and Shape Estimation [128.881857704338]
  We propose an end-to-end method that simultaneously detects 2D keypoints from an image and lifts them to 3D. The proposed method learns both 2D detection and 3D lifting only from 2D keypoints annotations. In addition to being end-to-end in image to 3D learning, our method also handles objects from multiple categories using a single neural network.
  arXiv  Detail & Related papers  (2021-12-19T17:10:40Z)
• AutoShape: Real-Time Shape-Aware Monocular 3D Object Detection [15.244852122106634]
  We propose an approach for incorporating the shape-aware 2D/3D constraints into the 3D detection framework. Specifically, we employ the deep neural network to learn distinguished 2D keypoints in the 2D image domain. For generating the ground truth of 2D/3D keypoints, an automatic model-fitting approach has been proposed.
  arXiv  Detail & Related papers  (2021-08-25T08:50:06Z)
• Learning Canonical 3D Object Representation for Fine-Grained Recognition [77.33501114409036]
  We propose a novel framework for fine-grained object recognition that learns to recover object variation in 3D space from a single image. We represent an object as a composition of 3D shape and its appearance, while eliminating the effect of camera viewpoint. By incorporating 3D shape and appearance jointly in a deep representation, our method learns the discriminative representation of the object.
  arXiv  Detail & Related papers  (2021-08-10T12:19:34Z)
• DeformerNet: A Deep Learning Approach to 3D Deformable Object Manipulation [5.733365759103406]
  We propose a novel approach to 3D deformable object manipulation leveraging a deep neural network called DeformerNet. We explicitly use 3D point clouds as the state representation and apply Convolutional Neural Network on point clouds to learn the 3D features. Once trained in an end-to-end fashion, DeformerNet directly maps the current point cloud of a deformable object, as well as a target point cloud shape, to the desired displacement in robot gripper position.
  arXiv  Detail & Related papers  (2021-07-16T18:20:58Z)
• Group-Free 3D Object Detection via Transformers [26.040378025818416]
  We present a simple yet effective method for directly detecting 3D objects from the 3D point cloud. Our method computes the feature of an object from all the points in the point cloud with the help of an attention mechanism in the Transformers citevaswaniattention. With few bells and whistles, the proposed method achieves state-of-the-art 3D object detection performance on two widely used benchmarks, ScanNet V2 and SUN RGB-D.
  arXiv  Detail & Related papers  (2021-04-01T17:59:36Z)
• DOPS: Learning to Detect 3D Objects and Predict their 3D Shapes [54.239416488865565]
  We propose a fast single-stage 3D object detection method for LIDAR data. The core novelty of our method is a fast, single-pass architecture that both detects objects in 3D and estimates their shapes. We find that our proposed method achieves state-of-the-art results by 5% on object detection in ScanNet scenes, and it gets top results by 3.4% in the Open dataset.
  arXiv  Detail & Related papers  (2020-04-02T17:48:50Z)
• Unsupervised Learning of Category-Specific Symmetric 3D Keypoints from Point Sets [71.84892018102465]
  This paper aims at learning category-specific 3D keypoints, in an unsupervised manner, using a collection of misaligned 3D point clouds of objects from an unknown category. To the best of our knowledge, this is the first work on learning such keypoints directly from 3D point clouds.
  arXiv  Detail & Related papers  (2020-03-17T10:28:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.