Deep Virtual Markers for Articulated 3D Shapes

• URL: http://arxiv.org/abs/2108.09000v1
• Date: Fri, 20 Aug 2021 04:55:23 GMT
• Title: Deep Virtual Markers for Articulated 3D Shapes
• Authors: Hyomin Kim, Jungeon Kim, Jaewon Kam, Jaesik Park, Seungyong Lee
• Abstract summary: We propose a framework that maps 3D points of 3D articulated models, like humans, into virtual marker labels. We adopt a sparse convolutional neural network and classify 3D points of an articulated model into virtual marker labels. We show additional applications using the estimated virtual markers, such as non-rigid registration, texture transfer, and realtime dense marker prediction from depth maps.
• Score: 14.986945006208849
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We propose deep virtual markers, a framework for estimating dense and accurate positional information for various types of 3D data. We design a concept and construct a framework that maps 3D points of 3D articulated models, like humans, into virtual marker labels. To realize the framework, we adopt a sparse convolutional neural network and classify 3D points of an articulated model into virtual marker labels. We propose to use soft labels for the classifier to learn rich and dense interclass relationships based on geodesic distance. To measure the localization accuracy of the virtual markers, we test FAUST challenge, and our result outperforms the state-of-the-art. We also observe outstanding performance on the generalizability test, unseen data evaluation, and different 3D data types (meshes and depth maps). We show additional applications using the estimated virtual markers, such as non-rigid registration, texture transfer, and realtime dense marker prediction from depth maps.

Related papers

• MMScan: A Multi-Modal 3D Scene Dataset with Hierarchical Grounded Language Annotations [55.022519020409405]
  This paper builds the first largest ever multi-modal 3D scene dataset and benchmark with hierarchical grounded language annotations, MMScan. The resulting multi-modal 3D dataset encompasses 1.4M meta-annotated captions on 109k objects and 7.7k regions as well as over 3.04M diverse samples for 3D visual grounding and question-answering benchmarks.
  arXiv  Detail & Related papers  (2024-06-13T17:59:30Z)
• FaceLift: Semi-supervised 3D Facial Landmark Localization [6.191692539328364]
  We introduce a novel semi-supervised learning approach that learns 3D landmarks by directly lifting hand-labeled 2D landmarks. We leverage 3D-aware GANs for better multi-view consistency learning and in-the-wild multi-frame videos for robust cross-generalization.
  arXiv  Detail & Related papers  (2024-05-30T02:58:15Z)
• Volumetric Semantically Consistent 3D Panoptic Mapping [77.13446499924977]
  We introduce an online 2D-to-3D semantic instance mapping algorithm aimed at generating semantic 3D maps suitable for autonomous agents in unstructured environments. It introduces novel ways of integrating semantic prediction confidence during mapping, producing semantic and instance-consistent 3D regions. The proposed method achieves accuracy superior to the state of the art on public large-scale datasets, improving on a number of widely used metrics.
  arXiv  Detail & Related papers  (2023-09-26T08:03:10Z)
• 3D Human Mesh Estimation from Virtual Markers [34.703241940871635]
  We present an intermediate representation, named virtual markers, which learns 64 landmark keypoints on the body surface. Our approach outperforms the state-of-the-art methods on three datasets.
  arXiv  Detail & Related papers  (2023-03-21T10:30:43Z)
• Object-level 3D Semantic Mapping using a Network of Smart Edge Sensors [25.393382192511716]
  We extend a multi-view 3D semantic mapping system consisting of a network of distributed edge sensors with object-level information. Our method is evaluated on the public Behave dataset where it shows pose estimation within a few centimeters and in real-world experiments with the sensor network in a challenging lab environment.
  arXiv  Detail & Related papers  (2022-11-21T11:13:08Z)
• Back to Reality: Weakly-supervised 3D Object Detection with Shape-guided Label Enhancement [93.77156425817178]
  We propose a weakly-supervised approach for 3D object detection, which makes it possible to train strong 3D detector with position-level annotations. Our method, namely Back to Reality (BR), makes use of synthetic 3D shapes to convert the weak labels into fully-annotated virtual scenes. With less than 5% of the labeling labor, we achieve comparable detection performance with some popular fully-supervised approaches on the widely used ScanNet dataset.
  arXiv  Detail & Related papers  (2022-03-10T08:51:32Z)
• Monocular Quasi-Dense 3D Object Tracking [99.51683944057191]
  A reliable and accurate 3D tracking framework is essential for predicting future locations of surrounding objects and planning the observer's actions in numerous applications such as autonomous driving. We propose a framework that can effectively associate moving objects over time and estimate their full 3D bounding box information from a sequence of 2D images captured on a moving platform.
  arXiv  Detail & Related papers  (2021-03-12T15:30:02Z)
• H3D: Benchmark on Semantic Segmentation of High-Resolution 3D Point Clouds and textured Meshes from UAV LiDAR and Multi-View-Stereo [4.263987603222371]
  This paper introduces a 3D dataset which is unique in three ways. It depicts the village of Hessigheim (Germany) henceforth referred to as H3D. It is designed for promoting research in the field of 3D data analysis on one hand and to evaluate and rank emerging approaches.
  arXiv  Detail & Related papers  (2021-02-10T09:33:48Z)
• DH3D: Deep Hierarchical 3D Descriptors for Robust Large-Scale 6DoF Relocalization [56.15308829924527]
  We propose a Siamese network that jointly learns 3D local feature detection and description directly from raw 3D points. For detecting 3D keypoints we predict the discriminativeness of the local descriptors in an unsupervised manner. Experiments on various benchmarks demonstrate that our method achieves competitive results for both global point cloud retrieval and local point cloud registration.
  arXiv  Detail & Related papers  (2020-07-17T20:21:22Z)
• D3Feat: Joint Learning of Dense Detection and Description of 3D Local Features [51.04841465193678]
  We leverage a 3D fully convolutional network for 3D point clouds. We propose a novel and practical learning mechanism that densely predicts both a detection score and a description feature for each 3D point. Our method achieves state-of-the-art results in both indoor and outdoor scenarios.
  arXiv  Detail & Related papers  (2020-03-06T12:51:09Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.