NICP: Neural ICP for 3D Human Registration at Scale

• URL: http://arxiv.org/abs/2312.14024v3
• Date: Sun, 21 Jul 2024 14:21:58 GMT
• Title: NICP: Neural ICP for 3D Human Registration at Scale
• Authors: Riccardo Marin, Enric Corona, Gerard Pons-Moll,
• Abstract summary: We propose a neural scalable registration method, NSR, for 3D Human registration. NSR generalizes and scales across thousands of shapes and more than ten different data sources. Our essential contribution is NICP, an ICP-style self-supervised task tailored to neural fields.
• Score: 35.631505786332454
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Aligning a template to 3D human point clouds is a long-standing problem crucial for tasks like animation, reconstruction, and enabling supervised learning pipelines. Recent data-driven methods leverage predicted surface correspondences. However, they are not robust to varied poses, identities, or noise. In contrast, industrial solutions often rely on expensive manual annotations or multi-view capturing systems. Recently, neural fields have shown promising results. Still, their purely data-driven and extrinsic nature does not incorporate any guidance toward the target surface, often resulting in a trivial misalignment of the template registration. Currently, no method can be considered the standard for 3D Human registration, limiting the scalability of downstream applications. In this work, we propose a neural scalable registration method, NSR, a pipeline that, for the first time, generalizes and scales across thousands of shapes and more than ten different data sources. Our essential contribution is NICP, an ICP-style self-supervised task tailored to neural fields. NSR takes a few seconds, is self-supervised, and works out of the box on pre-trained neural fields. NSR combines NICP with a localized neural field trained on a large MoCap dataset, achieving the state of the art over public benchmarks. The release of our code and checkpoints provides a powerful tool useful for many downstream tasks like dataset alignments, cleaning, or asset animation.

Related papers

• Efficient 3D Recognition with Event-driven Spike Sparse Convolution [15.20476631850388]
  Spiking Neural Networks (SNNs) provide an energy-efficient way to extract 3D-temporal features. We introduce the Spike Voxel Coding (SVC) scheme, which encodes the 3D point clouds into a sparse spike train space. We propose a Spike Sparse Convolution (SSC) model for efficiently extracting 3D sparse point cloud features.
  arXiv  Detail & Related papers  (2024-12-10T09:55:15Z)
• Generalized Label-Efficient 3D Scene Parsing via Hierarchical Feature Aligned Pre-Training and Region-Aware Fine-tuning [55.517000360348725]
  This work presents a framework for dealing with 3D scene understanding when the labeled scenes are quite limited. To extract knowledge for novel categories from the pre-trained vision-language models, we propose a hierarchical feature-aligned pre-training and knowledge distillation strategy. Experiments with both indoor and outdoor scenes demonstrated the effectiveness of our approach in both data-efficient learning and open-world few-shot learning.
  arXiv  Detail & Related papers  (2023-12-01T15:47:04Z)
• MLGCN: An Ultra Efficient Graph Convolution Neural Model For 3D Point Cloud Analysis [4.947552172739438]
  We introduce a novel Multi-level Graph Convolution Neural (MLGCN) model, which uses Graph Neural Networks (GNN) blocks to extract features from 3D point clouds at specific locality levels. Our approach produces comparable results to those of state-of-the-art models while requiring up to a thousand times fewer floating-point operations (FLOPs) and having significantly reduced storage requirements.
  arXiv  Detail & Related papers  (2023-03-31T00:15:22Z)
• Versatile Neural Processes for Learning Implicit Neural Representations [57.090658265140384]
  We propose Versatile Neural Processes (VNP), which largely increases the capability of approximating functions. Specifically, we introduce a bottleneck encoder that produces fewer and informative context tokens, relieving the high computational cost. We demonstrate the effectiveness of the proposed VNP on a variety of tasks involving 1D, 2D and 3D signals.
  arXiv  Detail & Related papers  (2023-01-21T04:08:46Z)
• ALSO: Automotive Lidar Self-supervision by Occupancy estimation [70.70557577874155]
  We propose a new self-supervised method for pre-training the backbone of deep perception models operating on point clouds. The core idea is to train the model on a pretext task which is the reconstruction of the surface on which the 3D points are sampled. The intuition is that if the network is able to reconstruct the scene surface, given only sparse input points, then it probably also captures some fragments of semantic information.
  arXiv  Detail & Related papers  (2022-12-12T13:10:19Z)
• Self-Supervised Learning with Multi-View Rendering for 3D Point Cloud Analysis [33.31864436614945]
  We propose a novel pre-training method for 3D point cloud models. Our pre-training is self-supervised by a local pixel/point level correspondence loss and a global image/point cloud level loss. These improved models outperform existing state-of-the-art methods on various datasets and downstream tasks.
  arXiv  Detail & Related papers  (2022-10-28T05:23:03Z)
• PointAttN: You Only Need Attention for Point Cloud Completion [89.88766317412052]
  Point cloud completion refers to completing 3D shapes from partial 3D point clouds. We propose a novel neural network for processing point cloud in a per-point manner to eliminate kNNs. The proposed framework, namely PointAttN, is simple, neat and effective, which can precisely capture the structural information of 3D shapes.
  arXiv  Detail & Related papers  (2022-03-16T09:20:01Z)
• Semi-supervised 3D Object Detection via Temporal Graph Neural Networks [17.90796183565084]
  3D object detection plays an important role in autonomous driving and other robotics applications. We propose leveraging large amounts of unlabeled point cloud videos by semi-supervised learning of 3D object detectors. Our method achieves state-of-the-art detection performance on the challenging nuScenes and H3D benchmarks.
  arXiv  Detail & Related papers  (2022-02-01T02:06:54Z)
• Keypoint Message Passing for Video-based Person Re-Identification [106.41022426556776]
  Video-based person re-identification (re-ID) is an important technique in visual surveillance systems which aims to match video snippets of people captured by different cameras. Existing methods are mostly based on convolutional neural networks (CNNs), whose building blocks either process local neighbor pixels at a time, or, when 3D convolutions are used to model temporal information, suffer from the misalignment problem caused by person movement. In this paper, we propose to overcome the limitations of normal convolutions with a human-oriented graph method. Specifically, features located at person joint keypoints are extracted and connected as a spatial-temporal graph
  arXiv  Detail & Related papers  (2021-11-16T08:01:16Z)
• 2nd Place Scheme on Action Recognition Track of ECCV 2020 VIPriors Challenges: An Efficient Optical Flow Stream Guided Framework [57.847010327319964]
  We propose a data-efficient framework that can train the model from scratch on small datasets. Specifically, by introducing a 3D central difference convolution operation, we proposed a novel C3D neural network-based two-stream framework. It is proved that our method can achieve a promising result even without a pre-trained model on large scale datasets.
  arXiv  Detail & Related papers  (2020-08-10T09:50:28Z)
• Self-Supervised Feature Extraction for 3D Axon Segmentation [7.181047714452116]
  Existing learning-based methods to automatically trace axons in 3D brain imagery often rely on manually annotated segmentation labels. We propose a self-supervised auxiliary task that utilizes the tube-like structure of axons to build a feature extractor from unlabeled data. We demonstrate improved segmentation performance over the 3D U-Net model on both the SHIELD PVGPe dataset and the BigNeuron Project, single neuron Janelia dataset.
  arXiv  Detail & Related papers  (2020-04-20T20:46:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.