Adversarial Self-Supervised Scene Flow Estimation

• URL: http://arxiv.org/abs/2011.00551v1
• Date: Sun, 1 Nov 2020 16:37:37 GMT
• Title: Adversarial Self-Supervised Scene Flow Estimation
• Authors: Victor Zuanazzi, Joris van Vugt, Olaf Booij and Pascal Mettes
• Abstract summary: This work proposes a metric learning approach for self-supervised scene flow estimation. We outline a benchmark for self-supervised scene flow estimation: the Scene Flow Sandbox.
• Score: 15.278302535191866
• License: http://creativecommons.org/publicdomain/zero/1.0/
• Abstract: This work proposes a metric learning approach for self-supervised scene flow estimation. Scene flow estimation is the task of estimating 3D flow vectors for consecutive 3D point clouds. Such flow vectors are fruitful, \eg for recognizing actions, or avoiding collisions. Training a neural network via supervised learning for scene flow is impractical, as this requires manual annotations for each 3D point at each new timestamp for each scene. To that end, we seek for a self-supervised approach, where a network learns a latent metric to distinguish between points translated by flow estimations and the target point cloud. Our adversarial metric learning includes a multi-scale triplet loss on sequences of two-point clouds as well as a cycle consistency loss. Furthermore, we outline a benchmark for self-supervised scene flow estimation: the Scene Flow Sandbox. The benchmark consists of five datasets designed to study individual aspects of flow estimation in progressive order of complexity, from a moving object to real-world scenes. Experimental evaluation on the benchmark shows that our approach obtains state-of-the-art self-supervised scene flow results, outperforming recent neighbor-based approaches. We use our proposed benchmark to expose shortcomings and draw insights on various training setups. We find that our setup captures motion coherence and preserves local geometries. Dealing with occlusions, on the other hand, is still an open challenge.

Related papers

• PointFlowHop: Green and Interpretable Scene Flow Estimation from Consecutive Point Clouds [49.7285297470392]
  An efficient 3D scene flow estimation method called PointFlowHop is proposed in this work. PointFlowHop takes two consecutive point clouds and determines the 3D flow vectors for every point in the first point cloud. It decomposes the scene flow estimation task into a set of subtasks, including ego-motion compensation, object association and object-wise motion estimation.
  arXiv  Detail & Related papers  (2023-02-27T23:06:01Z)
• SCOOP: Self-Supervised Correspondence and Optimization-Based Scene Flow [25.577386156273256]
  Scene flow estimation is a long-standing problem in computer vision, where the goal is to find the 3D motion of a scene from its consecutive observations. We introduce SCOOP, a new method for scene flow estimation that can be learned on a small amount of data without employing ground-truth flow supervision.
  arXiv  Detail & Related papers  (2022-11-25T10:52:02Z)
• Learning Scene Flow in 3D Point Clouds with Noisy Pseudo Labels [71.11151016581806]
  We propose a novel scene flow method that captures 3D motions from point clouds without relying on ground-truth scene flow annotations. Our method not only outperforms state-of-the-art self-supervised approaches, but also outperforms some supervised approaches that use accurate ground-truth flows.
  arXiv  Detail & Related papers  (2022-03-23T18:20:03Z)
• Residual 3D Scene Flow Learning with Context-Aware Feature Extraction [11.394559627312743]
  We propose a novel context-aware set conv layer to exploit contextual structure information of Euclidean space. We also propose an explicit residual flow learning structure in the residual flow refinement layer to cope with long-distance movement. Our method achieves state-of-the-art performance, surpassing all other previous works to the best of our knowledge by at least 25%.
  arXiv  Detail & Related papers  (2021-09-10T06:15:18Z)
• Self-Point-Flow: Self-Supervised Scene Flow Estimation from Point Clouds with Optimal Transport and Random Walk [59.87525177207915]
  We develop a self-supervised method to establish correspondences between two point clouds to approximate scene flow. Our method achieves state-of-the-art performance among self-supervised learning methods.
  arXiv  Detail & Related papers  (2021-05-18T03:12:42Z)
• SCTN: Sparse Convolution-Transformer Network for Scene Flow Estimation [71.2856098776959]
  Estimating 3D motions for point clouds is challenging, since a point cloud is unordered and its density is significantly non-uniform. We propose a novel architecture named Sparse Convolution-Transformer Network (SCTN) that equips the sparse convolution with the transformer. We show that the learned relation-based contextual information is rich and helpful for matching corresponding points, benefiting scene flow estimation.
  arXiv  Detail & Related papers  (2021-05-10T15:16:14Z)
• Occlusion Guided Self-supervised Scene Flow Estimation on 3D Point Clouds [4.518012967046983]
  Understanding the flow in 3D space of sparsely sampled points between two consecutive time frames is the core stone of modern geometric-driven systems. This work presents a new self-supervised training method and an architecture for the 3D scene flow estimation under occlusions.
  arXiv  Detail & Related papers  (2021-04-10T09:55:19Z)
• Weakly Supervised Learning of Rigid 3D Scene Flow [81.37165332656612]
  We propose a data-driven scene flow estimation algorithm exploiting the observation that many 3D scenes can be explained by a collection of agents moving as rigid bodies. We showcase the effectiveness and generalization capacity of our method on four different autonomous driving datasets.
  arXiv  Detail & Related papers  (2021-02-17T18:58:02Z)
• Do not trust the neighbors! Adversarial Metric Learning for Self-Supervised Scene Flow Estimation [0.0]
  Scene flow is the task of estimating 3D motion vectors to individual points of a dynamic 3D scene. We propose a 3D scene flow benchmark and a novel self-supervised setup for training flow models. We find that our setup is able to keep motion coherence and preserve local geometries, which many self-supervised baselines fail to grasp.
  arXiv  Detail & Related papers  (2020-11-01T17:41:32Z)
• Hierarchical Attention Learning of Scene Flow in 3D Point Clouds [28.59260783047209]
  This paper studies the problem of scene flow estimation from two consecutive 3D point clouds. A novel hierarchical neural network with double attention is proposed for learning the correlation of point features in adjacent frames. Experiments show that the proposed network outperforms the state-of-the-art performance of 3D scene flow estimation.
  arXiv  Detail & Related papers  (2020-10-12T14:56:08Z)
• Self-Supervised Learning of Non-Rigid Residual Flow and Ego-Motion [63.18340058854517]
  We present an alternative method for end-to-end scene flow learning by joint estimation of non-rigid residual flow and ego-motion flow for dynamic 3D scenes. We extend the supervised framework with self-supervisory signals based on the temporal consistency property of a point cloud sequence.
  arXiv  Detail & Related papers  (2020-09-22T11:39:19Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.