FESTA: Flow Estimation via Spatial-Temporal Attention for Scene Point Clouds

• URL: http://arxiv.org/abs/2104.00798v1
• Date: Thu, 1 Apr 2021 23:04:04 GMT
• Title: FESTA: Flow Estimation via Spatial-Temporal Attention for Scene Point Clouds
• Authors: Haiyan Wang, Jiahao Pang, Muhammad A. Lodhi, Yingli Tian, Dong Tian
• Abstract summary: Scene flow depicts the dynamics of a 3D scene, which is critical for various applications such as autonomous driving, robot navigation, AR/VR, etc. It remains challenging to extract scene flow from point clouds due to sparsity and irregularity in typical point cloud sampling patterns. A novel Spatial Abstraction with Attention (SA2) layer is proposed to alleviate the unstable abstraction problem. A Temporal Abstraction with Attention (TA2) layer is proposed to rectify attention in temporal domain, leading to benefits with motions scaled in a larger range.
• Score: 28.899804787744202
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Scene flow depicts the dynamics of a 3D scene, which is critical for various applications such as autonomous driving, robot navigation, AR/VR, etc. Conventionally, scene flow is estimated from dense/regular RGB video frames. With the development of depth-sensing technologies, precise 3D measurements are available via point clouds which have sparked new research in 3D scene flow. Nevertheless, it remains challenging to extract scene flow from point clouds due to the sparsity and irregularity in typical point cloud sampling patterns. One major issue related to irregular sampling is identified as the randomness during point set abstraction/feature extraction -- an elementary process in many flow estimation scenarios. A novel Spatial Abstraction with Attention (SA^2) layer is accordingly proposed to alleviate the unstable abstraction problem. Moreover, a Temporal Abstraction with Attention (TA^2) layer is proposed to rectify attention in temporal domain, leading to benefits with motions scaled in a larger range. Extensive analysis and experiments verified the motivation and significant performance gains of our method, dubbed as Flow Estimation via Spatial-Temporal Attention (FESTA), when compared to several state-of-the-art benchmarks of scene flow estimation.

Related papers

• PointFlowHop: Green and Interpretable Scene Flow Estimation from Consecutive Point Clouds [49.7285297470392]
  An efficient 3D scene flow estimation method called PointFlowHop is proposed in this work. PointFlowHop takes two consecutive point clouds and determines the 3D flow vectors for every point in the first point cloud. It decomposes the scene flow estimation task into a set of subtasks, including ego-motion compensation, object association and object-wise motion estimation.
  arXiv  Detail & Related papers  (2023-02-27T23:06:01Z)
• What Matters for 3D Scene Flow Network [44.02710380584977]
  3D scene flow estimation from point clouds is a low-level 3D motion perception task in computer vision. We propose a novel all-to-all flow embedding layer with backward reliability validation during the initial scene flow estimation. Our proposed model surpasses all existing methods by at least 38.2% on FlyingThings3D dataset and 24.7% on KITTI Scene Flow dataset for EPE3D metric.
  arXiv  Detail & Related papers  (2022-07-19T09:27:05Z)
• RCP: Recurrent Closest Point for Scene Flow Estimation on 3D Point Clouds [44.034836961967144]
  3D motion estimation including scene flow and point cloud registration has drawn increasing interest. Recent methods employ deep neural networks to construct the cost volume for estimating accurate 3D flow. We decompose the problem into two interlaced stages, where the 3D flows are optimized point-wisely at the first stage and then globally regularized in a recurrent network at the second stage.
  arXiv  Detail & Related papers  (2022-05-23T04:04:30Z)
• Learning Scene Flow in 3D Point Clouds with Noisy Pseudo Labels [71.11151016581806]
  We propose a novel scene flow method that captures 3D motions from point clouds without relying on ground-truth scene flow annotations. Our method not only outperforms state-of-the-art self-supervised approaches, but also outperforms some supervised approaches that use accurate ground-truth flows.
  arXiv  Detail & Related papers  (2022-03-23T18:20:03Z)
• Residual 3D Scene Flow Learning with Context-Aware Feature Extraction [11.394559627312743]
  We propose a novel context-aware set conv layer to exploit contextual structure information of Euclidean space. We also propose an explicit residual flow learning structure in the residual flow refinement layer to cope with long-distance movement. Our method achieves state-of-the-art performance, surpassing all other previous works to the best of our knowledge by at least 25%.
  arXiv  Detail & Related papers  (2021-09-10T06:15:18Z)
• SCTN: Sparse Convolution-Transformer Network for Scene Flow Estimation [71.2856098776959]
  Estimating 3D motions for point clouds is challenging, since a point cloud is unordered and its density is significantly non-uniform. We propose a novel architecture named Sparse Convolution-Transformer Network (SCTN) that equips the sparse convolution with the transformer. We show that the learned relation-based contextual information is rich and helpful for matching corresponding points, benefiting scene flow estimation.
  arXiv  Detail & Related papers  (2021-05-10T15:16:14Z)
• Weakly Supervised Learning of Rigid 3D Scene Flow [81.37165332656612]
  We propose a data-driven scene flow estimation algorithm exploiting the observation that many 3D scenes can be explained by a collection of agents moving as rigid bodies. We showcase the effectiveness and generalization capacity of our method on four different autonomous driving datasets.
  arXiv  Detail & Related papers  (2021-02-17T18:58:02Z)
• Occlusion Guided Scene Flow Estimation on 3D Point Clouds [4.518012967046983]
  3D scene flow estimation is a vital tool in perceiving our environment given depth or range sensors. Here we propose a new scene flow architecture called OGSF-Net which tightly couples the learning for both flow and occlusions between frames. Their coupled symbiosis results in a more accurate prediction of flow in space.
  arXiv  Detail & Related papers  (2020-11-30T15:22:03Z)
• Scene Flow from Point Clouds with or without Learning [47.03163552693887]
  Scene flow is the three-dimensional (3D) motion field of a scene. Current learning-based approaches seek to estimate the scene flow directly from point clouds. We present a simple and interpretable objective function to recover the scene flow from point clouds.
  arXiv  Detail & Related papers  (2020-10-31T17:24:48Z)
• Hierarchical Attention Learning of Scene Flow in 3D Point Clouds [28.59260783047209]
  This paper studies the problem of scene flow estimation from two consecutive 3D point clouds. A novel hierarchical neural network with double attention is proposed for learning the correlation of point features in adjacent frames. Experiments show that the proposed network outperforms the state-of-the-art performance of 3D scene flow estimation.
  arXiv  Detail & Related papers  (2020-10-12T14:56:08Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.