Motion and Structure from Event-based Normal Flow

• URL: http://arxiv.org/abs/2407.12239v2
• Date: Fri, 19 Jul 2024 11:13:16 GMT
• Title: Motion and Structure from Event-based Normal Flow
• Authors: Zhongyang Ren, Bangyan Liao, Delei Kong, Jinghang Li, Peidong Liu, Laurent Kneip, Guillermo Gallego, Yi Zhou,
• Abstract summary: Neuromorphic event-based cameras place great demands on approaches that use raw event data as input to solve this fundamental problem. Existing state-of-the-art solutions typically infer implicitly data association by iteratively reversing the event data generation process. We show that the event-based normal flow can be used, via the proposed geometric error term, as an alternative to the full flow in solving a family of geometric problems.
• Score: 26.513167481193225
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Recovering the camera motion and scene geometry from visual data is a fundamental problem in the field of computer vision. Its success in standard vision is attributed to the maturity of feature extraction, data association and multi-view geometry. The recent emergence of neuromorphic event-based cameras places great demands on approaches that use raw event data as input to solve this fundamental problem.Existing state-of-the-art solutions typically infer implicitly data association by iteratively reversing the event data generation process. However, the nonlinear nature of these methods limits their applicability in real-time tasks, and the constant-motion assumption leads to unstable results under agile motion. To this end, we rethink the problem formulation in a way that aligns better with the differential working principle of event cameras.We show that the event-based normal flow can be used, via the proposed geometric error term, as an alternative to the full flow in solving a family of geometric problems that involve instantaneous first-order kinematics and scene geometry. Furthermore, we develop a fast linear solver and a continuous-time nonlinear solver on top of the proposed geometric error term.Experiments on both synthetic and real data show the superiority of our linear solver in terms of accuracy and efficiency, and indicate its complementary feature as an initialization method for existing nonlinear solvers. Besides, our continuous-time non-linear solver exhibits exceptional capability in accommodating sudden variations in motion since it does not rely on the constant-motion assumption.

Related papers

• A model-constrained Discontinuous Galerkin Network (DGNet) for Compressible Euler Equations with Out-of-Distribution Generalization [0.0]
  We develop a model-constrained discontinuous Galerkin Network (DGNet) approach to solve compressible Euler equations. To validate the effectiveness, stability, and generalizability of our novel DGNet approach, we present numerical results for 1D and 2D compressible Euler equation problems.
  arXiv  Detail & Related papers  (2024-09-27T01:13:38Z)
• Event-Aided Time-to-Collision Estimation for Autonomous Driving [28.13397992839372]
  We present a novel method that estimates the time to collision using a neuromorphic event-based camera. The proposed algorithm consists of a two-step approach for efficient and accurate geometric model fitting on event data. Experiments on both synthetic and real data demonstrate the effectiveness of the proposed method.
  arXiv  Detail & Related papers  (2024-07-10T02:37:36Z)
• Shuffled Autoregression For Motion Interpolation [53.61556200049156]
  This work aims to provide a deep-learning solution for the motion task. We propose a novel framework, referred to as emphShuffled AutoRegression, which expands the autoregression to generate in arbitrary (shuffled) order. We also propose an approach to constructing a particular kind of dependency graph, with three stages assembled into an end-to-end spatial-temporal motion Transformer.
  arXiv  Detail & Related papers  (2023-06-10T07:14:59Z)
• Object-centric and memory-guided normality reconstruction for video anomaly detection [56.64792194894702]
  This paper addresses anomaly detection problem for videosurveillance. Due to the inherent rarity and heterogeneity of abnormal events, the problem is viewed as a normality modeling strategy. Our model learns object-centric normal patterns without seeing anomalous samples during training.
  arXiv  Detail & Related papers  (2022-03-07T19:28:39Z)
• Message Passing Neural PDE Solvers [60.77761603258397]
  We build a neural message passing solver, replacing allally designed components in the graph with backprop-optimized neural function approximators. We show that neural message passing solvers representationally contain some classical methods, such as finite differences, finite volumes, and WENO schemes. We validate our method on various fluid-like flow problems, demonstrating fast, stable, and accurate performance across different domain topologies, equation parameters, discretizations, etc., in 1D and 2D.
  arXiv  Detail & Related papers  (2022-02-07T17:47:46Z)
• Learning Discriminative Shrinkage Deep Networks for Image Deconvolution [122.79108159874426]
  We propose an effective non-blind deconvolution approach by learning discriminative shrinkage functions to implicitly model these terms. Experimental results show that the proposed method performs favorably against the state-of-the-art ones in terms of efficiency and accuracy.
  arXiv  Detail & Related papers  (2021-11-27T12:12:57Z)
• Pushing the Envelope of Rotation Averaging for Visual SLAM [69.7375052440794]
  We propose a novel optimization backbone for visual SLAM systems. We leverage averaging to improve the accuracy, efficiency and robustness of conventional monocular SLAM systems. Our approach can exhibit up to 10x faster with comparable accuracy against the state-art on public benchmarks.
  arXiv  Detail & Related papers  (2020-11-02T18:02:26Z)
• MORPH-DSLAM: Model Order Reduction for PHysics-based Deformable SLAM [0.0]
  We propose a new methodology to estimate the 3D displacement field of deformable objects from video sequences using standard monocular cameras. We solve in real time the complete (possibly visco-elastic)hyperlinearity problem to properly describe the strain and stress fields that are consistent with the displacements captured by the images, constrained by real physics.
  arXiv  Detail & Related papers  (2020-09-01T17:06:41Z)
• Renormalization for Initialization of Rolling Shutter Visual-Inertial Odometry [5.33024001730262]
  Initialization is a prerequisite for using inertial signals and fusing them with visual data. We propose a novel statistical solution to the problem on visual and inertial data simultaneously, by casting it into the renormalization scheme of Kanatani. Extensive evaluations on ground truth exhibit superior performance and a gain in accuracy of up to $20%$ over the originally proposed Least Squares solution.
  arXiv  Detail & Related papers  (2020-08-14T14:54:15Z)
• Differentiable Causal Discovery from Interventional Data [141.41931444927184]
  We propose a theoretically-grounded method based on neural networks that can leverage interventional data. We show that our approach compares favorably to the state of the art in a variety of settings.
  arXiv  Detail & Related papers  (2020-07-03T15:19:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.