NeuFlow v2: High-Efficiency Optical Flow Estimation on Edge Devices

• URL: http://arxiv.org/abs/2408.10161v2
• Date: Wed, 21 Aug 2024 23:23:10 GMT
• Title: NeuFlow v2: High-Efficiency Optical Flow Estimation on Edge Devices
• Authors: Zhiyong Zhang, Aniket Gupta, Huaizu Jiang, Hanumant Singh,
• Abstract summary: We propose a highly efficient optical flow method that balances high accuracy with reduced computational demands. We introduce new components including a much more light-weight backbone and a fast refinement module. Our model achieves a 10x-70x speedup while maintaining comparable performance on both synthetic and real-world data.
• Score: 6.157420789049589
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Real-time high-accuracy optical flow estimation is crucial for various real-world applications. While recent learning-based optical flow methods have achieved high accuracy, they often come with significant computational costs. In this paper, we propose a highly efficient optical flow method that balances high accuracy with reduced computational demands. Building upon NeuFlow v1, we introduce new components including a much more light-weight backbone and a fast refinement module. Both these modules help in keeping the computational demands light while providing close to state of the art accuracy. Compares to other state of the art methods, our model achieves a 10x-70x speedup while maintaining comparable performance on both synthetic and real-world data. It is capable of running at over 20 FPS on 512x384 resolution images on a Jetson Orin Nano. The full training and evaluation code is available at https://github.com/neufieldrobotics/NeuFlow_v2.

Related papers

• MemFlow: Optical Flow Estimation and Prediction with Memory [54.22820729477756]
  We present MemFlow, a real-time method for optical flow estimation and prediction with memory. Our method enables memory read-out and update modules for aggregating historical motion information in real-time. Our approach seamlessly extends to the future prediction of optical flow based on past observations.
  arXiv  Detail & Related papers  (2024-04-07T04:56:58Z)
• NeuFlow: Real-time, High-accuracy Optical Flow Estimation on Robots Using Edge Devices [6.470511497023878]
  Real-time high-accuracy optical flow estimation is a crucial component in various applications. We propose a highly efficient optical flow architecture, called NeuFlow, that addresses both high accuracy and computational cost concerns. Our approach achieves around 30 FPS on edge computing platforms, which represents a significant breakthrough in deploying complex computer vision tasks.
  arXiv  Detail & Related papers  (2024-03-15T15:58:51Z)
• RealFlow: EM-based Realistic Optical Flow Dataset Generation from Videos [28.995525297929348]
  RealFlow is a framework that can create large-scale optical flow datasets directly from unlabeled realistic videos. We first estimate optical flow between a pair of video frames, and then synthesize a new image from this pair based on the predicted flow. Our approach achieves state-of-the-art performance on two standard benchmarks compared with both supervised and unsupervised optical flow methods.
  arXiv  Detail & Related papers  (2022-07-22T13:33:03Z)
• GMFlow: Learning Optical Flow via Global Matching [124.57850500778277]
  We propose a GMFlow framework for learning optical flow estimation. It consists of three main components: a customized Transformer for feature enhancement, a correlation and softmax layer for global feature matching, and a self-attention layer for flow propagation. Our new framework outperforms 32-iteration RAFT's performance on the challenging Sintel benchmark.
  arXiv  Detail & Related papers  (2021-11-26T18:59:56Z)
• Dense Optical Flow from Event Cameras [55.79329250951028]
  We propose to incorporate feature correlation and sequential processing into dense optical flow estimation from event cameras. Our proposed approach computes dense optical flow and reduces the end-point error by 23% on MVSEC.
  arXiv  Detail & Related papers  (2021-08-24T07:39:08Z)
• AutoFlow: Learning a Better Training Set for Optical Flow [62.40293188964933]
  AutoFlow is a method to render training data for optical flow. AutoFlow achieves state-of-the-art accuracy in pre-training both PWC-Net and RAFT.
  arXiv  Detail & Related papers  (2021-04-29T17:55:23Z)
• FastFlowNet: A Lightweight Network for Fast Optical Flow Estimation [81.76975488010213]
  Dense optical flow estimation plays a key role in many robotic vision tasks. Current networks often occupy large number of parameters and require heavy computation costs. Our proposed FastFlowNet works in the well-known coarse-to-fine manner with following innovations.
  arXiv  Detail & Related papers  (2021-03-08T03:09:37Z)
• Reinforcement Learning with Latent Flow [78.74671595139613]
  Flow of Latents for Reinforcement Learning (Flare) is a network architecture for RL that explicitly encodes temporal information through latent vector differences. We show that Flare recovers optimal performance in state-based RL without explicit access to the state velocity. We also show that Flare achieves state-of-the-art performance on pixel-based challenging continuous control tasks within the DeepMind control benchmark suite.
  arXiv  Detail & Related papers  (2021-01-06T03:50:50Z)
• FDFlowNet: Fast Optical Flow Estimation using a Deep Lightweight Network [12.249680550252327]
  We present a lightweight yet effective model for real-time optical flow estimation, termed FDFlowNet (fast deep flownet) We achieve better or similar accuracy on the challenging KITTI and Sintel benchmarks while being about 2 times faster than PWC-Net.
  arXiv  Detail & Related papers  (2020-06-22T14:01:01Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.