NeuFlow: Real-time, High-accuracy Optical Flow Estimation on Robots Using Edge Devices

• URL: http://arxiv.org/abs/2403.10425v1
• Date: Fri, 15 Mar 2024 15:58:51 GMT
• Title: NeuFlow: Real-time, High-accuracy Optical Flow Estimation on Robots Using Edge Devices
• Authors: Zhiyong Zhang, Huaizu Jiang, Hanumant Singh,
• Abstract summary: Real-time high-accuracy optical flow estimation is a crucial component in various applications. We propose a highly efficient optical flow architecture, called NeuFlow, that addresses both high accuracy and computational cost concerns. Our approach achieves around 30 FPS on edge computing platforms, which represents a significant breakthrough in deploying complex computer vision tasks.
• Score: 6.470511497023878
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Real-time high-accuracy optical flow estimation is a crucial component in various applications, including localization and mapping in robotics, object tracking, and activity recognition in computer vision. While recent learning-based optical flow methods have achieved high accuracy, they often come with heavy computation costs. In this paper, we propose a highly efficient optical flow architecture, called NeuFlow, that addresses both high accuracy and computational cost concerns. The architecture follows a global-to-local scheme. Given the features of the input images extracted at different spatial resolutions, global matching is employed to estimate an initial optical flow on the 1/16 resolution, capturing large displacement, which is then refined on the 1/8 resolution with lightweight CNN layers for better accuracy. We evaluate our approach on Jetson Orin Nano and RTX 2080 to demonstrate efficiency improvements across different computing platforms. We achieve a notable 10x-80x speedup compared to several state-of-the-art methods, while maintaining comparable accuracy. Our approach achieves around 30 FPS on edge computing platforms, which represents a significant breakthrough in deploying complex computer vision tasks such as SLAM on small robots like drones. The full training and evaluation code is available at https://github.com/neufieldrobotics/NeuFlow.

Related papers

• Task-Oriented Real-time Visual Inference for IoVT Systems: A Co-design Framework of Neural Networks and Edge Deployment [61.20689382879937]
  Task-oriented edge computing addresses this by shifting data analysis to the edge. Existing methods struggle to balance high model performance with low resource consumption. We propose a novel co-design framework to optimize neural network architecture.
  arXiv  Detail & Related papers  (2024-10-29T19:02:54Z)
• NeuFlow v2: High-Efficiency Optical Flow Estimation on Edge Devices [6.157420789049589]
  We propose a highly efficient optical flow method that balances high accuracy with reduced computational demands. We introduce new components including a much more light-weight backbone and a fast refinement module. Our model achieves a 10x-70x speedup while maintaining comparable performance on both synthetic and real-world data.
  arXiv  Detail & Related papers  (2024-08-19T17:13:34Z)
• Efficient Single Object Detection on Image Patches with Early Exit Enhanced High-Precision CNNs [0.0]
  This paper proposes a novel approach for detecting objects using mobile robots in the context of the RoboCup Standard Platform League. The challenge lies in detecting a dynamic object in varying lighting conditions and blurred images caused by fast movements. To address this challenge, the paper presents a convolutional neural network architecture designed specifically for computationally constrained robotic platforms.
  arXiv  Detail & Related papers  (2023-09-07T07:23:55Z)
• Dynamic Frame Interpolation in Wavelet Domain [57.25341639095404]
  Video frame is an important low-level computation vision task, which can increase frame rate for more fluent visual experience. Existing methods have achieved great success by employing advanced motion models and synthesis networks. WaveletVFI can reduce computation up to 40% while maintaining similar accuracy, making it perform more efficiently against other state-of-the-arts.
  arXiv  Detail & Related papers  (2023-09-07T06:41:15Z)
• Towards High-Frequency Tracking and Fast Edge-Aware Optimization [2.2662585107579165]
  This dissertation advances the state of the art for AR/VR tracking systems by increasing the tracking frequency by orders of magnitude. It proposes an efficient algorithm for the problem of edge-aware optimization.
  arXiv  Detail & Related papers  (2023-09-02T01:20:34Z)
• FlowNAS: Neural Architecture Search for Optical Flow Estimation [65.44079917247369]
  We propose a neural architecture search method named FlowNAS to automatically find the better encoder architecture for flow estimation task. Experimental results show that the discovered architecture with the weights inherited from the super-network achieves 4.67% F1-all error on KITTI.
  arXiv  Detail & Related papers  (2022-07-04T09:05:25Z)
• Deep Learning for Real Time Satellite Pose Estimation on Low Power Edge TPU [58.720142291102135]
  In this paper we propose a pose estimation software exploiting neural network architectures. We show how low power machine learning accelerators could enable Artificial Intelligence exploitation in space.
  arXiv  Detail & Related papers  (2022-04-07T08:53:18Z)
• Dense Optical Flow from Event Cameras [55.79329250951028]
  We propose to incorporate feature correlation and sequential processing into dense optical flow estimation from event cameras. Our proposed approach computes dense optical flow and reduces the end-point error by 23% on MVSEC.
  arXiv  Detail & Related papers  (2021-08-24T07:39:08Z)
• FastFlowNet: A Lightweight Network for Fast Optical Flow Estimation [81.76975488010213]
  Dense optical flow estimation plays a key role in many robotic vision tasks. Current networks often occupy large number of parameters and require heavy computation costs. Our proposed FastFlowNet works in the well-known coarse-to-fine manner with following innovations.
  arXiv  Detail & Related papers  (2021-03-08T03:09:37Z)
• Lightweight Convolutional Neural Network with Gaussian-based Grasping Representation for Robotic Grasping Detection [4.683939045230724]
  Current object detectors are difficult to strike a balance between high accuracy and fast inference speed. We present an efficient and robust fully convolutional neural network model to perform robotic grasping pose estimation. The network is an order of magnitude smaller than other excellent algorithms.
  arXiv  Detail & Related papers  (2021-01-25T16:36:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.