FastFlowNet: A Lightweight Network for Fast Optical Flow Estimation
- URL: http://arxiv.org/abs/2103.04524v1
- Date: Mon, 8 Mar 2021 03:09:37 GMT
- Title: FastFlowNet: A Lightweight Network for Fast Optical Flow Estimation
- Authors: Lingtong Kong, Chunhua Shen, Jie Yang
- Abstract summary: Dense optical flow estimation plays a key role in many robotic vision tasks.
Current networks often occupy large number of parameters and require heavy computation costs.
Our proposed FastFlowNet works in the well-known coarse-to-fine manner with following innovations.
- Score: 81.76975488010213
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Dense optical flow estimation plays a key role in many robotic vision tasks.
It has been predicted with satisfying accuracy than traditional methods with
advent of deep learning. However, current networks often occupy large number of
parameters and require heavy computation costs. These drawbacks have hindered
applications on power- or memory-constrained mobile devices. To deal with these
challenges, in this paper, we dive into designing efficient structure for fast
and accurate optical flow prediction. Our proposed FastFlowNet works in the
well-known coarse-to-fine manner with following innovations. First, a new head
enhanced pooling pyramid (HEPP) feature extractor is employed to intensify
high-resolution pyramid feature while reducing parameters. Second, we introduce
a novel center dense dilated correlation (CDDC) layer for constructing compact
cost volume that can keep large search radius with reduced computation burden.
Third, an efficient shuffle block decoder (SBD) is implanted into each pyramid
level to acclerate flow estimation with marginal drops in accuracy. Experiments
on both synthetic Sintel and real-world KITTI datasets demonstrate the
effectiveness of proposed approaches, which consumes only 1/10 computation of
comparable networks to get 90% of their performance. In particular, FastFlowNet
only contains 1.37 M parameters and runs at 90 or 5.7 fps with one desktop
NVIDIA GTX 1080 Ti or embedded Jetson TX2 GPU on Sintel resolution images.
Related papers
- NeuFlow v2: High-Efficiency Optical Flow Estimation on Edge Devices [6.157420789049589]
We propose a highly efficient optical flow method that balances high accuracy with reduced computational demands.
We introduce new components including a much more light-weight backbone and a fast refinement module.
Our model achieves a 10x-70x speedup while maintaining comparable performance on both synthetic and real-world data.
arXiv Detail & Related papers (2024-08-19T17:13:34Z) - NeuFlow: Real-time, High-accuracy Optical Flow Estimation on Robots Using Edge Devices [6.470511497023878]
Real-time high-accuracy optical flow estimation is a crucial component in various applications.
We propose a highly efficient optical flow architecture, called NeuFlow, that addresses both high accuracy and computational cost concerns.
Our approach achieves around 30 FPS on edge computing platforms, which represents a significant breakthrough in deploying complex computer vision tasks.
arXiv Detail & Related papers (2024-03-15T15:58:51Z) - Latency-aware Unified Dynamic Networks for Efficient Image Recognition [72.8951331472913]
LAUDNet is a framework to bridge the theoretical and practical efficiency gap in dynamic networks.
It integrates three primary dynamic paradigms-spatially adaptive computation, dynamic layer skipping, and dynamic channel skipping.
It can notably reduce the latency of models like ResNet by over 50% on platforms such as V100,3090, and TX2 GPUs.
arXiv Detail & Related papers (2023-08-30T10:57:41Z) - Pushing the Limits of Asynchronous Graph-based Object Detection with
Event Cameras [62.70541164894224]
We introduce several architecture choices which allow us to scale the depth and complexity of such models while maintaining low computation.
Our method runs 3.7 times faster than a dense graph neural network, taking only 8.4 ms per forward pass.
arXiv Detail & Related papers (2022-11-22T15:14:20Z) - NAF: Neural Attenuation Fields for Sparse-View CBCT Reconstruction [79.13750275141139]
This paper proposes a novel and fast self-supervised solution for sparse-view CBCT reconstruction.
The desired attenuation coefficients are represented as a continuous function of 3D spatial coordinates, parameterized by a fully-connected deep neural network.
A learning-based encoder entailing hash coding is adopted to help the network capture high-frequency details.
arXiv Detail & Related papers (2022-09-29T04:06:00Z) - Faster Attention Is What You Need: A Fast Self-Attention Neural Network
Backbone Architecture for the Edge via Double-Condensing Attention Condensers [71.40595908386477]
We introduce a new faster attention condenser design called double-condensing attention condensers.
The resulting backbone (which we name AttendNeXt) achieves significantly higher inference throughput on an embedded ARM processor.
These promising results demonstrate that exploring different efficient architecture designs and self-attention mechanisms can lead to interesting new building blocks for TinyML applications.
arXiv Detail & Related papers (2022-08-15T02:47:33Z) - An Adaptive Device-Edge Co-Inference Framework Based on Soft
Actor-Critic [72.35307086274912]
High-dimension parameter model and large-scale mathematical calculation restrict execution efficiency, especially for Internet of Things (IoT) devices.
We propose a new Deep Reinforcement Learning (DRL)-Soft Actor Critic for discrete (SAC-d), which generates the emphexit point, emphexit point, and emphcompressing bits by soft policy iterations.
Based on the latency and accuracy aware reward design, such an computation can well adapt to the complex environment like dynamic wireless channel and arbitrary processing, and is capable of supporting the 5G URL
arXiv Detail & Related papers (2022-01-09T09:31:50Z) - a novel attention-based network for fast salient object detection [14.246237737452105]
In the current salient object detection network, the most popular method is using U-shape structure.
We propose a new deep convolution network architecture with three contributions.
Results demonstrate that the proposed method can compress the model to 1/3 of the original size nearly without losing the accuracy.
arXiv Detail & Related papers (2021-12-20T12:30:20Z) - Accelerating Sparse Deep Neural Networks [20.6942347219753]
We present the design and behavior of Sparse Cores, which exploit a 2:4 (25%) sparsity pattern that leads to twice the math throughput of dense matrix units.
We also describe a simple workflow for training networks that both satisfy the 2:4 sparsity pattern requirements and maintain accuracy.
arXiv Detail & Related papers (2021-04-16T21:27:32Z) - FDFlowNet: Fast Optical Flow Estimation using a Deep Lightweight Network [12.249680550252327]
We present a lightweight yet effective model for real-time optical flow estimation, termed FDFlowNet (fast deep flownet)
We achieve better or similar accuracy on the challenging KITTI and Sintel benchmarks while being about 2 times faster than PWC-Net.
arXiv Detail & Related papers (2020-06-22T14:01:01Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.