ASFlow: Unsupervised Optical Flow Learning with Adaptive Pyramid Sampling

• URL: http://arxiv.org/abs/2104.03560v1
• Date: Thu, 8 Apr 2021 07:22:35 GMT
• Title: ASFlow: Unsupervised Optical Flow Learning with Adaptive Pyramid Sampling
• Authors: Kunming Luo, Ao Luo, Chuan Wang, Haoqiang Fan, Shuaicheng Liu
• Abstract summary: We present an unsupervised optical flow estimation method by proposing an adaptive pyramid sampling in the deep pyramid network. Our method achieves the best performance for unsupervised optical flow estimation on multiple leading benchmarks, including MPI-SIntel, KITTI 2012 and KITTI 2015.
• Score: 26.868635622137106
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We present an unsupervised optical flow estimation method by proposing an adaptive pyramid sampling in the deep pyramid network. Specifically, in the pyramid downsampling, we propose an Content Aware Pooling (CAP) module, which promotes local feature gathering by avoiding cross region pooling, so that the learned features become more representative. In the pyramid upsampling, we propose an Adaptive Flow Upsampling (AFU) module, where cross edge interpolation can be avoided, producing sharp motion boundaries. Equipped with these two modules, our method achieves the best performance for unsupervised optical flow estimation on multiple leading benchmarks, including MPI-SIntel, KITTI 2012 and KITTI 2015. Particuarlly, we achieve EPE=1.5 on KITTI 2012 and F1=9.67% KITTI 2015, which outperform the previous state-of-the-art methods by 16.7% and 13.1%, respectively.

Related papers

• Efficient Learning of POMDPs with Known Observation Model in Average-Reward Setting [56.92178753201331]
  We propose the Observation-Aware Spectral (OAS) estimation technique, which enables the POMDP parameters to be learned from samples collected using a belief-based policy. We show the consistency of the OAS procedure, and we prove a regret guarantee of order $mathcalO(sqrtT log(T)$ for the proposed OAS-UCRL algorithm.
  arXiv  Detail & Related papers  (2024-10-02T08:46:34Z)
• Semi-Federated Learning: Convergence Analysis and Optimization of A Hybrid Learning Framework [70.83511997272457]
  We propose a semi-federated learning (SemiFL) paradigm to leverage both the base station (BS) and devices for a hybrid implementation of centralized learning (CL) and FL. We propose a two-stage algorithm to solve this intractable problem, in which we provide the closed-form solutions to the beamformers.
  arXiv  Detail & Related papers  (2023-10-04T03:32:39Z)
• SAMFlow: Eliminating Any Fragmentation in Optical Flow with Segment Anything Model [17.88914104216893]
  We propose a solution to embed the frozen SAM image encoder into FlowFormer to enhance object perception. Our proposed SAMFlow model reaches 0.86/2.10 clean/final EPE and 3.55/12.32 EPE/F1-all on Sintel and KITTI-15 training set, surpassing Flowformer by 8.5%/9.9% and 13.2%/16.3%.
  arXiv  Detail & Related papers  (2023-07-31T11:40:53Z)
• GMFlow: Learning Optical Flow via Global Matching [124.57850500778277]
  We propose a GMFlow framework for learning optical flow estimation. It consists of three main components: a customized Transformer for feature enhancement, a correlation and softmax layer for global feature matching, and a self-attention layer for flow propagation. Our new framework outperforms 32-iteration RAFT's performance on the challenging Sintel benchmark.
  arXiv  Detail & Related papers  (2021-11-26T18:59:56Z)
• Detail Preserving Residual Feature Pyramid Modules for Optical Flow [3.3301533805099357]
  Residual Feature Pyramid Module (RFPM) retains important details in the feature map without changing the overall iterative refinement design of the optical flow estimation. Results show that our RFPM visibly reduces flow errors and improves state-of-art performance in the clean pass of Sintel.
  arXiv  Detail & Related papers  (2021-07-23T01:53:04Z)
• Normalized Convolution Upsampling for Refined Optical Flow Estimation [23.652615797842085]
  Normalized Convolution UPsampler (NCUP) is an efficient joint upsampling approach to produce the full-resolution flow during the training of optical flow CNNs. Our proposed approach formulates the upsampling task as a sparse problem and employs the normalized convolutional neural networks to solve it. We achieve state-of-the-art results on Sintel benchmark with 6% error reduction, and on-par on the KITTI dataset, while having 7.5% fewer parameters.
  arXiv  Detail & Related papers  (2021-02-13T18:34:03Z)
• UPFlow: Upsampling Pyramid for Unsupervised Optical Flow Learning [34.580309867067946]
  We present an unsupervised learning approach for optical flow estimation. We design a self-guided upsample module to tackle the blur problem caused by bilinear upsampling between pyramid levels. We propose a pyramid distillation loss to add supervision for intermediate levels via distilling the finest flow as pseudo labels.
  arXiv  Detail & Related papers  (2020-12-01T01:57:46Z)
• Learning Affinity-Aware Upsampling for Deep Image Matting [83.02806488958399]
  We show that learning affinity in upsampling provides an effective and efficient approach to exploit pairwise interactions in deep networks. In particular, results on the Composition-1k matting dataset show that A2U achieves a 14% relative improvement in the SAD metric against a strong baseline. Compared with the state-of-the-art matting network, we achieve 8% higher performance with only 40% model complexity.
  arXiv  Detail & Related papers  (2020-11-29T05:09:43Z)
• ScopeFlow: Dynamic Scene Scoping for Optical Flow [94.42139459221784]
  We propose to modify the common training protocols of optical flow. The improvement is based on observing the bias in sampling challenging data. We find that both regularization and augmentation should decrease during the training protocol.
  arXiv  Detail & Related papers  (2020-02-25T09:58:49Z)
• FPCR-Net: Feature Pyramidal Correlation and Residual Reconstruction for Optical Flow Estimation [72.41370576242116]
  We propose a semi-supervised Feature Pyramidal Correlation and Residual Reconstruction Network (FPCR-Net) for optical flow estimation from frame pairs. It consists of two main modules: pyramid correlation mapping and residual reconstruction. Experiment results show that the proposed scheme achieves the state-of-the-art performance, with improvement by 0.80, 1.15 and 0.10 in terms of average end-point error (AEE) against competing baseline methods.
  arXiv  Detail & Related papers  (2020-01-17T07:13:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.