SEA-RAFT: Simple, Efficient, Accurate RAFT for Optical Flow

• URL: http://arxiv.org/abs/2405.14793v1
• Date: Thu, 23 May 2024 17:04:04 GMT
• Title: SEA-RAFT: Simple, Efficient, Accurate RAFT for Optical Flow
• Authors: Yihan Wang, Lahav Lipson, Jia Deng,
• Abstract summary: We introduce SEA-RAFT, a more simple, efficient, and accurate RAFT for optical flow. SEA-RAFT achieves state-of-the-art accuracy on the Spring benchmark with a 3.69 endpoint-error (EPE) and a 0.36 1-pixel outlier rate (1px) With its high efficiency, SEA-RAFT operates at least 2.3x faster than existing methods while maintaining competitive performance.
• Score: 29.823972546363716
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We introduce SEA-RAFT, a more simple, efficient, and accurate RAFT for optical flow. Compared with RAFT, SEA-RAFT is trained with a new loss (mixture of Laplace). It directly regresses an initial flow for faster convergence in iterative refinements and introduces rigid-motion pre-training to improve generalization. SEA-RAFT achieves state-of-the-art accuracy on the Spring benchmark with a 3.69 endpoint-error (EPE) and a 0.36 1-pixel outlier rate (1px), representing 22.9% and 17.8% error reduction from best published results. In addition, SEA-RAFT obtains the best cross-dataset generalization on KITTI and Spring. With its high efficiency, SEA-RAFT operates at least 2.3x faster than existing methods while maintaining competitive performance. The code is publicly available at https://github.com/princeton-vl/SEA-RAFT.

Related papers

• UniPTS: A Unified Framework for Proficient Post-Training Sparsity [67.16547529992928]
  Post-training Sparsity (PTS) is a newly emerged avenue that chases efficient network sparsity with limited data in need. In this paper, we attempt to reconcile this disparity by transposing three cardinal factors that profoundly alter the performance of conventional sparsity into the context of PTS. Our framework, termed UniPTS, is validated to be much superior to existing PTS methods across extensive benchmarks.
  arXiv  Detail & Related papers  (2024-05-29T06:53:18Z)
• Efficient LoFTR: Semi-Dense Local Feature Matching with Sparse-Like Speed [42.861344584752]
  Previous detector-free matcher LoFTR has shown remarkable matching capability in handling large-viewpoint change and texture-poor scenarios. We revisit its design choices and derive multiple improvements for both efficiency and accuracy. Our method can achieve higher accuracy compared with competitive semi-dense matchers.
  arXiv  Detail & Related papers  (2024-03-07T18:58:40Z)
• Rethinking RAFT for Efficient Optical Flow [9.115508086522887]
  This paper proposes a novel approach based on the RAFT framework. It incorporates the attention mechanism to handle global feature extraction and address repetitive patterns. The proposed method, Efficient RAFT (Ef-RAFT),achieves significant improvements of 10% on the Sintel dataset and 5% on the KITTI dataset over RAFT.
  arXiv  Detail & Related papers  (2024-01-01T18:23:39Z)
• DIP: Deep Inverse Patchmatch for High-Resolution Optical Flow [7.73554718719193]
  We propose a novel Patchmatch-based framework to work on high-resolution optical flow estimation. It can get high-precision results with lower memory benefiting from propagation and local search of Patchmatch. Our method ranks first on all the metrics on the popular KITTI2015 benchmark, and ranks second on EPE on the Sintel clean benchmark among published optical flow methods.
  arXiv  Detail & Related papers  (2022-04-01T10:13:59Z)
• Efficient Few-Shot Object Detection via Knowledge Inheritance [62.36414544915032]
  Few-shot object detection (FSOD) aims at learning a generic detector that can adapt to unseen tasks with scarce training samples. We present an efficient pretrain-transfer framework (PTF) baseline with no computational increment. We also propose an adaptive length re-scaling (ALR) strategy to alleviate the vector length inconsistency between the predicted novel weights and the pretrained base weights.
  arXiv  Detail & Related papers  (2022-03-23T06:24:31Z)
• GMFlow: Learning Optical Flow via Global Matching [124.57850500778277]
  We propose a GMFlow framework for learning optical flow estimation. It consists of three main components: a customized Transformer for feature enhancement, a correlation and softmax layer for global feature matching, and a self-attention layer for flow propagation. Our new framework outperforms 32-iteration RAFT's performance on the challenging Sintel benchmark.
  arXiv  Detail & Related papers  (2021-11-26T18:59:56Z)
• Faster Non-Convex Federated Learning via Global and Local Momentum [57.52663209739171]
  textttFedGLOMO is the first (first-order) FLtexttFedGLOMO algorithm. Our algorithm is provably optimal even with communication between the clients and the server.
  arXiv  Detail & Related papers  (2020-12-07T21:05:31Z)
• RAFT: Recurrent All-Pairs Field Transforms for Optical Flow [78.92562539905951]
  We introduce Recurrent All-Pairs Field Transforms (RAFT), a new deep network architecture for optical flow. RAFT extracts per-pixel features, builds multi-scale 4D correlation volumes for all pairs of pixels, and iteratively updates a flow field. RAFT achieves state-of-the-art performance.
  arXiv  Detail & Related papers  (2020-03-26T17:12:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.