STC-Flow: Spatio-temporal Context-aware Optical Flow Estimation

• URL: http://arxiv.org/abs/2003.00434v2
• Date: Tue, 3 Nov 2020 09:11:06 GMT
• Title: STC-Flow: Spatio-temporal Context-aware Optical Flow Estimation
• Authors: Xiaolin Song, Yuyang Zhao, and Jingyu Yang
• Abstract summary: We propose a-temporal contextual network, STC-Flow, for optical flow estimation. STC-Flow contains three key context modules - pyramid spatial context module, temporal context correlation module and residual contextual upsampling module. Experimental results indicate that the proposed scheme achieves the state-of-the-art performance of two-frame based methods on the Sintel dataset and the KITTI 2012/2015 datasets.
• Score: 13.649566468445258
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In this paper, we propose a spatio-temporal contextual network, STC-Flow, for optical flow estimation. Unlike previous optical flow estimation approaches with local pyramid feature extraction and multi-level correlation, we propose a contextual relation exploration architecture by capturing rich long-range dependencies in spatial and temporal dimensions. Specifically, STC-Flow contains three key context modules - pyramidal spatial context module, temporal context correlation module and recurrent residual contextual upsampling module, to build the relationship in each stage of feature extraction, correlation, and flow reconstruction, respectively. Experimental results indicate that the proposed scheme achieves the state-of-the-art performance of two-frame based methods on the Sintel dataset and the KITTI 2012/2015 datasets.

Related papers

• DSLO: Deep Sequence LiDAR Odometry Based on Inconsistent Spatio-temporal Propagation [66.8732965660931]
  paper introduces a 3D point cloud sequence learning model based on inconsistent-temporal propagation for LiDAR odometry DSLO. It consists of a pyramid structure with a sequential pose module, a hierarchical pose refinement module, and a temporal feature propagation module.
  arXiv  Detail & Related papers  (2024-09-01T15:12:48Z)
• Hierarchical Temporal Context Learning for Camera-based Semantic Scene Completion [57.232688209606515]
  We present HTCL, a novel Temporal Temporal Context Learning paradigm for improving camera-based semantic scene completion. Our method ranks $1st$ on the Semantic KITTI benchmark and even surpasses LiDAR-based methods in terms of mIoU.
  arXiv  Detail & Related papers  (2024-07-02T09:11:17Z)
• TCCT-Net: Two-Stream Network Architecture for Fast and Efficient Engagement Estimation via Behavioral Feature Signals [58.865901821451295]
  We present a novel two-stream feature fusion "Tensor-Convolution and Convolution-Transformer Network" (TCCT-Net) architecture. To better learn the meaningful patterns in the temporal-spatial domain, we design a "CT" stream that integrates a hybrid convolutional-transformer. In parallel, to efficiently extract rich patterns from the temporal-frequency domain, we introduce a "TC" stream that uses Continuous Wavelet Transform (CWT) to represent information in a 2D tensor form.
  arXiv  Detail & Related papers  (2024-04-15T06:01:48Z)
• OpenSTL: A Comprehensive Benchmark of Spatio-Temporal Predictive Learning [67.07363529640784]
  We propose OpenSTL to categorize prevalent approaches into recurrent-based and recurrent-free models. We conduct standard evaluations on datasets across various domains, including synthetic moving object trajectory, human motion, driving scenes, traffic flow and forecasting weather. We find that recurrent-free models achieve a good balance between efficiency and performance than recurrent models.
  arXiv  Detail & Related papers  (2023-06-20T03:02:14Z)
• SSTM: Spatiotemporal Recurrent Transformers for Multi-frame Optical Flow Estimation [0.0]
  In optical flow estimates in and near cluded regions, and out-of-boundary regions are two of the current significant limitations of optical flow estimation algorithms. Recent state-of-the-art optical flow estimation algorithms are two-frame based methods where optical flow is estimated sequentially for each consecutive image pair in a sequence. We propose a learning-based multi-frame optical flow estimation method that estimates two or more consecutive optical flows in parallel from multi-frame image sequences.
  arXiv  Detail & Related papers  (2023-04-26T23:39:40Z)
• A Correlation Information-based Spatiotemporal Network for Traffic Flow Forecasting [4.933291769305828]
  We propose a novel correlation information-basedtemporal network (CorrSTN) for predicting traffic patterns. In particular, our model makes significant improvements compared with the latest model ASTGNN by 12.7%, 14.4% and 27.4% in the metrics of MAE, RMSE and MAPE, respectively.
  arXiv  Detail & Related papers  (2022-05-20T04:38:49Z)
• Detail Preserving Residual Feature Pyramid Modules for Optical Flow [3.3301533805099357]
  Residual Feature Pyramid Module (RFPM) retains important details in the feature map without changing the overall iterative refinement design of the optical flow estimation. Results show that our RFPM visibly reduces flow errors and improves state-of-art performance in the clean pass of Sintel.
  arXiv  Detail & Related papers  (2021-07-23T01:53:04Z)
• Multiple Object Tracking with Correlation Learning [16.959379957515974]
  We propose to exploit the local correlation module to model the topological relationship between targets and their surrounding environment. Specifically, we establish dense correspondences of each spatial location and its context, and explicitly constrain the correlation volumes through self-supervised learning. Our approach demonstrates the effectiveness of correlation learning with the superior performance and obtains state-of-the-art MOTA of 76.5% and IDF1 of 73.6% on MOT17.
  arXiv  Detail & Related papers  (2021-04-08T06:48:02Z)
• STaRFlow: A SpatioTemporal Recurrent Cell for Lightweight Multi-Frame Optical Flow Estimation [64.99259320624148]
  We present a new lightweight CNN-based algorithm for multi-frame optical flow estimation. The resulting STaRFlow algorithm gives state-of-the-art performances on MPI Sintel and Kitti2015.
  arXiv  Detail & Related papers  (2020-07-10T17:01:34Z)
• FPCR-Net: Feature Pyramidal Correlation and Residual Reconstruction for Optical Flow Estimation [72.41370576242116]
  We propose a semi-supervised Feature Pyramidal Correlation and Residual Reconstruction Network (FPCR-Net) for optical flow estimation from frame pairs. It consists of two main modules: pyramid correlation mapping and residual reconstruction. Experiment results show that the proposed scheme achieves the state-of-the-art performance, with improvement by 0.80, 1.15 and 0.10 in terms of average end-point error (AEE) against competing baseline methods.
  arXiv  Detail & Related papers  (2020-01-17T07:13:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.