ResFlow: Fine-tuning Residual Optical Flow for Event-based High Temporal Resolution Motion Estimation

• URL: http://arxiv.org/abs/2412.09105v1
• Date: Thu, 12 Dec 2024 09:35:47 GMT
• Title: ResFlow: Fine-tuning Residual Optical Flow for Event-based High Temporal Resolution Motion Estimation
• Authors: Qianang Zhou, Zhiyu Zhu, Junhui Hou, Yongjian Deng, Youfu Li, Junlin Xiong,
• Abstract summary: Event cameras hold significant promise for high-temporal-resolution (HTR) motion estimation. We propose a residual-based paradigm for estimating HTR optical flow with event data.
• Score: 50.80115710105251
• License:
• Abstract: Event cameras hold significant promise for high-temporal-resolution (HTR) motion estimation. However, estimating event-based HTR optical flow faces two key challenges: the absence of HTR ground-truth data and the intrinsic sparsity of event data. Most existing approaches rely on the flow accumulation paradigms to indirectly supervise intermediate flows, often resulting in accumulation errors and optimization difficulties. To address these challenges, we propose a residual-based paradigm for estimating HTR optical flow with event data. Our approach separates HTR flow estimation into two stages: global linear motion estimation and HTR residual flow refinement. The residual paradigm effectively mitigates the impacts of event sparsity on optimization and is compatible with any LTR algorithm. Next, to address the challenge posed by the absence of HTR ground truth, we incorporate novel learning strategies. Specifically, we initially employ a shared refiner to estimate the residual flows, enabling both LTR supervision and HTR inference. Subsequently, we introduce regional noise to simulate the residual patterns of intermediate flows, facilitating the adaptation from LTR supervision to HTR inference. Additionally, we show that the noise-based strategy supports in-domain self-supervised training. Comprehensive experimental results demonstrate that our approach achieves state-of-the-art accuracy in both LTR and HTR metrics, highlighting its effectiveness and superiority.

Related papers

• PreAdaptFWI: Pretrained-Based Adaptive Residual Learning for Full-Waveform Inversion Without Dataset Dependency [8.719356558714246]
  Full-waveform inversion (FWI) is a method that utilizes seismic data to invert the physical parameters of subsurface media. Due to its ill-posed nature, FWI is susceptible to getting trapped in local minima. Various research efforts have attempted to combine neural networks with FWI to stabilize the inversion process.
  arXiv  Detail & Related papers  (2025-02-17T15:30:17Z)
• Provably Efficient RLHF Pipeline: A Unified View from Contextual Bandits [59.30310692855397]
  We propose a unified framework for the RLHF pipeline from the view of contextual bandits. We decompose the RLHF process into two distinct stages: (post-)training and deployment. We then develop novel algorithms for each stage, demonstrating significant improvements in both statistical and computational efficiency.
  arXiv  Detail & Related papers  (2025-02-11T02:36:01Z)
• Zeroth-order Informed Fine-Tuning for Diffusion Model: A Recursive Likelihood Ratio Optimizer [9.153197757307762]
  probabilistic diffusion model (DM) is a powerful framework for visual generation. How to efficiently align the foundation DM is a crucial task. We propose the Recursive Likelihood Ratio (RLR), a zeroth-order informed fine-tuning paradigm for DM.
  arXiv  Detail & Related papers  (2025-02-02T03:00:26Z)
• Efficient Diffusion as Low Light Enhancer [63.789138528062225]
  Reflectance-Aware Trajectory Refinement (RATR) is a simple yet effective module to refine the teacher trajectory using the reflectance component of images. textbfReflectance-aware textbfDiffusion with textbfDistilled textbfTrajectory (textbfReDDiT) is an efficient and flexible distillation framework tailored for Low-Light Image Enhancement (LLIE)
  arXiv  Detail & Related papers  (2024-10-16T08:07:18Z)
• Unbiased and Efficient Self-Supervised Incremental Contrastive Learning [31.763904668737304]
  We propose a self-supervised Incremental Contrastive Learning (ICL) framework consisting of a novel Incremental InfoNCE (NCE-II) loss function. ICL achieves up to 16.7x training speedup and 16.8x faster convergence with competitive results.
  arXiv  Detail & Related papers  (2023-01-28T06:11:31Z)
• Enhancing Deep Traffic Forecasting Models with Dynamic Regression [15.31488551912888]
  This paper introduces a dynamic regression (DR) framework to enhance existing traffic forecasting models by structured learning for the residual process. We evaluate the effectiveness of the proposed framework on deep traffic forecasting models using both speed and flow datasets.
  arXiv  Detail & Related papers  (2023-01-17T01:12:44Z)
• ProgressiveMotionSeg: Mutually Reinforced Framework for Event-Based Motion Segmentation [101.19290845597918]
  This paper presents a Motion Estimation (ME) module and an Event Denoising (ED) module jointly optimized in a mutually reinforced manner. Taking temporal correlation as guidance, ED module calculates the confidence that each event belongs to real activity events, and transmits it to ME module to update energy function of motion segmentation for noise suppression.
  arXiv  Detail & Related papers  (2022-03-22T13:40:26Z)
• False Correlation Reduction for Offline Reinforcement Learning [115.11954432080749]
  We propose falSe COrrelation REduction (SCORE) for offline RL, a practically effective and theoretically provable algorithm. We empirically show that SCORE achieves the SoTA performance with 3.1x acceleration on various tasks in a standard benchmark (D4RL)
  arXiv  Detail & Related papers  (2021-10-24T15:34:03Z)
• FPCR-Net: Feature Pyramidal Correlation and Residual Reconstruction for Optical Flow Estimation [72.41370576242116]
  We propose a semi-supervised Feature Pyramidal Correlation and Residual Reconstruction Network (FPCR-Net) for optical flow estimation from frame pairs. It consists of two main modules: pyramid correlation mapping and residual reconstruction. Experiment results show that the proposed scheme achieves the state-of-the-art performance, with improvement by 0.80, 1.15 and 0.10 in terms of average end-point error (AEE) against competing baseline methods.
  arXiv  Detail & Related papers  (2020-01-17T07:13:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.