Event-Based Tracking Any Point with Motion-Augmented Temporal Consistency

• URL: http://arxiv.org/abs/2412.01300v1
• Date: Mon, 02 Dec 2024 09:13:29 GMT
• Title: Event-Based Tracking Any Point with Motion-Augmented Temporal Consistency
• Authors: Han Han, Wei Zhai, Yang Cao, Bin Li, Zheng-jun Zha,
• Abstract summary: This paper presents an event-based framework for tracking any point. It tackles the challenges posed by spatial sparsity and motion sensitivity in events. It achieves 150% faster processing with competitive model parameters.
• Score: 58.719310295870024
• License:
• Abstract: Tracking Any Point (TAP) plays a crucial role in motion analysis. Video-based approaches rely on iterative local matching for tracking, but they assume linear motion during the blind time between frames, which leads to target point loss under large displacements or nonlinear motion. The high temporal resolution and motion blur-free characteristics of event cameras provide continuous, fine-grained motion information, capturing subtle variations with microsecond precision. This paper presents an event-based framework for tracking any point, which tackles the challenges posed by spatial sparsity and motion sensitivity in events through two tailored modules. Specifically, to resolve ambiguities caused by event sparsity, a motion-guidance module incorporates kinematic features into the local matching process. Additionally, a variable motion aware module is integrated to ensure temporally consistent responses that are insensitive to varying velocities, thereby enhancing matching precision. To validate the effectiveness of the approach, an event dataset for tracking any point is constructed by simulation, and is applied in experiments together with two real-world datasets. The experimental results show that the proposed method outperforms existing SOTA methods. Moreover, it achieves 150\% faster processing with competitive model parameters.

Related papers

• DATAP-SfM: Dynamic-Aware Tracking Any Point for Robust Structure from Motion in the Wild [85.03973683867797]
  This paper proposes a concise, elegant, and robust pipeline to estimate smooth camera trajectories and obtain dense point clouds for casual videos in the wild. We show that the proposed method achieves state-of-the-art performance in terms of camera pose estimation even in complex dynamic challenge scenes.
  arXiv  Detail & Related papers  (2024-11-20T13:01:16Z)
• ESVO2: Direct Visual-Inertial Odometry with Stereo Event Cameras [33.81592783496106]
  Event-based visual odometry aims at solving tracking and mapping subproblems (typically in parallel) We build an event-based stereo visual-inertial odometry system on top of a direct pipeline. The resulting system scales well with modern high-resolution event cameras.
  arXiv  Detail & Related papers  (2024-10-12T05:35:27Z)
• MotionTrack: Learning Motion Predictor for Multiple Object Tracking [68.68339102749358]
  We introduce a novel motion-based tracker, MotionTrack, centered around a learnable motion predictor. Our experimental results demonstrate that MotionTrack yields state-of-the-art performance on datasets such as Dancetrack and SportsMOT.
  arXiv  Detail & Related papers  (2023-06-05T04:24:11Z)
• Continuous-Time Gaussian Process Motion-Compensation for Event-vision Pattern Tracking with Distance Fields [4.168157981135697]
  This work addresses the issue of motion compensation and pattern tracking in event camera data. The proposed method decomposes the tracking problem into a local SE(2) motion-compensation step followed by a homography registration of small motion-compensated event batches. Our open-source implementation performs high-accuracy motion compensation and produces high-quality tracks in real-world scenarios.
  arXiv  Detail & Related papers  (2023-03-05T13:48:20Z)
• ProgressiveMotionSeg: Mutually Reinforced Framework for Event-Based Motion Segmentation [101.19290845597918]
  This paper presents a Motion Estimation (ME) module and an Event Denoising (ED) module jointly optimized in a mutually reinforced manner. Taking temporal correlation as guidance, ED module calculates the confidence that each event belongs to real activity events, and transmits it to ME module to update energy function of motion segmentation for noise suppression.
  arXiv  Detail & Related papers  (2022-03-22T13:40:26Z)
• Event-based Motion Segmentation by Cascaded Two-Level Multi-Model Fitting [44.97191206895915]
  We present a cascaded two-level multi-model fitting method for identifying independently moving objects with a monocular event camera. Experiments demonstrate the effectiveness and versatility of our method in real-world scenes with different motion patterns and an unknown number of moving objects.
  arXiv  Detail & Related papers  (2021-11-05T12:59:41Z)
• Motion Deblurring with Real Events [50.441934496692376]
  We propose an end-to-end learning framework for event-based motion deblurring in a self-supervised manner. Real-world events are exploited to alleviate the performance degradation caused by data inconsistency.
  arXiv  Detail & Related papers  (2021-09-28T13:11:44Z)
• Event-based Motion Segmentation with Spatio-Temporal Graph Cuts [51.17064599766138]
  We have developed a method to identify independently objects acquired with an event-based camera. The method performs on par or better than the state of the art without having to predetermine the number of expected moving objects.
  arXiv  Detail & Related papers  (2020-12-16T04:06:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.