Track-On2: Enhancing Online Point Tracking with Memory

• URL: http://arxiv.org/abs/2509.19115v1
• Date: Tue, 23 Sep 2025 15:00:18 GMT
• Title: Track-On2: Enhancing Online Point Tracking with Memory
• Authors: Görkay Aydemir, Weidi Xie, Fatma Güney,
• Abstract summary: We extend our prior model Track-On into Track-On2, a simple and efficient transformer-based model for online long-term tracking.<n>Track-On2 improves both performance and efficiency through architectural refinements, more effective use of memory, and improved synthetic training strategies.
• Score: 57.820749134569574
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: In this paper, we consider the problem of long-term point tracking, which requires consistent identification of points across video frames under significant appearance changes, motion, and occlusion. We target the online setting, i.e. tracking points frame-by-frame, making it suitable for real-time and streaming applications. We extend our prior model Track-On into Track-On2, a simple and efficient transformer-based model for online long-term tracking. Track-On2 improves both performance and efficiency through architectural refinements, more effective use of memory, and improved synthetic training strategies. Unlike prior approaches that rely on full-sequence access or iterative updates, our model processes frames causally and maintains temporal coherence via a memory mechanism, which is key to handling drift and occlusions without requiring future frames. At inference, we perform coarse patch-level classification followed by refinement. Beyond architecture, we systematically study synthetic training setups and their impact on memory behavior, showing how they shape temporal robustness over long sequences. Through comprehensive experiments, Track-On2 achieves state-of-the-art results across five synthetic and real-world benchmarks, surpassing prior online trackers and even strong offline methods that exploit bidirectional context. These results highlight the effectiveness of causal, memory-based architectures trained purely on synthetic data as scalable solutions for real-world point tracking. Project page: https://kuis-ai.github.io/track_on2

Related papers

• CoWTracker: Tracking by Warping instead of Correlation [53.834673070954494]
  We propose a dense point tracker that eschews cost volumes in favor of warping.<n>Inspired by recent advances in optical flow, our approach iteratively refines track estimates by warping features from the target frame to the query frame based on the current estimate.<n>Our model is simple and achieves state-of-the-art performance on standard dense point tracking benchmarks, including TAP-Vid-DAVIS, TAP-Vid-Kinetics, and Robo-TAP.
  arXiv  Detail & Related papers  (2026-02-04T18:58:59Z)
• Online Long-term Point Tracking in the Foundation Model Era [0.0]
  Point tracking aims to identify the same physical point across video frames and serves as a geometry-aware representation of motion.<n>This thesis addresses the problem of long-term point tracking in an online setting, where frames are processed sequentially without access to future information.<n>We introduce Track-On, a transformer-based model that treats each tracked point as a query and processes video frames one at a time.
  arXiv  Detail & Related papers  (2025-07-12T09:24:28Z)
• Tracktention: Leveraging Point Tracking to Attend Videos Faster and Better [61.381599921020175]
  Temporal consistency is critical in video prediction to ensure that outputs are coherent and free of artifacts.<n>Traditional methods, such as temporal attention and 3D convolution, may struggle with significant object motion.<n>We propose the Tracktention Layer, a novel architectural component that explicitly integrates motion information using point tracks.
  arXiv  Detail & Related papers  (2025-03-25T17:58:48Z)
• Track-On: Transformer-based Online Point Tracking with Memory [34.744546679670734]
  We introduce Track-On, a simple transformer-based model designed for online long-term point tracking.<n>Unlike prior methods that depend on full temporal modeling, our model processes video frames causally without access to future frames.<n>At inference time, it employs patch classification and refinement to identify correspondences and track points with high accuracy.
  arXiv  Detail & Related papers  (2025-01-30T17:04:11Z)
• Is a Pure Transformer Effective for Separated and Online Multi-Object Tracking? [36.5272157173876]
  Multi-Object Tracking (MOT) has demonstrated success in short-term association within the separated tracking-by-detection online paradigm.<n>In this paper, we review the concept of trajectory graphs and propose a novel perspective by representing them as directed acyclic graphs.<n>We introduce a concise Pure Transformer (PuTR) to validate the effectiveness of Transformer in unifying short- and long-term tracking for separated online MOT.
  arXiv  Detail & Related papers  (2024-05-23T02:44:46Z)
• Exploring Dynamic Transformer for Efficient Object Tracking [58.120191254379854]
  We propose DyTrack, a dynamic transformer framework for efficient tracking.<n>DyTrack automatically learns to configure proper reasoning routes for various inputs, gaining better utilization of the available computational budget.<n>Experiments on multiple benchmarks demonstrate that DyTrack achieves promising speed-precision trade-offs with only a single model.
  arXiv  Detail & Related papers  (2024-03-26T12:31:58Z)
• ODTrack: Online Dense Temporal Token Learning for Visual Tracking [22.628561792412686]
  ODTrack is a video-level tracking pipeline that densely associates contextual relationships of video frames in an online token propagation manner. It achieves a new itSOTA performance on seven benchmarks, while running at real-time speed.
  arXiv  Detail & Related papers  (2024-01-03T11:44:09Z)
• TAPIR: Tracking Any Point with per-frame Initialization and temporal Refinement [64.11385310305612]
  We present a novel model for Tracking Any Point (TAP) that effectively tracks any queried point on any physical surface throughout a video sequence. Our approach employs two stages: (1) a matching stage, which independently locates a suitable candidate point match for the query point on every other frame, and (2) a refinement stage, which updates both the trajectory and query features based on local correlations. The resulting model surpasses all baseline methods by a significant margin on the TAP-Vid benchmark, as demonstrated by an approximate 20% absolute average Jaccard (AJ) improvement on DAVIS.
  arXiv  Detail & Related papers  (2023-06-14T17:07:51Z)
• Modeling Continuous Motion for 3D Point Cloud Object Tracking [54.48716096286417]
  This paper presents a novel approach that views each tracklet as a continuous stream. At each timestamp, only the current frame is fed into the network to interact with multi-frame historical features stored in a memory bank. To enhance the utilization of multi-frame features for robust tracking, a contrastive sequence enhancement strategy is proposed.
  arXiv  Detail & Related papers  (2023-03-14T02:58:27Z)
• Context-aware Visual Tracking with Joint Meta-updating [11.226947525556813]
  We propose a context-aware tracking model to optimize the tracker over the representation space, which jointly meta-update both branches by exploiting information along the whole sequence. The proposed tracking method achieves an EAO score of 0.514 on VOT2018 with the speed of 40FPS, demonstrating its capability of improving the accuracy and robustness of the underlying tracker with little speed drop.
  arXiv  Detail & Related papers  (2022-04-04T14:16:00Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.