QueST: Persistent Queries as Semantic Monitors for Drift Suppression in Long-Horizon Tracking

• URL: http://arxiv.org/abs/2605.09513v1
• Date: Sun, 10 May 2026 12:48:01 GMT
• Title: QueST: Persistent Queries as Semantic Monitors for Drift Suppression in Long-Horizon Tracking
• Abstract summary: QueST is a monitoring-by-design framework that treats interaction-relevant entities as persistent semantic queries.<n>Instead of local propagation, each query attends globally over-temporal video features at every time-step.<n>QueST substantially reduces terminal drift achieving a 67.7% Absolute Point Error (APE)<n>Our results show that embedding semantic monitoring directly into perception enables more reliable long-horizon tracking under shiftibility.
• Score: 2.064813317204182
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Tracking points in videos is typically formulated as frame-to-frame correspondence, where each point is matched locally to the next frame. While this works over short horizons, errors accumulate under articulation, occlusion, and viewpoint change, leading to silent semantic drift that existing trackers cannot detect or correct. In this work, we revisit long-horizon tracking from a monitoring perspective and introduce QueST, a monitoring-by-design framework that treats interaction-relevant entities as persistent semantic queries rather than transient point tracks. Instead of local propagation, each query attends globally over spatio-temporal video features at every time-step, providing a stable semantic anchor across time. We further constrain query trajectories with lightweight 3D physical grounding, using geometric plausibility to suppress unbounded drift under occlusion. We evaluate QueST on long-horizon articulated sequences from PartNet-Mobility in SAPIEN and compare against RAFT-3D, CoTracker, and TAP-Net. QueST substantially reduces terminal drift achieving a 67.7% Absolute Point Error (APE) improvement over TAP-Net while better preserving identity over extended horizons. Our results show that embedding semantic monitoring directly into perception enables more reliable long-horizon tracking under distribution shift.

Related papers

• PS-MOT: Cultivating Instance Awareness from Point Seeds for Multi-Object Tracking [33.76342104593764]
  We introduce Point-supervised Multi-Object Tracking (PS-MOT) as a cost-effective alternative to traditional bounding box supervision.<n>PS-Track provides a feasible and effective point-supervised alternative across diverse tracking scenarios.
  arXiv  Detail & Related papers  (2026-06-29T15:36:00Z)
• Efficient Spatio-Temporal Grounding with Multimodal Large Models via Second-Level Tracking and RL Verification [11.171207895220578]
  temporal-temporal grounding in long videos requires precise temporal localization and robust object tracking conditioned on natural-language queries.<n>We propose a practical pipeline that shifts from frame-level to second-level tracking and performs cross-second annotations to preserve continuity while reducing sequence length.
  arXiv  Detail & Related papers  (2026-06-27T17:42:32Z)
• PCSTracker: Long-Term Scene Flow Estimation for Point Cloud Sequences [20.800932584993706]
  PCSTracker is an end-to-end framework specifically designed for consistent scene flow estimation in point cloud sequences.<n> PCSTracker achieves the best accuracy in long-term scene flow estimation and maintains real-time performance at 32.5 FPS.
  arXiv  Detail & Related papers  (2026-03-20T08:45:27Z)
• Look Around and Pay Attention: Multi-camera Point Tracking Reimagined with Transformers [5.025261312338861]
  LAPA (Look Around and Pay Attention) is a novel end-to-end transformer-based architecture for multi-camera point tracking.<n>Instead of relying on classical triangulation, we construct 3D point representations via attention-weighted aggregation.<n>Experiments on challenging datasets, including our newly created multi-camera (MC) versions of TAPVid-3D panoptic and PointOdyssey, demonstrate that our unified approach significantly outperforms existing methods.
  arXiv  Detail & Related papers  (2025-12-03T19:34:08Z)
• Delving into Dynamic Scene Cue-Consistency for Robust 3D Multi-Object Tracking [16.366398265001422]
  3D multi-object tracking is a critical and challenging task in the field of autonomous driving.<n>We introduce the Dynamic Scene Cue-Consistency Tracker (DSC-Track) to implement this principle.
  arXiv  Detail & Related papers  (2025-08-15T08:48:13Z)
• Online Dense Point Tracking with Streaming Memory [54.22820729477756]
  Dense point tracking is a challenging task requiring the continuous tracking of every point in the initial frame throughout a substantial portion of a video.<n>Recent point tracking algorithms usually depend on sliding windows for indirect information propagation from the first frame to the current one.<n>We present a lightweight and fast model with textbfStreaming memory for dense textbfPOint textbfTracking and online video processing.
  arXiv  Detail & Related papers  (2025-03-09T06:16:49Z)
• LEAP-VO: Long-term Effective Any Point Tracking for Visual Odometry [53.5449912019877]
  We present the Long-term Effective Any Point Tracking (LEAP) module.<n>LEAP innovatively combines visual, inter-track, and temporal cues with mindfully selected anchors for dynamic track estimation.<n>Based on these traits, we develop LEAP-VO, a robust visual odometry system adept at handling occlusions and dynamic scenes.
  arXiv  Detail & Related papers  (2024-01-03T18:57:27Z)
• Tracking Objects and Activities with Attention for Temporal Sentence Grounding [51.416914256782505]
  Temporal sentence (TSG) aims to localize the temporal segment which is semantically aligned with a natural language query in an untrimmed segment. We propose a novel Temporal Sentence Tracking Network (TSTNet), which contains (A) a Cross-modal Targets Generator to generate multi-modal and search space, and (B) a Temporal Sentence Tracker to track multi-modal targets' behavior and to predict query-related segment.
  arXiv  Detail & Related papers  (2023-02-21T16:42:52Z)
• Tracking by Associating Clips [110.08925274049409]
  In this paper, we investigate an alternative by treating object association as clip-wise matching. Our new perspective views a single long video sequence as multiple short clips, and then the tracking is performed both within and between the clips. The benefits of this new approach are two folds. First, our method is robust to tracking error accumulation or propagation, as the video chunking allows bypassing the interrupted frames. Second, the multiple frame information is aggregated during the clip-wise matching, resulting in a more accurate long-range track association than the current frame-wise matching.
  arXiv  Detail & Related papers  (2022-12-20T10:33:17Z)
• Unsupervised Learning of Accurate Siamese Tracking [68.58171095173056]
  We present a novel unsupervised tracking framework, in which we can learn temporal correspondence both on the classification branch and regression branch. Our tracker outperforms preceding unsupervised methods by a substantial margin, performing on par with supervised methods on large-scale datasets such as TrackingNet and LaSOT.
  arXiv  Detail & Related papers  (2022-04-04T13:39:43Z)
• Learning to Track with Object Permanence [61.36492084090744]
  We introduce an end-to-end trainable approach for joint object detection and tracking. Our model, trained jointly on synthetic and real data, outperforms the state of the art on KITTI, and MOT17 datasets.
  arXiv  Detail & Related papers  (2021-03-26T04:43:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.