Decoupled Spatio-Temporal Consistency Learning for Self-Supervised Tracking

• URL: http://arxiv.org/abs/2507.21606v1
• Date: Tue, 29 Jul 2025 09:04:03 GMT
• Title: Decoupled Spatio-Temporal Consistency Learning for Self-Supervised Tracking
• Authors: Yaozong Zheng, Bineng Zhong, Qihua Liang, Ning Li, Shuxiang Song,
• Abstract summary: We present a Self-Supervised Tracking framework named textbftracker designed to eliminate the need of box annotations.<n>We show that tracker surpasses textitOTA self-supervised tracking methods, achieving an improvement of more than 25.3%, 20.4%, and 14.8% in AUC (AO) score on the GOT10K, LaSOT, TrackingNet datasets, respectively.
• Score: 12.910676293067231
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The success of visual tracking has been largely driven by datasets with manual box annotations. However, these box annotations require tremendous human effort, limiting the scale and diversity of existing tracking datasets. In this work, we present a novel Self-Supervised Tracking framework named \textbf{{\tracker}}, designed to eliminate the need of box annotations. Specifically, a decoupled spatio-temporal consistency training framework is proposed to learn rich target information across timestamps through global spatial localization and local temporal association. This allows for the simulation of appearance and motion variations of instances in real-world scenarios. Furthermore, an instance contrastive loss is designed to learn instance-level correspondences from a multi-view perspective, offering robust instance supervision without additional labels. This new design paradigm enables {\tracker} to effectively learn generic tracking representations in a self-supervised manner, while reducing reliance on extensive box annotations. Extensive experiments on nine benchmark datasets demonstrate that {\tracker} surpasses \textit{SOTA} self-supervised tracking methods, achieving an improvement of more than 25.3\%, 20.4\%, and 14.8\% in AUC (AO) score on the GOT10K, LaSOT, TrackingNet datasets, respectively. Code: https://github.com/GXNU-ZhongLab/SSTrack.

Related papers

• Unified People Tracking with Graph Neural Networks [39.22185669123208]
  We present a unified, fully differentiable model for multi-people tracking that learns to associate detections into trajectories.<n>The model builds a dynamic graph that aggregates spatial, contextual, and temporal information.<n>We also introduce a new scale dataset with 25 partially overlapping views, detailed scene reconstructions, and extensive occlusions.
  arXiv  Detail & Related papers  (2025-07-11T11:17:25Z)
• SPAMming Labels: Efficient Annotations for the Trackers of Tomorrow [35.76243023101549]
  SPAM is a video label engine that provides high-quality labels with minimal human intervention. We use a unified graph formulation to address the annotation of both detections and identity association for tracks across time. We demonstrate that trackers trained on SPAM labels achieve comparable performance to those trained on human annotations.
  arXiv  Detail & Related papers  (2024-04-17T14:33:41Z)
• Learning Tracking Representations from Single Point Annotations [49.47550029470299]
  We propose to learn tracking representations from single point annotations in a weakly supervised manner. Specifically, we propose a soft contrastive learning framework that incorporates target objectness prior to end-to-end contrastive learning.
  arXiv  Detail & Related papers  (2024-04-15T06:50:58Z)
• ACTrack: Adding Spatio-Temporal Condition for Visual Object Tracking [0.5371337604556311]
  Efficiently modeling-temporal relations of objects is a key challenge in visual object tracking (VOT) Existing methods track by appearance-based similarity or long-term relation modeling, resulting in rich temporal contexts between consecutive frames being easily overlooked. In this paper we present ACTrack, a new framework with additive pre-temporal tracking framework with large memory conditions. It preserves the quality and capabilities of the pre-trained backbone by freezing its parameters, and makes a trainable lightweight additive net to model temporal relations in tracking. We design an additive siamese convolutional network to ensure the integrity of spatial features and temporal sequence
  arXiv  Detail & Related papers  (2024-02-27T07:34:08Z)
• End-to-end Tracking with a Multi-query Transformer [96.13468602635082]
  Multiple-object tracking (MOT) is a challenging task that requires simultaneous reasoning about location, appearance, and identity of the objects in the scene over time. Our aim in this paper is to move beyond tracking-by-detection approaches, to class-agnostic tracking that performs well also for unknown object classes.
  arXiv  Detail & Related papers  (2022-10-26T10:19:37Z)
• Dynamic Supervisor for Cross-dataset Object Detection [52.95818230087297]
  Cross-dataset training in object detection tasks is complicated because the inconsistency in the category range across datasets transforms fully supervised learning into semi-supervised learning. We propose a dynamic supervisor framework that updates the annotations multiple times through multiple-updated submodels trained using hard and soft labels. In the final generated annotations, both recall and precision improve significantly through the integration of hard-label training with soft-label training.
  arXiv  Detail & Related papers  (2022-04-01T03:18:46Z)
• Video Annotation for Visual Tracking via Selection and Refinement [74.08109740917122]
  We present a new framework to facilitate bounding box annotations for video sequences. A temporal assessment network is proposed which is able to capture the temporal coherence of target locations. A visual-geometry refinement network is also designed to further enhance the selected tracking results.
  arXiv  Detail & Related papers  (2021-08-09T05:56:47Z)
• Learning to Track with Object Permanence [61.36492084090744]
  We introduce an end-to-end trainable approach for joint object detection and tracking. Our model, trained jointly on synthetic and real data, outperforms the state of the art on KITTI, and MOT17 datasets.
  arXiv  Detail & Related papers  (2021-03-26T04:43:04Z)
• A Closer Look at Temporal Sentence Grounding in Videos: Datasets and Metrics [70.45937234489044]
  We re- organize two widely-used TSGV datasets (Charades-STA and ActivityNet Captions) to make it different from the training split. We introduce a new evaluation metric "dR@$n$,IoU@$m$" to calibrate the basic IoU scores. All the results demonstrate that the re-organized datasets and new metric can better monitor the progress in TSGV.
  arXiv  Detail & Related papers  (2021-01-22T09:59:30Z)
• Learning to Count in the Crowd from Limited Labeled Data [109.2954525909007]
  We focus on reducing the annotation efforts by learning to count in the crowd from limited number of labeled samples. Specifically, we propose a Gaussian Process-based iterative learning mechanism that involves estimation of pseudo-ground truth for the unlabeled data.
  arXiv  Detail & Related papers  (2020-07-07T04:17:01Z)
• Unsupervised Multiple Person Tracking using AutoEncoder-Based Lifted Multicuts [11.72025865314187]
  We present an unsupervised multiple object tracking approach based on minimum visual features and lifted multicuts. We show that, despite being trained without using the provided annotations, our model provides competitive results on the challenging MOT Benchmark for pedestrian tracking.
  arXiv  Detail & Related papers  (2020-02-04T09:42:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.