Learning to Track from Privileged Target Appearances

• URL: http://arxiv.org/abs/2609.02471v1
• Date: Wed, 02 Sep 2026 11:43:43 GMT
• Title: Learning to Track from Privileged Target Appearances
• Abstract summary: We quantify a bottleneck with a non-deployable oracle that supplies an exact current-frame target crop.<n>We introduce Privileged Appearance Transfer for Tracking (PATT), a teacher-student training framework that transfers these privileged appearances to a deployable tracker.<n>PATT achieves consistent gains under both long- and short-term tracking protocols.
• Score: 67.80976117600649
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Target templates define what a visual tracker searches for, yet the templates available at inference trade off localization certainty with appearance freshness: the initial ground-truth template is exact but becomes stale, whereas recent templates better reflect the current appearance but are cropped from uncertain predictions. We quantify this bottleneck with a non-deployable oracle that supplies an exact current-frame target crop, improving AUC on LaSOT by 15.2 percentage points. This gap reveals a training-only opportunity: frame-level ground truths provide exact current- and future-frame target crops, although such crops are unavailable at deployment. We introduce Privileged Appearance Transfer for Tracking (PATT), a teacher-student training framework that transfers these privileged appearances to a deployable tracker through multi-level representation prediction. The privileged teacher observes exact target crops from past, current, and future frames, whereas the student receives only past-frame templates and learns to predict the teacher's search representations. To avoid transferring unreliable teacher signals, PATT weights this transfer by the teacher's relative localization advantage over the student and its absolute localization accuracy. After training, the teacher, latent predictor, reliability weights, and privileged crops are removed, leaving standard student-only inference. Across seven benchmarks at two model scales, PATT achieves consistent gains under both long- and short-term tracking protocols.

Related papers

• Distill What the Student Can See: Fisher-Projected On-Policy Distillation for Vision-Language Models [23.382646724389545]
  On-policy distillation (OPD) samples trajectories from the current student policy and minimizes token-level divergence between student and teacher next-token distributions.<n>We propose emphFisher-Projected On-Policy Distillation (FP-OPD), which distills only locally realizable teacher corrections.
  arXiv  Detail & Related papers  (2026-08-02T14:16:14Z)
• Learning from the Future: Privileged Self-Distillation for Sequential Recommendation [51.993699802866956]
  We propose a framework that separates learning-time information from inference-time information.<n> Privileged Self-Distillation (PSD) separates learning-time information from inference-time information.<n>PSD is optimized end-to-end in a single stage, leaving the deployed model and inference cost unchanged.
  arXiv  Detail & Related papers  (2026-07-29T15:47:22Z)
• When In-Distribution Gains Fail: Evaluating Weak-to-Strong Reward Models under Preference Shift [72.0686608132772]
  We study W2S preference learning under zero-shot distribution shift.<n>We find that strong students trained on weak preference labels can appear successful in-distribution while failing to transfer across preference datasets.<n>We propose Representation Anchoring (Anchor), a simple yet effective regularizer that constrains excessive drift from the pretrained strong model's representation space.
  arXiv  Detail & Related papers  (2026-05-25T09:30:49Z)
• SAE as a Crystal Ball: Interpretable Features Predict Cross-domain Transferability of LLMs without Training [70.84726713548099]
  SAE-based Transferability Score (STS) is a new metric to forecast post-training transferability.<n>We show that STS accurately predicts the transferability of supervised fine-tuning, achieving Pearson correlation coefficients above 0.7 with actual performance changes.
  arXiv  Detail & Related papers  (2026-03-03T12:01:09Z)
• Towards the Mitigation of Confirmation Bias in Semi-supervised Learning: a Debiased Training Perspective [6.164100243945264]
  Semi-supervised learning (SSL) commonly exhibits confirmation bias, where models disproportionately favor certain classes. We introduce TaMatch, a unified framework for debiased training in SSL. We show that TaMatch significantly outperforms existing state-of-the-art methods across a range of challenging image classification tasks.
  arXiv  Detail & Related papers  (2024-09-26T21:50:30Z)
• ACTRESS: Active Retraining for Semi-supervised Visual Grounding [52.08834188447851]
  A previous study, RefTeacher, makes the first attempt to tackle this task by adopting the teacher-student framework to provide pseudo confidence supervision and attention-based supervision. This approach is incompatible with current state-of-the-art visual grounding models, which follow the Transformer-based pipeline. Our paper proposes the ACTive REtraining approach for Semi-Supervised Visual Grounding, abbreviated as ACTRESS.
  arXiv  Detail & Related papers  (2024-07-03T16:33:31Z)
• Debiased Fine-Tuning for Vision-language Models by Prompt Regularization [56.48290708901531]
  We present a new paradigm for fine-tuning large-scale vision pre-trained models on downstream task, dubbed Prompt Regularization (ProReg)<n>ProReg uses the prediction by prompting the pretrained model to regularize the fine-tuning.<n>We show the consistently strong performance of ProReg compared with conventional fine-tuning, zero-shot prompt, prompt tuning, and other state-of-the-art methods.
  arXiv  Detail & Related papers  (2023-01-29T11:53:55Z)
• Learning from Future: A Novel Self-Training Framework for Semantic Segmentation [33.66516999361252]
  Self-training has shown great potential in semi-supervised learning. We propose a novel self-training strategy, which allows the model to learn from the future. We experimentally demonstrate the effectiveness and superiority of our approach under a wide range of settings.
  arXiv  Detail & Related papers  (2022-09-15T01:39:46Z)
• Improving Localization for Semi-Supervised Object Detection [3.5493798890908104]
  We introduce an additional classification task for bounding box localization to improve the filtering of predicted bounding boxes. Our experiments show that our IL-net increases SSOD performance by 1.14% AP on dataset in limited-annotation regime.
  arXiv  Detail & Related papers  (2022-06-21T08:39:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.