Optimizing Multi-Modal Trackers via Sensitivity-aware Regularized Tuning

• URL: http://arxiv.org/abs/2508.17488v1
• Date: Sun, 24 Aug 2025 18:42:47 GMT
• Title: Optimizing Multi-Modal Trackers via Sensitivity-aware Regularized Tuning
• Authors: Zhiwen Chen, Jinjian Wu, Zhiyu Zhu, Yifan Zhang, Guangming Shi, Junhui Hou,
• Abstract summary: This paper tackles the challenge of optimizing multi-modal trackers by effectively adapting the pre-trained models for RGB data.<n>Existing fine-tuning paradigms oscillate between excessive freedom and over-restriction, leading to a suboptimal plasticity-stability trade-off.<n>We propose a novel sensitivity-aware regularized tuning framework, which delicately refines the learning process by incorporating intrinsic parameter sensitivities.
• Score: 112.12667472919723
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: This paper tackles the critical challenge of optimizing multi-modal trackers by effectively adapting the pre-trained models for RGB data. Existing fine-tuning paradigms oscillate between excessive freedom and over-restriction, both leading to a suboptimal plasticity-stability trade-off. To mitigate this dilemma, we propose a novel sensitivity-aware regularized tuning framework, which delicately refines the learning process by incorporating intrinsic parameter sensitivities. Through a comprehensive investigation from pre-trained to multi-modal contexts, we identify that parameters sensitive to pivotal foundational patterns and cross-domain shifts are primary drivers of this issue. Specifically, we first analyze the tangent space of pre-trained weights to measure and orient prior sensitivities, dedicated to preserving generalization. Then, we further explore transfer sensitivities during the tuning phase, emphasizing adaptability and stability. By incorporating these sensitivities as regularization terms, our method significantly enhances the transferability across modalities. Extensive experiments showcase the superior performance of the proposed method, surpassing current state-of-the-art techniques across various multi-modal tracking. The source code and models will be publicly available at https://github.com/zhiwen-xdu/SRTrack.

Related papers

• TRACE: A Generalizable Drift Detector for Streaming Data-Driven Optimization [18.46974867492826]
  Many optimization tasks involve streaming data with unknown concept drifts, posing a significant challenge as Streaming Data-Driven Optimization (SDDO)<n>We propose TRACE, a TRAnsferable Concept-drift Estimator that effectively detects distributional changes in streaming data with varying time scales.<n> Comprehensive experimental results on diverse benchmarks demonstrate the superior generalization, robustness, and effectiveness of our approach in SDDO scenarios.
  arXiv  Detail & Related papers  (2025-12-08T01:33:16Z)
• ResAD: Normalized Residual Trajectory Modeling for End-to-End Autonomous Driving [64.42138266293202]
  ResAD is a Normalized Residual Trajectory Modeling framework.<n>It reframes the learning task to predict the residual deviation from an inertial reference.<n>On the NAVSIM benchmark, ResAD achieves a state-of-the-art PDMS of 88.6 using a vanilla diffusion policy.
  arXiv  Detail & Related papers  (2025-10-09T17:59:36Z)
• RAMCT: Novel Region-adaptive Multi-channel Tracker with Iterative Tikhonov Regularization for Thermal Infrared Tracking [10.58716694795395]
  We propose RAMCT, a region-adaptive sparse correlation filter tracker.<n>It integrates multi-channel feature opti-mization with an adaptive regularization strategy.<n>It outperforms other state-of-the-art trackers in terms of accuracy and robustness.
  arXiv  Detail & Related papers  (2025-04-19T12:18:36Z)
• UP-dROM : Uncertainty-Aware and Parametrised dynamic Reduced-Order Model, application to unsteady flows [27.50487430169627]
  Reduced order models (ROMs) play a critical role in fluid mechanics by providing low-cost predictions.<n>For ROMs to be widely applicable, they must not only generalise well across different regimes, but also provide a measure of confidence in their predictions.<n>We present a nonlinear reduction strategy specifically designed for transient flows.
  arXiv  Detail & Related papers  (2025-03-29T22:17:36Z)
• Direct Preference Optimization-Enhanced Multi-Guided Diffusion Model for Traffic Scenario Generation [0.0]
  Diffusion-based models are recognized for their effectiveness in using real-world driving data to generate realistic traffic scenarios.<n>These models employ guided sampling to incorporate specific traffic preferences and enhance scenario realism.<n>We introduce a multi-guided diffusion model that utilizes a novel training strategy to closely adhere to traffic priors.
  arXiv  Detail & Related papers  (2025-02-14T05:29:43Z)
• Large Continual Instruction Assistant [59.585544987096974]
  Continual Instruction Tuning (CIT) is adopted to instruct Large Models to follow human intent data by data.<n>Existing update gradient would heavily destroy the performance on previous datasets during CIT process.<n>We propose a general continual instruction tuning framework to address the challenge.
  arXiv  Detail & Related papers  (2024-10-08T11:24:59Z)
• Middle Fusion and Multi-Stage, Multi-Form Prompts for Robust RGB-T Tracking [1.8843687952462744]
  M3PT is a novel RGB-T prompt tracking method that leverages middle fusion and multi-modal and multi-stage visual prompts to overcome challenges. Based on the meta-framework, we utilize multiple flexible prompt strategies to adapt the pre-trained model to comprehensive exploration of uni-modal patterns.
  arXiv  Detail & Related papers  (2024-03-27T02:06:25Z)
• End-to-End Meta-Bayesian Optimisation with Transformer Neural Processes [52.818579746354665]
  This paper proposes the first end-to-end differentiable meta-BO framework that generalises neural processes to learn acquisition functions via transformer architectures. We enable this end-to-end framework with reinforcement learning (RL) to tackle the lack of labelled acquisition data.
  arXiv  Detail & Related papers  (2023-05-25T10:58:46Z)
• An automatic differentiation system for the age of differential privacy [65.35244647521989]
  Tritium is an automatic differentiation-based sensitivity analysis framework for differentially private (DP) machine learning (ML) We introduce Tritium, an automatic differentiation-based sensitivity analysis framework for differentially private (DP) machine learning (ML)
  arXiv  Detail & Related papers  (2021-09-22T08:07:42Z)
• Extrapolation for Large-batch Training in Deep Learning [72.61259487233214]
  We show that a host of variations can be covered in a unified framework that we propose. We prove the convergence of this novel scheme and rigorously evaluate its empirical performance on ResNet, LSTM, and Transformer.
  arXiv  Detail & Related papers  (2020-06-10T08:22:41Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.