Uni-MDTrack: Learning Decoupled Memory and Dynamic States for Parameter-Efficient Visual Tracking in All Modality

• URL: http://arxiv.org/abs/2603.14452v1
• Date: Sun, 15 Mar 2026 15:56:41 GMT
• Title: Uni-MDTrack: Learning Decoupled Memory and Dynamic States for Parameter-Efficient Visual Tracking in All Modality
• Abstract summary: We propose Uni-MDTrack, which consists of two core components: Memory-Aware Compression Prompt (MCP) module and Dynamic State Fusion (DSF) module.<n>MCP effectively compresses rich memory features into memory-aware prompt tokens, which deeply interact with the input throughout the backbone.<n>DSF also introduces updated dynamic state features from shallow to deep layers, while also preserving high efficiency.
• Score: 44.07279162081132
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: With the advent of Transformer-based one-stream trackers that possess strong capability in inter-frame relation modeling, recent research has increasingly focused on how to introduce spatio-temporal context. However, most existing methods rely on a limited number of historical frames, which not only leads to insufficient utilization of the context, but also inevitably increases the length of input and incurs prohibitive computational overhead. Methods that query an external memory bank, on the other hand, suffer from inadequate fusion between the retrieved spatio-temporal features and the backbone. Moreover, using discrete historical frames as context overlooks the rich dynamics of the target. To address the issues, we propose Uni-MDTrack, which consists of two core components: Memory-Aware Compression Prompt (MCP) module and Dynamic State Fusion (DSF) module. MCP effectively compresses rich memory features into memory-aware prompt tokens, which deeply interact with the input throughout the entire backbone, significantly enhancing the performance while maintaining a stable computational load. DSF complements the discrete memory by capturing the continuous dynamic, progressively introducing the updated dynamic state features from shallow to deep layers, while also preserving high efficiency. Uni-MDTrack also supports unified tracking across RGB, RGB-D/T/E, and RGB-Language modalities. Experiments show that in Uni-MDTrack, training only the MCP, DSF, and prediction head, keeping the proportion of trainable parameters around 30%, yields substantial performance gains, achieves state-of-the-art results on 10 datasets spanning five modalities. Furthermore, both MCP and DSF exhibit excellent generality, functioning as plug-and-play components that can boost the performance of various baseline trackers, while significantly outperforming existing parameter-efficient training approaches.

Related papers

• Dynamic Context Adapters: Efficiently Infusing History into Vision-and-Language Models [19.861339694649256]
  We introduce Dynamic Context Adapter (DCA), a novel context injection approach for pretrained Vision-Language Models (VLMs)<n>Our method employs fixed-size, dynamically compressed memory to preserve historical semantics without frame concatenation.<n>DCA achieves over $25%$ reduction in attention FLOPs and $13%$ memory savings while improving performance on long-horizon tasks.
  arXiv  Detail & Related papers  (2026-08-11T06:05:05Z)
• MemDLM: Memory-Enhanced DLM Training [56.40248490616793]
  Diffusion Language Models (DLMs) offer attractive advantages over Auto-Regressive (AR) models.<n>They suffer from a notable train-inference mismatch.<n>We propose MemDLM (Memory-Enhanced DLM), which embeds a simulated denoising process into training.
  arXiv  Detail & Related papers  (2026-03-23T17:39:56Z)
• Exploring Reliable Spatiotemporal Dependencies for Efficient Visual Tracking [9.64398631601942]
  STDTrack is a framework that pioneers the integration of reliabletemporal dependencies into lightweight trackers.<n>We introduce a temporally propagatingtemporal token to guide per-frame feature extraction.<n>We develop a multi-scale prediction head to dynamically adapt to objects of different sizes.
  arXiv  Detail & Related papers  (2026-01-14T02:22:05Z)
• SEDM: Scalable Self-Evolving Distributed Memory for Agents [23.182291416527764]
  SEDM is a verifiable and adaptive framework that transforms memory from a passive repository into an active, self-optimizing component.<n>We show that SEDM improves reasoning accuracy while reducing token overhead compared with strong memory baselines.<n>Results highlight SEDM as a scalable and sustainable memory mechanism for open-ended multi-agent collaboration.
  arXiv  Detail & Related papers  (2025-09-11T14:37:37Z)
• DAF: An Efficient End-to-End Dynamic Activation Framework for on-Device DNN Training [41.09085549544767]
  We introduce a Dynamic Activation Framework (DAF) that enables scalable and efficient on-device training through system-level optimizations.<n>DAF achieves both memory- and time-efficient dynamic quantization training by addressing key system bottlenecks.<n> Evaluations on various deep learning models across embedded and mobile platforms demonstrate up to a $22.9times$ reduction in memory usage and a $3.2times$ speedup.
  arXiv  Detail & Related papers  (2025-07-09T08:59:30Z)
• Learning Frequency and Memory-Aware Prompts for Multi-Modal Object Tracking [74.15663758681849]
  We present Learning Frequency and Memory-Aware Prompts, a dual-adapter framework that injects lightweight prompts into a frozen RGB tracker.<n>A frequency-guided visual adapter adaptively transfers complementary cues across modalities.<n>A multilevel memory adapter with short, long, and permanent memory stores, updates, and retrieves reliable temporal context.
  arXiv  Detail & Related papers  (2025-06-30T15:38:26Z)
• MEM1: Learning to Synergize Memory and Reasoning for Efficient Long-Horizon Agents [84.62985963113245]
  We introduce MEM1, an end-to-end reinforcement learning framework that enables agents to operate with constant memory across long multi-turn tasks.<n>At each turn, MEM1 updates a compact shared internal state that jointly supports memory consolidation and reasoning.<n>We show that MEM1-7B improves performance by 3.5x while reducing memory usage by 3.7x compared to Qwen2.5-14B-Instruct on a 16-objective multi-hop QA task.
  arXiv  Detail & Related papers  (2025-06-18T19:44:46Z)
• Online Dense Point Tracking with Streaming Memory [54.22820729477756]
  Dense point tracking is a challenging task requiring the continuous tracking of every point in the initial frame throughout a substantial portion of a video.<n>Recent point tracking algorithms usually depend on sliding windows for indirect information propagation from the first frame to the current one.<n>We present a lightweight and fast model with textbfStreaming memory for dense textbfPOint textbfTracking and online video processing.
  arXiv  Detail & Related papers  (2025-03-09T06:16:49Z)
• ContextFormer: Redefining Efficiency in Semantic Segmentation [48.81126061219231]
  Convolutional methods, although capturing local dependencies well, struggle with long-range relationships.<n>Vision Transformers (ViTs) excel in global context capture but are hindered by high computational demands.<n>We propose ContextFormer, a hybrid framework leveraging the strengths of CNNs and ViTs in the bottleneck to balance efficiency, accuracy, and robustness for real-time semantic segmentation.
  arXiv  Detail & Related papers  (2025-01-31T16:11:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.