Spatio-Temporal Conditional Denoising Transformer for Modality-Missing RGBT Tracking

• URL: http://arxiv.org/abs/2607.24701v1
• Date: Mon, 27 Jul 2026 17:44:00 GMT
• Title: Spatio-Temporal Conditional Denoising Transformer for Modality-Missing RGBT Tracking
• Abstract summary: We propose a Spatio-temporal Conditional Denoising Transformer (SCDT) for robust modality-missing RGBT tracking.<n>SCDT integrates the spatial cues and the temporal context to adaptively perform information reconstruction of missing modalities.<n>Our method consistently outperforms state-of-the-art methods.
• Score: 22.984699044574175
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Missing modalities in RGBT tracking often lead to incomplete and unstable multimodal feature representations that greatly degrade the performance. Existing methods typically attempt to recover missing modalities from available ones, but the quality of data generated in challenging scenarios might be unsatisfactory. In addition, current approaches exhibit limited flexibility in processing both missing and complete data. To overcome these limitations, we propose a Spatio-temporal Conditional Denoising Transformer (SCDT), which integrates the spatial cues and the temporal context to adaptively perform information reconstruction of missing modalities and feature enhancement of weak modalities in a unified framework, for robust modality-missing RGBT tracking. In particular, SCDT leverages the short-term temporal cues from recent historical frames to capture the fine-grained temporal correlations and the long-term temporal cues encoding modality evolution to capture the global context. By jointly exploiting long short-term temporal contexts as the conditions, SCDT progressively guides noisy features of available modalities to learn reliable and temporally consistent multimodal representations. Furthermore, SCDT introduces a noisemodulated adaptation mechanism that dynamically adjusts its behavior according to the modal availability, enabling a single framework to unify feature learning under both modality-missing and complete scenarios without changing the architecture or parameters. Extensive experiments on three public benchmark datasets demonstrate that our method consistently outperforms state-of-the-art methods. The code is available here.

Related papers

• SegDiff: Segmented Trajectory Diffusion for Consistent and Adaptive Robot Manipulation [65.81476183180527]
  We introduce SegDiff, a closed-loop visuomotor policy that integrates the strengths of both paradigms.<n>SegDiff decomposes demonstrations into motion segments between keyposes and learns to predict the continuous trajectory from the current state to the next keypose.<n>SegDiff demonstrates significant performance gains over existing approaches across various simulated and real-world scenarios.
  arXiv  Detail & Related papers  (2026-07-13T02:48:03Z)
• TTCD:Transformer Integrated Temporal Causal Discovery from Non-Stationary Time Series Data [0.18097009159779617]
  We propose an end-to-end approach that learns contemporaneous and lagged causal relations from non-stationary time series.<n>A key innovation is causal signal distillation, to distill essential causal signals through the reconstruction process of the transformer decoder.<n>Experiments on synthetic, benchmark, and real world datasets show that TTCD consistently outperforms state-of-the-art baselines in both accuracy and consistency with domain knowledge.
  arXiv  Detail & Related papers  (2026-04-27T19:44:38Z)
• TimelineReasoner: Advancing Timeline Summarization with Large Reasoning Models [54.737142557469916]
  We propose TimelineReasoner, a framework that shifts Timeline Summarization from static generation to an active, reasoning-driven process.<n>We show that TimelineReasoner significantly outperforms existing LLM-based TLS methods in terms of timeline accuracy, coverage, and coherence.<n>This work not only pushes the boundaries of TLS but also highlights the broader potential of LRM-based reasoning frameworks for timeline summarization.
  arXiv  Detail & Related papers  (2026-04-03T12:30:15Z)
• Temporally Decoupled Diffusion Planning for Autonomous Driving [6.701508758642954]
  Motion planning in dynamic urban environments requires balancing immediate safety with long-term goals.<n>We propose Temporally Decoupled Diffusion Model (TDDM), which reformulates trajectory generation via a noise-as-mask paradigm.<n>By partitioning trajectories into segments with independent noise levels, we implicitly treat high noise as information voids and weak noise as contextual cues.<n>This compels the model to corrupted near-term states by leveraging internal correlations with better-preserved temporal contexts.
  arXiv  Detail & Related papers  (2026-03-26T14:04:15Z)
• MEMTS: Internalizing Domain Knowledge via Parameterized Memory for Retrieval-Free Domain Adaptation of Time Series Foundation Models [51.506429027626005]
  Memory for Time Series (MEMTS) is a lightweight and plug-and-play method for retrieval-free domain adaptation in time series forecasting.<n>Key component of MEMTS is a Knowledge Persistence Module (KPM), which internalizes domain-specific temporal dynamics.<n>This paradigm shift enables MEMTS to achieve accurate domain adaptation with constant-time inference and near-zero latency.
  arXiv  Detail & Related papers  (2026-02-14T14:00:06Z)
• A Representation-Consistent Gated Recurrent Framework for Robust Medical Time-Series Classification [0.0]
  We propose a representation-consistent gated recurrent framework (RC-GRF) that introduces a principled regularization strategy to enforce temporal consistency in hidden-state representations.<n>The proposed framework is model-agnostic and can be integrated into existing gated recurrent architectures without modifying their internal gating mechanisms.
  arXiv  Detail & Related papers  (2026-02-10T17:16:49Z)
• Contextual and Seasonal LSTMs for Time Series Anomaly Detection [49.50689313712684]
  We propose a novel prediction-based framework named Contextual and Seasonal LSTMs (CS-LSTMs)<n>CS-LSTMs are built upon a noise decomposition strategy and jointly leverage contextual dependencies and seasonal patterns.<n>They consistently outperform state-of-the-art methods, highlighting their effectiveness and practical value in robust time series anomaly detection.
  arXiv  Detail & Related papers  (2026-02-10T11:46:15Z)
• UniDiff: A Unified Diffusion Framework for Multimodal Time Series Forecasting [90.47915032778366]
  We propose UniDiff, a unified diffusion framework for multimodal time series forecasting.<n>At its core lies a unified and parallel fusion module, where a single cross-attention mechanism integrates structural information from timestamps and semantic context from texts.<n>Experiments on real-world benchmark datasets across eight domains demonstrate that the proposed UniDiff model achieves state-of-the-art performance.
  arXiv  Detail & Related papers  (2025-12-08T05:36:14Z)
• Real-Time Inference for Distributed Multimodal Systems under Communication Delay Uncertainty [37.15356899831919]
  Connected cyber-physical systems perform inference based on real-time inputs from multiple data streams.<n>We propose a novel neuro-inspired non-blocking inference paradigm that employs adaptive temporal windows of integration.<n>Our framework achieves robust real-time inference with finer-grained control over the accuracy-latency tradeoff.
  arXiv  Detail & Related papers  (2025-11-20T10:48:54Z)
• Unsupervised Online 3D Instance Segmentation with Synthetic Sequences and Dynamic Loss [52.28880405119483]
  Unsupervised online 3D instance segmentation is a fundamental yet challenging task.<n>Existing methods, such as UNIT, have made progress in this direction but remain constrained by limited training diversity.<n>We propose a new framework that enriches the training distribution through synthetic point cloud sequence generation.
  arXiv  Detail & Related papers  (2025-09-27T08:53:27Z)
• Modality-missing RGBT Tracking: Invertible Prompt Learning and High-quality Benchmarks [21.139161163767884]
  Modal information might miss due to factors such as thermal sensor self-calibration and data transmission error.<n>We propose a novel invertible prompt learning approach, which integrates the content-preserving prompts into a well-trained tracking model.<n>Our method achieves significant performance improvements compared with state-of-the-art methods.
  arXiv  Detail & Related papers  (2023-12-25T11:39:00Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.