FTPFusion: Frequency-Aware Infrared and Visible Video Fusion with Temporal Perturbation

• URL: http://arxiv.org/abs/2604.01900v1
• Date: Thu, 02 Apr 2026 11:08:14 GMT
• Title: FTPFusion: Frequency-Aware Infrared and Visible Video Fusion with Temporal Perturbation
• Abstract summary: FTP-Fusion is a frequency-aware infrared and visible video fusion method based on temporal and sparse cross-modal interaction.<n> FTP-Fusion consistently outperforms state-of-the-art methods across multiple metrics in both spatial fidelity and temporal consistency.
• Score: 5.5275479200431406
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Infrared and visible video fusion plays a critical role in intelligent surveillance and low-light monitoring. However, maintaining temporal stability while preserving spatial detail remains a fundamental challenge. Existing methods either focus on frame-wise enhancement with limited temporal modeling or rely on heavy spatio-temporal aggregation that often sacrifices high-frequency details. In this paper, we propose FTPFusion, a frequency-aware infrared and visible video fusion method based on temporal perturbation and sparse cross-modal interaction. Specifically, FTPFusion decomposes the feature representations into high-frequency and low-frequency components for collaborative modeling. The high-frequency branch performs sparse cross-modal spatio-temporal interaction to capture motion-related context and complementary details. The low-frequency branch introduces a temporal perturbation strategy to enhance robustness against complex video variations, such as flickering, jitter, and local misalignment. Furthermore, we design an offset-aware temporal consistency constraint to explicitly stabilize cross-frame representations under temporal disturbances. Extensive experiments on multiple public benchmarks demonstrate that FTPFusion consistently outperforms state-of-the-art methods across multiple metrics in both spatial fidelity and temporal consistency. The source code will be available at https://github.com/ixilai/FTPFusion.

Related papers

• Asynchronous Multimodal Diffusion Policy Composition via Latency-Aware Guidance Fusion [52.50952605013304]
  We propose LAG-Fusion, a latency-aware guidance fusion framework for asynchronous multimodal diffusion policy composition.<n>We instantiate LAG-Fusion in contact-rich manipulation by composing a low-frequency vision policy with a high-frequency force policy.
  arXiv  Detail & Related papers  (2026-07-19T13:48:52Z)
• FATE: Pillar Encoding and Frequency-Aware Training for Event-Based Object Detection [0.0]
  Event cameras offer inherent advantages in high-speed and high-dynamic-range scenarios.<n>Current approaches partition the accumulation window into fixed temporal sub-bins.<n>We propose FATE, a unified framework built upon a novel spatial framework.
  arXiv  Detail & Related papers  (2026-06-15T22:32:09Z)
• DRFusion: Drift-Resilient Temporally Consistent Infrared-Visible Video Fusion [21.262410801452948]
  Infrared and visible video fusion is essential for achieving comprehensive perception in dynamic scenes.<n>Maintaining temporal consistency remains a formidable challenge.<n>We propose a drift-resilient video fusion method that reformulates the task as history-conditioned motion generation.
  arXiv  Detail & Related papers  (2026-05-25T12:23:55Z)
• ReMATF: Recurrent Motion-Adaptive Multi-scale Turbulence Mitigation for Dynamic Scenes [4.282240730573401]
  Atmospheric turbulence severely degrades video quality by introducing distortions such as geometric warping, blur, and temporal flickering.<n>We propose ReMATF, a lightweight recurrent framework that restores videos using only two frames at a time while preserving spatial detail and temporal stability.
  arXiv  Detail & Related papers  (2026-05-20T17:28:49Z)
• TIE: Time Interval Encoding for Video Generation over Events [50.66585165263848]
  Director-style prompting, robotic action prediction, and interactive video agents demand temporal grounding over concurrent events.<n>Modern video generators represent time as discrete points through point-wise positional encodings.<n>Time Interval TIE is a principled, plug-and-play interval-aware generalization.
  arXiv  Detail & Related papers  (2026-05-11T13:23:14Z)
• RAFT-MSF++: Temporal Geometry-Motion Feature Fusion for Self-Supervised Monocular Scene Flow [51.43025173196566]
  Monocular scene flow estimation aims to recover dense 3D motion from image sequences.<n>We propose RAFT-MSF++, a self-supervised multi-frame framework that fuses temporal features to jointly estimate depth and scene flow.<n>Experiments show that RAFT-MSF++ achieves 24.14% SF-all on the KITTI Scene Flow benchmark, with a 30.99% improvement over the baseline and better robustness in occluded regions.
  arXiv  Detail & Related papers  (2026-04-21T11:32:49Z)
• MAVFusion: Efficient Infrared and Visible Video Fusion via Motion-Aware Sparse Interaction [22.27085934763657]
  Infrared and visible video fusion combines the object saliency from infrared images with the texture details from visible images to produce semantically rich fusion results.<n>Current methods improve temporal consistency by introducing interactions across frames, but they often require high computational cost.<n>We propose MAVFusion, an end-to-end video fusion framework featuring a motion-aware sparse interaction mechanism.
  arXiv  Detail & Related papers  (2026-04-02T12:20:45Z)
• TAPFormer: Robust Arbitrary Point Tracking via Transient Asynchronous Fusion of Frames and Events [37.273066799679135]
  We introduce TAPFormer, a framework that performs temporal-consistent asynchronous fusion of frames and events for point tracking.<n>Key innovation is a Transient Asynchronous Fusion mechanism, which explicitly models the temporal evolution between discrete frames.<n>Our method outperforms existing point trackers, achieving a 28.2% improvement in average pixel error within threshold.
  arXiv  Detail & Related papers  (2026-03-05T09:32:24Z)
• PAS: A Training-Free Stabilizer for Temporal Encoding in Video LLMs [57.790910044227935]
  Video LLMs suffer from temporal inconsistency: small shifts in frame timing can flip attention and suppress relevant frames.<n>We present Phase Aggregated Smoothing (PAS), a training-free mechanism that applies small opposed phase offsets across heads and then aggregates their outputs.<n>Our analysis shows that the RoPE rotated logit can be approximated as a content dot product scaled by a time kernel; smoothing this kernel yields Lipschitz stability of attention to small temporal shifts; multi phase averaging attenuates high frequency ripples while preserving per-head spectra under Nyquist-valid sampling.
  arXiv  Detail & Related papers  (2025-11-14T05:56:47Z)
• Freqformer: Image-Demoiréing Transformer via Efficient Frequency Decomposition [83.40450475728792]
  We present Freqformer, a Transformer-based framework specifically designed for image demoir'eing through targeted frequency separation.<n>Our method performs an effective frequency decomposition that explicitly splits moir'e patterns into high-frequency spatially-localized textures and low-frequency scale-robust color distortions.<n>Experiments on various demoir'eing benchmarks demonstrate that Freqformer achieves state-of-the-art performance with a compact model size.
  arXiv  Detail & Related papers  (2025-05-25T12:23:10Z)
• FUSE: Label-Free Image-Event Joint Monocular Depth Estimation via Frequency-Decoupled Alignment and Degradation-Robust Fusion [92.4205087439928]
  Image-event joint depth estimation methods leverage complementary modalities for robust perception, yet face challenges in generalizability.<n>We propose the Self-supervised Transfer (PST) and the FrequencyDe-coupled Fusion module (FreDF)<n>PST establishes cross-modal knowledge transfer through latent space alignment with image foundation models, effectively mitigating data scarcity.<n>FreDF explicitly decouples high-frequency edge features from low-frequency structural components, resolving modality-specific frequency mismatches.<n>This combined approach enables FUSE to construct a universal image-event that only requires lightweight decoder adaptation for target datasets.
  arXiv  Detail & Related papers  (2025-03-25T15:04:53Z)
• SVDC: Consistent Direct Time-of-Flight Video Depth Completion with Frequency Selective Fusion [8.23422182642083]
  Lightweight direct Time-of-Flight (dToF) sensors are ideal for 3D sensing on mobile devices.<n>In this paper, we propose a novel video depth completion method, called SVDC, by fusing the sparse dToF data with the corresponding RGB guidance.<n>Our method employs a multi-frame fusion scheme to mitigate the spatial ambiguity resulting from the sparse dToF imaging.
  arXiv  Detail & Related papers  (2025-03-03T07:32:25Z)
• Intrinsic Temporal Regularization for High-resolution Human Video Synthesis [59.54483950973432]
  temporal consistency is crucial for extending image processing pipelines to the video domain. We propose an effective intrinsic temporal regularization scheme, where an intrinsic confidence map is estimated via the frame generator to regulate motion estimation. We apply our intrinsic temporal regulation to single-image generator, leading to a powerful " INTERnet" capable of generating $512times512$ resolution human action videos.
  arXiv  Detail & Related papers  (2020-12-11T05:29:45Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.