MAVFusion: Efficient Infrared and Visible Video Fusion via Motion-Aware Sparse Interaction

• URL: http://arxiv.org/abs/2604.01958v1
• Date: Thu, 02 Apr 2026 12:20:45 GMT
• Title: MAVFusion: Efficient Infrared and Visible Video Fusion via Motion-Aware Sparse Interaction
• Abstract summary: Infrared and visible video fusion combines the object saliency from infrared images with the texture details from visible images to produce semantically rich fusion results.<n>Current methods improve temporal consistency by introducing interactions across frames, but they often require high computational cost.<n>We propose MAVFusion, an end-to-end video fusion framework featuring a motion-aware sparse interaction mechanism.
• Score: 22.27085934763657
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Infrared and visible video fusion combines the object saliency from infrared images with the texture details from visible images to produce semantically rich fusion results. However, most existing methods are designed for static image fusion and cannot effectively handle frame-to-frame motion in videos. Current video fusion methods improve temporal consistency by introducing interactions across frames, but they often require high computational cost. To mitigate these challenges, we propose MAVFusion, an end-to-end video fusion framework featuring a motion-aware sparse interaction mechanism that enhances efficiency while maintaining superior fusion quality. Specifically, we leverage optical flow to identify dynamic regions in multi-modal sequences, adaptively allocating computationally intensive cross-modal attention to these sparse areas to capture salient transitions and facilitate inter-modal information exchange. For static background regions, a lightweight weak interaction module is employed to maintain structural and appearance integrity. By decoupling the processing of dynamic and static regions, MAVFusion simultaneously preserves temporal consistency and fine-grained details while significantly accelerating inference. Extensive experiments demonstrate that MAVFusion achieves state-of-the-art performance on multiple infrared and visible video benchmarks, achieving a speed of 14.16\,FPS at $640 \times 480$ resolution. The source code will be available at https://github.com/ixilai/MAVFusion.

Related papers

• Bridging Event Streams and DiT: Event-Guided Video Frame Interpolation [75.95737622994548]
  Event cameras provide high-resolution motion cues that are well suited for bridging temporal gaps.<n>We propose an adapter-based framework that incorporates event-derived cues into a pre-trained image-to-video diffusion model.
  arXiv  Detail & Related papers  (2026-08-11T04:47:44Z)
• FTPFusion: Frequency-Aware Infrared and Visible Video Fusion with Temporal Perturbation [5.5275479200431406]
  FTP-Fusion is a frequency-aware infrared and visible video fusion method based on temporal and sparse cross-modal interaction.<n> FTP-Fusion consistently outperforms state-of-the-art methods across multiple metrics in both spatial fidelity and temporal consistency.
  arXiv  Detail & Related papers  (2026-04-02T11:08:14Z)
• MambaVF: State Space Model for Efficient Video Fusion [44.038619918204496]
  MambaVF is an efficient fusion framework based on state space models (SSMs) that performs temporal modeling without explicit motion estimation.<n>MambaVF captures long-range temporal dependencies with linear complexity while significantly reducing complexity and memory costs.<n>We highlight that MambaVF enjoys high efficiency, reducing up to 92.25% of parameters and 88.79% of computational FLOPs and a 2.1x speedup compared to existing methods.
  arXiv  Detail & Related papers  (2026-02-05T18:53:47Z)
• MambaSeg: Harnessing Mamba for Accurate and Efficient Image-Event Semantic Segmentation [17.515348703686232]
  We propose MambaSeg, a novel dual-branch semantic segmentation framework that employs parallel Mamba encoders to efficiently model RGB images and event streams.<n>MambaSeg achieves state-of-the-art segmentation performance while significantly reducing computational cost, showcasing its promise for efficient, scalable, and robust multimodal perception.
  arXiv  Detail & Related papers  (2025-12-30T14:09:17Z)
• VideoFusion: A Spatio-Temporal Collaborative Network for Multi-modal Video Fusion and Restoration [26.59510171451438]
  Existing multi-sensor fusion research predominantly integrates complementary from multiple images rather than videos.<n>VideoFusion exploits cross-modal complementarity and temporal dynamics to generate context-temporally coherent videos.<n>Extensive experiments reveal that VideoFusion outperforms existing image-oriented fusion paradigms in sequential scenarios.
  arXiv  Detail & Related papers  (2025-03-30T08:27:18Z)
• Motion-aware Latent Diffusion Models for Video Frame Interpolation [51.78737270917301]
  Motion estimation between neighboring frames plays a crucial role in avoiding motion ambiguity. We propose a novel diffusion framework, motion-aware latent diffusion models (MADiff) Our method achieves state-of-the-art performance significantly outperforming existing approaches.
  arXiv  Detail & Related papers  (2024-04-21T05:09:56Z)
• MaeFuse: Transferring Omni Features with Pretrained Masked Autoencoders for Infrared and Visible Image Fusion via Guided Training [95.24751989263117]
  MaeFuse is a novel autoencoder model designed for Infrared and Visible Image Fusion (IVIF)<n>Our model utilizes a pretrained encoder from Masked Autoencoders (MAE), which facilities the omni features extraction for low-level reconstruction and high-level vision tasks.<n>MaeFuse not only introduces a novel perspective in the realm of fusion techniques but also stands out with impressive performance across various public datasets.
  arXiv  Detail & Related papers  (2024-04-17T02:47:39Z)
• Motion-Aware Video Frame Interpolation [49.49668436390514]
  We introduce a Motion-Aware Video Frame Interpolation (MA-VFI) network, which directly estimates intermediate optical flow from consecutive frames. It not only extracts global semantic relationships and spatial details from input frames with different receptive fields, but also effectively reduces the required computational cost and complexity.
  arXiv  Detail & Related papers  (2024-02-05T11:00:14Z)
• CDDFuse: Correlation-Driven Dual-Branch Feature Decomposition for Multi-Modality Image Fusion [138.40422469153145]
  We propose a novel Correlation-Driven feature Decomposition Fusion (CDDFuse) network. We show that CDDFuse achieves promising results in multiple fusion tasks, including infrared-visible image fusion and medical image fusion.
  arXiv  Detail & Related papers  (2022-11-26T02:40:28Z)
• Implicit Motion-Compensated Network for Unsupervised Video Object Segmentation [25.41427065435164]
  Unsupervised video object segmentation (UVOS) aims at automatically separating the primary foreground object(s) from the background in a video sequence. Existing UVOS methods either lack robustness when there are visually similar surroundings (appearance-based) or suffer from deterioration in the quality of their predictions because of dynamic background and inaccurate flow (flow-based) We propose an implicit motion-compensated network (IMCNet) combining complementary cues ($textiti.e.$, appearance and motion) with aligned motion information from the adjacent frames to the current frame at the feature level.
  arXiv  Detail & Related papers  (2022-04-06T13:03:59Z)
• All at Once: Temporally Adaptive Multi-Frame Interpolation with Advanced Motion Modeling [52.425236515695914]
  State-of-the-art methods are iterative solutions interpolating one frame at the time. This work introduces a true multi-frame interpolator. It utilizes a pyramidal style network in the temporal domain to complete the multi-frame task in one-shot.
  arXiv  Detail & Related papers  (2020-07-23T02:34:39Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.