Current Injection Spiking Neural Network for Infrared and Visible Image Fusion

• URL: http://arxiv.org/abs/2607.19879v1
• Date: Wed, 22 Jul 2026 08:10:02 GMT
• Title: Current Injection Spiking Neural Network for Infrared and Visible Image Fusion
• Abstract summary: Infrared and visible image fusion (IVIF) integrates the complementary information of two modalities into a single image with richer scene content.<n>We propose CIS-Fuse, a spiking network that performs cross-modal fusion directly at the membrane-potential level.<n>Experiments on four IVIF benchmarks and on downstream detection and segmentation show that CIS-Fuse achieves fusion quality on par with state-of-the-art ANN-based methods.
• Score: 55.20584003351512
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Infrared and visible image fusion (IVIF) integrates the complementary information of two modalities into a single image with richer scene content. While existing methods are largely built on artificial neural networks (ANNs), which densely compute over all activations, spiking neural networks (SNNs) communicate through sparse binary spikes and compute only where and when a spike occurs, offering a route to more energy-efficient fusion. However, directly applying SNNs to IVIF creates a fundamental tension: cross-modal fusion relies on fine-grained responses from both modalities, whereas binary spikes can discard complementary cues that remain below the firing threshold. The membrane potential retains these subthreshold responses before firing, letting both modalities jointly shape the output when integrated at this stage. Building on this, we propose CIS-Fuse, a spiking network that performs cross-modal fusion directly at the membrane-potential level. At its core is the current injection spiking (CIS) operator, which injects one modality as a gated auxiliary current into the driving neuron of the other, so the two integrate before spike firing, with a per-channel learnable injection strength that adaptively regulates the modulation magnitude. Building on CIS, we construct a bidirectional cross-modal fusion (BCMF) module and deploy it on a dual-branch architecture with asymmetric stacking depths, where the two branches develop a clear functional specialization. Extensive experiments on four IVIF benchmarks and on downstream detection and segmentation show that CIS-Fuse achieves fusion quality on par with state-of-the-art ANN-based methods while inheriting the energy efficiency of spike-based computation, with roughly an order of magnitude lower inference energy than the similarly-sized ANN-based DCEvo. Code will be released upon publication.

Related papers

• LI-DSN: A Layer-wise Interactive Dual-Stream Network for EEG Decoding [7.8846145693499246]
  Current dual-stream neural networks for EEG often process temporal and spatial features independently through parallel branches.<n>We propose LI-DSN, a layer-wise interactive dual-stream network that facilitates progressive, cross-stream communication at each layer.<n> LI-DSN significantly outperforms 13 state-of-the-art (SOTA) baseline models, showcasing its superior robustness and decoding performance.
  arXiv  Detail & Related papers  (2026-04-02T10:52:04Z)
• Brain-Inspired Multimodal Spiking Neural Network for Image-Text Retrieval [79.04396446873932]
  Spiking neural networks (SNNs) have recently shown strong potential in unimodal visual and textual tasks.<n>Building a directly trained, low-energy, and high-performance SNN for multimodal applications such as image-text retrieval (ITR) remains highly challenging.<n>We present a brain-inspired Cross-Modal Spike Fusion network (CMSF) and apply it to ITR for the first time.
  arXiv  Detail & Related papers  (2026-03-25T08:41:07Z)
• SCRNet: Spatial-Channel Regulation Network for Medical Ultrasound Image Segmentation [1.4930126157970809]
  CNN-based methods tend to disregard long-range dependencies, while Transformer-based methods may overlook local contextual information.<n>We propose a novel Feature Aggregation Module (FAM) designed to process two input features from the preceding layer.<n>This strategy enables our module to focus concurrently on both long-range dependencies and local contextual information.
  arXiv  Detail & Related papers  (2025-08-19T15:02:27Z)
• CodeBrain: Towards Decoupled Interpretability and Multi-Scale Architecture for EEG Foundation Model [52.466542039411515]
  EEG foundation models (EFMs) have emerged to address the scalability issues of task-specific models.<n>We present CodeBrain, a two-stage EFM designed to fill this gap.<n>In the first stage, we introduce the TFDual-Tokenizer, which decouples heterogeneous temporal and frequency EEG signals into discrete tokens.<n>In the second stage, we propose the multi-scale EEGSSM architecture, which combines structured global convolution with sliding window attention.
  arXiv  Detail & Related papers  (2025-06-10T17:20:39Z)
• Channel Fingerprint Construction for Massive MIMO: A Deep Conditional Generative Approach [65.47969413708344]
  We introduce the concept of CF twins and design a conditional generative diffusion model (CGDM)<n>We employ a variational inference technique to derive the evidence lower bound (ELBO) for the log-marginal distribution of the observed fine-grained CF conditioned on the coarse-grained CF.<n>We show that the proposed approach exhibits significant improvement in reconstruction performance compared to the baselines.
  arXiv  Detail & Related papers  (2025-05-12T01:36:06Z)
• Double-Shot 3D Shape Measurement with a Dual-Branch Network for Structured Light Projection Profilometry [14.749887303860717]
  We propose a dual-branch Convolutional Neural Network (CNN)-Transformer network (PDCNet) to process different structured light (SL) modalities.<n>Within PDCNet, a Transformer branch is used to capture global perception in the fringe images, while a CNN branch is designed to collect local details in the speckle images.<n>Our method can reduce fringe order ambiguity while producing high-accuracy results on self-made datasets.
  arXiv  Detail & Related papers  (2024-07-19T10:49:26Z)
• GAN-driven Electromagnetic Imaging of 2-D Dielectric Scatterers [4.510838705378781]
  Inverse scattering problems are inherently challenging, given the fact they are ill-posed and nonlinear. This paper presents a powerful deep learning-based approach that relies on generative adversarial networks. A cohesive inverse neural network (INN) framework is set up comprising a sequence of appropriately designed dense layers. The trained INN demonstrates an enhanced robustness, evidenced by a mean binary cross-entropy (BCE) loss of $0.13$ and a structure similarity index (SSI) of $0.90$.
  arXiv  Detail & Related papers  (2024-02-16T17:03:08Z)
• CDDFuse: Correlation-Driven Dual-Branch Feature Decomposition for Multi-Modality Image Fusion [138.40422469153145]
  We propose a novel Correlation-Driven feature Decomposition Fusion (CDDFuse) network. We show that CDDFuse achieves promising results in multiple fusion tasks, including infrared-visible image fusion and medical image fusion.
  arXiv  Detail & Related papers  (2022-11-26T02:40:28Z)
• Bandwidth-efficient distributed neural network architectures with application to body sensor networks [73.02174868813475]
  This paper describes a conceptual design methodology to design distributed neural network architectures. We show that the proposed framework enables up to a factor 20 in bandwidth reduction with minimal loss. While the application focus of this paper is on wearable brain-computer interfaces, the proposed methodology can be applied in other sensor network-like applications as well.
  arXiv  Detail & Related papers  (2022-10-14T12:35:32Z)
• ACMP: Allen-Cahn Message Passing with Attractive and Repulsive Forces for Graph Neural Networks [29.755769118770733]
  We model Allen-Cahn message passing in graph neural networks.<n>The dynamics of the system is a reaction-diffusion process which can separate particles without blowing up.<n>It propels the network depth up to one hundred layers with theoretically proven strictly positive lower bound of the Dirichlet energy.
  arXiv  Detail & Related papers  (2022-06-11T06:26:12Z)
• X-volution: On the unification of convolution and self-attention [52.80459687846842]
  We propose a multi-branch elementary module composed of both convolution and self-attention operation. The proposed X-volution achieves highly competitive visual understanding improvements.
  arXiv  Detail & Related papers  (2021-06-04T04:32:02Z)
• Diffusion Mechanism in Residual Neural Network: Theory and Applications [12.573746641284849]
  In many learning tasks with limited training samples, the diffusion connects the labeled and unlabeled data points. We propose a novel diffusion residual network (Diff-ResNet) internally introduces diffusion into the architectures of neural networks. Under the structured data assumption, it is proved that the proposed diffusion block can increase the distance-diameter ratio that improves the separability of inter-class points.
  arXiv  Detail & Related papers  (2021-05-07T10:42:59Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.