MMIF-AMIN: Adaptive Loss-Driven Multi-Scale Invertible Dense Network for Multimodal Medical Image Fusion

• URL: http://arxiv.org/abs/2508.08679v1
• Date: Tue, 12 Aug 2025 06:55:38 GMT
• Title: MMIF-AMIN: Adaptive Loss-Driven Multi-Scale Invertible Dense Network for Multimodal Medical Image Fusion
• Authors: Tao Luo, Weihua Xu,
• Abstract summary: Multimodal medical image fusion (MMIF) aims to integrate images from different modalities to produce a comprehensive image.<n>Capturing both the unique and complementary information across multiple modalities simultaneously is a key research challenge in MMIF.<n>This paper proposes a novel image fusion method, MMIF-AMIN, which features a new architecture that can effectively extract these unique and complementary features.
• Score: 3.507349476832754
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Multimodal medical image fusion (MMIF) aims to integrate images from different modalities to produce a comprehensive image that enhances medical diagnosis by accurately depicting organ structures, tissue textures, and metabolic information. Capturing both the unique and complementary information across multiple modalities simultaneously is a key research challenge in MMIF. To address this challenge, this paper proposes a novel image fusion method, MMIF-AMIN, which features a new architecture that can effectively extract these unique and complementary features. Specifically, an Invertible Dense Network (IDN) is employed for lossless feature extraction from individual modalities. To extract complementary information between modalities, a Multi-scale Complementary Feature Extraction Module (MCFEM) is designed, which incorporates a hybrid attention mechanism, convolutional layers of varying sizes, and Transformers. An adaptive loss function is introduced to guide model learning, addressing the limitations of traditional manually-designed loss functions and enhancing the depth of data mining. Extensive experiments demonstrate that MMIF-AMIN outperforms nine state-of-the-art MMIF methods, delivering superior results in both quantitative and qualitative analyses. Ablation experiments confirm the effectiveness of each component of the proposed method. Additionally, extending MMIF-AMIN to other image fusion tasks also achieves promising performance.

Related papers

• HistoSpeckle-Net: Mutual Information-Guided Deep Learning for high-fidelity reconstruction of complex OrganAMNIST images via perturbed Multimode Fibers [0.0]
  HistoSpeckle-Net is a deep learning architecture designed to reconstruct structurally rich medical images from MMF speckles.<n>Our experiments on the complex OrganAMNIST dataset demonstrate that HistoSpeckle-Net achieves higher fidelity than baseline models.
  arXiv  Detail & Related papers  (2025-11-25T12:20:50Z)
• Task-Generalized Adaptive Cross-Domain Learning for Multimodal Image Fusion [15.666336202108862]
  Multimodal Image Fusion (MMIF) aims to integrate complementary information from different imaging modalities to overcome the limitations of individual sensors.<n>Current MMIF methods face challenges such as modality misalignment, high-frequency detail destruction, and task-specific limitations.<n>We propose AdaSFFuse, a novel framework for task-generalized MMIF through adaptive cross-domain co-fusion learning.
  arXiv  Detail & Related papers  (2025-08-21T12:31:14Z)
• DM-FNet: Unified multimodal medical image fusion via diffusion process-trained encoder-decoder [13.87371547830489]
  Multimodal medical image fusion (MMIF) extracts the most meaningful information from multiple source images.<n>Existing MMIF methods have limited capacity to capture detailed features during conventional training.<n>This study proposes a two-stage diffusion model-based fusion network (DM-FNet) to achieve unified MMIF.
  arXiv  Detail & Related papers  (2025-06-18T07:55:06Z)
• Multi-modal Contrastive Learning for Tumor-specific Missing Modality Synthesis [1.4132765964347058]
  High-quality multi-modal MRI in a clinical setting is difficult due to time constraints, high costs, and patient movement artifacts.<n>Our team, PLAVE, design a generative model for missing MRI that integrates multi-modal contrastive learning with a focus on critical tumor regions.<n>Our results in the Brain MR Image Synthesis challenge demonstrate that the proposed model excelled in generating the missing modality.
  arXiv  Detail & Related papers  (2025-02-26T18:34:58Z)
• Modality Prompts for Arbitrary Modality Salient Object Detection [57.610000247519196]
  This paper delves into the task of arbitrary modality salient object detection (AM SOD) It aims to detect salient objects from arbitrary modalities, eg RGB images, RGB-D images, and RGB-D-T images. A novel modality-adaptive Transformer (MAT) will be proposed to investigate two fundamental challenges of AM SOD.
  arXiv  Detail & Related papers  (2024-05-06T11:02:02Z)
• NeuroPictor: Refining fMRI-to-Image Reconstruction via Multi-individual Pretraining and Multi-level Modulation [55.51412454263856]
  This paper proposes to directly modulate the generation process of diffusion models using fMRI signals. By training with about 67,000 fMRI-image pairs from various individuals, our model enjoys superior fMRI-to-image decoding capacity.
  arXiv  Detail & Related papers  (2024-03-27T02:42:52Z)
• AdaFuse: Adaptive Medical Image Fusion Based on Spatial-Frequential Cross Attention [6.910879180358217]
  We propose AdaFuse, in which multimodal image information is fused adaptively through frequency-guided attention mechanism. The proposed method outperforms state-of-the-art methods in terms of both visual quality and quantitative metrics.
  arXiv  Detail & Related papers  (2023-10-09T07:10:30Z)
• Equivariant Multi-Modality Image Fusion [124.11300001864579]
  We propose the Equivariant Multi-Modality imAge fusion paradigm for end-to-end self-supervised learning. Our approach is rooted in the prior knowledge that natural imaging responses are equivariant to certain transformations. Experiments confirm that EMMA yields high-quality fusion results for infrared-visible and medical images.
  arXiv  Detail & Related papers  (2023-05-19T05:50:24Z)
• Multi-modal Aggregation Network for Fast MR Imaging [85.25000133194762]
  We propose a novel Multi-modal Aggregation Network, named MANet, which is capable of discovering complementary representations from a fully sampled auxiliary modality. In our MANet, the representations from the fully sampled auxiliary and undersampled target modalities are learned independently through a specific network. Our MANet follows a hybrid domain learning framework, which allows it to simultaneously recover the frequency signal in the $k$-space domain.
  arXiv  Detail & Related papers  (2021-10-15T13:16:59Z)
• Robust Multimodal Brain Tumor Segmentation via Feature Disentanglement and Gated Fusion [71.87627318863612]
  We propose a novel multimodal segmentation framework which is robust to the absence of imaging modalities. Our network uses feature disentanglement to decompose the input modalities into the modality-specific appearance code. We validate our method on the important yet challenging multimodal brain tumor segmentation task with the BRATS challenge dataset.
  arXiv  Detail & Related papers  (2020-02-22T14:32:04Z)
• Hi-Net: Hybrid-fusion Network for Multi-modal MR Image Synthesis [143.55901940771568]
  We propose a novel Hybrid-fusion Network (Hi-Net) for multi-modal MR image synthesis. In our Hi-Net, a modality-specific network is utilized to learn representations for each individual modality. A multi-modal synthesis network is designed to densely combine the latent representation with hierarchical features from each modality.
  arXiv  Detail & Related papers  (2020-02-11T08:26:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.