FusionINN: Decomposable Image Fusion for Brain Tumor Monitoring

• URL: http://arxiv.org/abs/2403.15769v3
• Date: Mon, 10 Jun 2024 13:09:53 GMT
• Title: FusionINN: Decomposable Image Fusion for Brain Tumor Monitoring
• Authors: Nishant Kumar, Ziyan Tao, Jaikirat Singh, Yang Li, Peiwen Sun, Binghui Zhao, Stefan Gumhold,
• Abstract summary: We introduce FusionINN, a novel decomposable image fusion framework. We are the first to investigate the decomposability of fused images. Our approach offers faster and qualitatively better fusion results.
• Score: 6.45135260209391
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Image fusion typically employs non-invertible neural networks to merge multiple source images into a single fused image. However, for clinical experts, solely relying on fused images may be insufficient for making diagnostic decisions, as the fusion mechanism blends features from source images, thereby making it difficult to interpret the underlying tumor pathology. We introduce FusionINN, a novel decomposable image fusion framework, capable of efficiently generating fused images and also decomposing them back to the source images. FusionINN is designed to be bijective by including a latent image alongside the fused image, while ensuring minimal transfer of information from the source images to the latent representation. To the best of our knowledge, we are the first to investigate the decomposability of fused images, which is particularly crucial for life-sensitive applications such as medical image fusion compared to other tasks like multi-focus or multi-exposure image fusion. Our extensive experimentation validates FusionINN over existing discriminative and generative fusion methods, both subjectively and objectively. Moreover, compared to a recent denoising diffusion-based fusion model, our approach offers faster and qualitatively better fusion results.

Related papers

• Fusion from Decomposition: A Self-Supervised Approach for Image Fusion and Beyond [74.96466744512992]
  The essence of image fusion is to integrate complementary information from source images. DeFusion++ produces versatile fused representations that can enhance the quality of image fusion and the effectiveness of downstream high-level vision tasks.
  arXiv  Detail & Related papers  (2024-10-16T06:28:49Z)
• Fuse4Seg: Image-Level Fusion Based Multi-Modality Medical Image Segmentation [13.497613339200184]
  We argue the current feature-level fusion strategy is prone to semantic inconsistencies and misalignments. We introduce a novel image-level fusion based multi-modality medical image segmentation method, Fuse4Seg. The resultant fused image is a coherent representation that accurately amalgamates information from all modalities.
  arXiv  Detail & Related papers  (2024-09-16T14:39:04Z)
• A New Multimodal Medical Image Fusion based on Laplacian Autoencoder with Channel Attention [3.1531360678320897]
  Deep learning models have achieved end-to-end image fusion with highly robust and accurate performance. Most DL-based fusion models perform down-sampling on the input images to minimize the number of learnable parameters and computations. We propose a new multimodal medical image fusion model is proposed that is based on integrated Laplacian-Gaussian concatenation with attention pooling.
  arXiv  Detail & Related papers  (2023-10-18T11:29:53Z)
• A Task-guided, Implicitly-searched and Meta-initialized Deep Model for Image Fusion [69.10255211811007]
  We present a Task-guided, Implicit-searched and Meta- generalizationd (TIM) deep model to address the image fusion problem in a challenging real-world scenario. Specifically, we propose a constrained strategy to incorporate information from downstream tasks to guide the unsupervised learning process of image fusion. Within this framework, we then design an implicit search scheme to automatically discover compact architectures for our fusion model with high efficiency.
  arXiv  Detail & Related papers  (2023-05-25T08:54:08Z)
• Equivariant Multi-Modality Image Fusion [124.11300001864579]
  We propose the Equivariant Multi-Modality imAge fusion paradigm for end-to-end self-supervised learning. Our approach is rooted in the prior knowledge that natural imaging responses are equivariant to certain transformations. Experiments confirm that EMMA yields high-quality fusion results for infrared-visible and medical images.
  arXiv  Detail & Related papers  (2023-05-19T05:50:24Z)
• DDFM: Denoising Diffusion Model for Multi-Modality Image Fusion [144.9653045465908]
  We propose a novel fusion algorithm based on the denoising diffusion probabilistic model (DDPM) Our approach yields promising fusion results in infrared-visible image fusion and medical image fusion.
  arXiv  Detail & Related papers  (2023-03-13T04:06:42Z)
• An Attention-based Multi-Scale Feature Learning Network for Multimodal Medical Image Fusion [24.415389503712596]
  Multimodal medical images could provide rich information about patients for physicians to diagnose. The image fusion technique is able to synthesize complementary information from multimodal images into a single image. We introduce a novel Dilated Residual Attention Network for the medical image fusion task.
  arXiv  Detail & Related papers  (2022-12-09T04:19:43Z)
• CDDFuse: Correlation-Driven Dual-Branch Feature Decomposition for Multi-Modality Image Fusion [138.40422469153145]
  We propose a novel Correlation-Driven feature Decomposition Fusion (CDDFuse) network. We show that CDDFuse achieves promising results in multiple fusion tasks, including infrared-visible image fusion and medical image fusion.
  arXiv  Detail & Related papers  (2022-11-26T02:40:28Z)
• CoCoNet: Coupled Contrastive Learning Network with Multi-level Feature Ensemble for Multi-modality Image Fusion [72.8898811120795]
  We propose a coupled contrastive learning network, dubbed CoCoNet, to realize infrared and visible image fusion. Our method achieves state-of-the-art (SOTA) performance under both subjective and objective evaluation.
  arXiv  Detail & Related papers  (2022-11-20T12:02:07Z)
• Coupled Feature Learning for Multimodal Medical Image Fusion [42.23662451234756]
  Multimodal image fusion aims to combine relevant information from images acquired with different sensors. In this paper, we propose a novel multimodal image fusion method based on coupled dictionary learning.
  arXiv  Detail & Related papers  (2021-02-17T09:13:28Z)
• FuseVis: Interpreting neural networks for image fusion using per-pixel saliency visualization [10.156766309614113]
  Unsupervised learning based convolutional neural networks (CNNs) have been utilized for different types of image fusion tasks. It is challenging to analyze the reliability of these CNNs for the image fusion tasks since no groundtruth is available. We present a novel real-time visualization tool, named FuseVis, with which the end-user can compute per-pixel saliency maps.
  arXiv  Detail & Related papers  (2020-12-06T10:03:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.