Coupled Feature Learning for Multimodal Medical Image Fusion

• URL: http://arxiv.org/abs/2102.08641v1
• Date: Wed, 17 Feb 2021 09:13:28 GMT
• Title: Coupled Feature Learning for Multimodal Medical Image Fusion
• Authors: Farshad G. Veshki, Nora Ouzir, Sergiy A. Vorobyov, Esa Ollila
• Abstract summary: Multimodal image fusion aims to combine relevant information from images acquired with different sensors. In this paper, we propose a novel multimodal image fusion method based on coupled dictionary learning.
• Score: 42.23662451234756
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Multimodal image fusion aims to combine relevant information from images acquired with different sensors. In medical imaging, fused images play an essential role in both standard and automated diagnosis. In this paper, we propose a novel multimodal image fusion method based on coupled dictionary learning. The proposed method is general and can be employed for different medical imaging modalities. Unlike many current medical fusion methods, the proposed approach does not suffer from intensity attenuation nor loss of critical information. Specifically, the images to be fused are decomposed into coupled and independent components estimated using sparse representations with identical supports and a Pearson correlation constraint, respectively. An alternating minimization algorithm is designed to solve the resulting optimization problem. The final fusion step uses the max-absolute-value rule. Experiments are conducted using various pairs of multimodal inputs, including real MR-CT and MR-PET images. The resulting performance and execution times show the competitiveness of the proposed method in comparison with state-of-the-art medical image fusion methods.

Related papers

• Simultaneous Tri-Modal Medical Image Fusion and Super-Resolution using Conditional Diffusion Model [2.507050016527729]
  Tri-modal medical image fusion can provide a more comprehensive view of the disease's shape, location, and biological activity. Due to the limitations of imaging equipment and considerations for patient safety, the quality of medical images is usually limited. There is an urgent need for a technology that can both enhance image resolution and integrate multi-modal information.
  arXiv  Detail & Related papers  (2024-04-26T12:13:41Z)
• A New Multimodal Medical Image Fusion based on Laplacian Autoencoder with Channel Attention [3.1531360678320897]
  Deep learning models have achieved end-to-end image fusion with highly robust and accurate performance. Most DL-based fusion models perform down-sampling on the input images to minimize the number of learnable parameters and computations. We propose a new multimodal medical image fusion model is proposed that is based on integrated Laplacian-Gaussian concatenation with attention pooling.
  arXiv  Detail & Related papers  (2023-10-18T11:29:53Z)
• Three-Dimensional Medical Image Fusion with Deformable Cross-Attention [10.26573411162757]
  Multimodal medical image fusion plays an instrumental role in several areas of medical image processing. Traditional fusion methods tend to process each modality independently before combining the features and reconstructing the fusion image. In this study, we introduce an innovative unsupervised feature mutual learning fusion network designed to rectify these limitations.
  arXiv  Detail & Related papers  (2023-10-10T04:10:56Z)
• AdaFuse: Adaptive Medical Image Fusion Based on Spatial-Frequential Cross Attention [6.910879180358217]
  We propose AdaFuse, in which multimodal image information is fused adaptively through frequency-guided attention mechanism. The proposed method outperforms state-of-the-art methods in terms of both visual quality and quantitative metrics.
  arXiv  Detail & Related papers  (2023-10-09T07:10:30Z)
• Equivariant Multi-Modality Image Fusion [124.11300001864579]
  We propose the Equivariant Multi-Modality imAge fusion paradigm for end-to-end self-supervised learning. Our approach is rooted in the prior knowledge that natural imaging responses are equivariant to certain transformations. Experiments confirm that EMMA yields high-quality fusion results for infrared-visible and medical images.
  arXiv  Detail & Related papers  (2023-05-19T05:50:24Z)
• DDFM: Denoising Diffusion Model for Multi-Modality Image Fusion [144.9653045465908]
  We propose a novel fusion algorithm based on the denoising diffusion probabilistic model (DDPM) Our approach yields promising fusion results in infrared-visible image fusion and medical image fusion.
  arXiv  Detail & Related papers  (2023-03-13T04:06:42Z)
• An Attention-based Multi-Scale Feature Learning Network for Multimodal Medical Image Fusion [24.415389503712596]
  Multimodal medical images could provide rich information about patients for physicians to diagnose. The image fusion technique is able to synthesize complementary information from multimodal images into a single image. We introduce a novel Dilated Residual Attention Network for the medical image fusion task.
  arXiv  Detail & Related papers  (2022-12-09T04:19:43Z)
• Hyperspectral-Multispectral Image Fusion with Weighted LASSO [68.04032419397677]
  We propose an approach for fusing hyperspectral and multispectral images to provide high-quality hyperspectral output. We demonstrate that the proposed sparse fusion and reconstruction provides quantitatively superior results when compared to existing methods on publicly available images.
  arXiv  Detail & Related papers  (2020-03-15T23:07:56Z)
• Robust Multimodal Brain Tumor Segmentation via Feature Disentanglement and Gated Fusion [71.87627318863612]
  We propose a novel multimodal segmentation framework which is robust to the absence of imaging modalities. Our network uses feature disentanglement to decompose the input modalities into the modality-specific appearance code. We validate our method on the important yet challenging multimodal brain tumor segmentation task with the BRATS challenge dataset.
  arXiv  Detail & Related papers  (2020-02-22T14:32:04Z)
• Hi-Net: Hybrid-fusion Network for Multi-modal MR Image Synthesis [143.55901940771568]
  We propose a novel Hybrid-fusion Network (Hi-Net) for multi-modal MR image synthesis. In our Hi-Net, a modality-specific network is utilized to learn representations for each individual modality. A multi-modal synthesis network is designed to densely combine the latent representation with hierarchical features from each modality.
  arXiv  Detail & Related papers  (2020-02-11T08:26:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.