Dynamic Brightness Adaptation for Robust Multi-modal Image Fusion

• URL: http://arxiv.org/abs/2411.04697v1
• Date: Thu, 07 Nov 2024 13:31:07 GMT
• Title: Dynamic Brightness Adaptation for Robust Multi-modal Image Fusion
• Authors: Yiming Sun, Bing Cao, Pengfei Zhu, Qinghua Hu,
• Abstract summary: Visible imaging in real-world scenarios is susceptible to dynamic environmental brightness fluctuations, leading to texture degradation. We propose the Brightness Adaptive multimodal dynamic fusion framework (BA-Fusion), which achieves robust image fusion despite dynamic brightness fluctuations. Our method surpasses state-of-the-art methods in preserving multi-modal image information and visual fidelity, while exhibiting remarkable robustness across varying brightness levels.
• Score: 53.72174230024836
• License:
• Abstract: Infrared and visible image fusion aim to integrate modality strengths for visually enhanced, informative images. Visible imaging in real-world scenarios is susceptible to dynamic environmental brightness fluctuations, leading to texture degradation. Existing fusion methods lack robustness against such brightness perturbations, significantly compromising the visual fidelity of the fused imagery. To address this challenge, we propose the Brightness Adaptive multimodal dynamic fusion framework (BA-Fusion), which achieves robust image fusion despite dynamic brightness fluctuations. Specifically, we introduce a Brightness Adaptive Gate (BAG) module, which is designed to dynamically select features from brightness-related channels for normalization, while preserving brightness-independent structural information within the source images. Furthermore, we propose a brightness consistency loss function to optimize the BAG module. The entire framework is tuned via alternating training strategies. Extensive experiments validate that our method surpasses state-of-the-art methods in preserving multi-modal image information and visual fidelity, while exhibiting remarkable robustness across varying brightness levels. Our code is available: https://github.com/SunYM2020/BA-Fusion.

Related papers

• LTCF-Net: A Transformer-Enhanced Dual-Channel Fourier Framework for Low-Light Image Restoration [1.049712834719005]
  We introduce LTCF-Net, a novel network architecture designed for enhancing low-light images. Our approach utilizes two color spaces - LAB and YUV - to efficiently separate and process color information. Our model incorporates the Transformer architecture to comprehensively understand image content.
  arXiv  Detail & Related papers  (2024-11-24T07:21:17Z)
• ZePo: Zero-Shot Portrait Stylization with Faster Sampling [61.14140480095604]
  This paper presents an inversion-free portrait stylization framework based on diffusion models that accomplishes content and style feature fusion in merely four sampling steps. We propose a feature merging strategy to amalgamate redundant features in Consistency Features, thereby reducing the computational load of attention control.
  arXiv  Detail & Related papers  (2024-08-10T08:53:41Z)
• CodeEnhance: A Codebook-Driven Approach for Low-Light Image Enhancement [97.95330185793358]
  Low-light image enhancement (LLIE) aims to improve low-illumination images. Existing methods face two challenges: uncertainty in restoration from diverse brightness degradations and loss of texture and color information. We propose a novel enhancement approach, CodeEnhance, by leveraging quantized priors and image refinement.
  arXiv  Detail & Related papers  (2024-04-08T07:34:39Z)
• AoSRNet: All-in-One Scene Recovery Networks via Multi-knowledge Integration [17.070755601209136]
  We propose an all-in-one scene recovery network via multi-knowledge integration (termed AoSRNet) It combines gamma correction (GC) and optimized linear stretching (OLS) to create the detail enhancement module (DEM) and color restoration module ( CRM) Comprehensive experimental results demonstrate the effectiveness and stability of AoSRNet compared to other state-of-the-art methods.
  arXiv  Detail & Related papers  (2024-02-06T06:12:03Z)
• A Non-Uniform Low-Light Image Enhancement Method with Multi-Scale Attention Transformer and Luminance Consistency Loss [11.585269110131659]
  Low-light image enhancement aims to improve the perception of images collected in dim environments. Existing methods cannot adaptively extract the differentiated luminance information, which will easily cause over-exposure and under-exposure. We propose a multi-scale attention Transformer named MSATr, which sufficiently extracts local and global features for light balance to improve the visual quality.
  arXiv  Detail & Related papers  (2023-12-27T10:07:11Z)
• Diving into Darkness: A Dual-Modulated Framework for High-Fidelity Super-Resolution in Ultra-Dark Environments [51.58771256128329]
  This paper proposes a specialized dual-modulated learning framework that attempts to deeply dissect the nature of the low-light super-resolution task. We develop Illuminance-Semantic Dual Modulation (ISDM) components to enhance feature-level preservation of illumination and color details. Comprehensive experiments showcases the applicability and generalizability of our approach to diverse and challenging ultra-low-light conditions.
  arXiv  Detail & Related papers  (2023-09-11T06:55:32Z)
• Equivariant Multi-Modality Image Fusion [124.11300001864579]
  We propose the Equivariant Multi-Modality imAge fusion paradigm for end-to-end self-supervised learning. Our approach is rooted in the prior knowledge that natural imaging responses are equivariant to certain transformations. Experiments confirm that EMMA yields high-quality fusion results for infrared-visible and medical images.
  arXiv  Detail & Related papers  (2023-05-19T05:50:24Z)
• Multi-modal Gated Mixture of Local-to-Global Experts for Dynamic Image Fusion [59.19469551774703]
  Infrared and visible image fusion aims to integrate comprehensive information from multiple sources to achieve superior performances on various practical tasks. We propose a dynamic image fusion framework with a multi-modal gated mixture of local-to-global experts. Our model consists of a Mixture of Local Experts (MoLE) and a Mixture of Global Experts (MoGE) guided by a multi-modal gate.
  arXiv  Detail & Related papers  (2023-02-02T20:06:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.