AWM-Fuse: Multi-Modality Image Fusion for Adverse Weather via Global and Local Text Perception

• URL: http://arxiv.org/abs/2508.16881v1
• Date: Sat, 23 Aug 2025 02:55:19 GMT
• Title: AWM-Fuse: Multi-Modality Image Fusion for Adverse Weather via Global and Local Text Perception
• Abstract summary: AWM-Fuse is a novel fusion method for adverse weather conditions.<n>It is designed to handle multiple degradations through global and local text perception.
• Score: 18.397042539210055
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Multi-modality image fusion (MMIF) in adverse weather aims to address the loss of visual information caused by weather-related degradations, providing clearer scene representations. Although less studies have attempted to incorporate textual information to improve semantic perception, they often lack effective categorization and thorough analysis of textual content. In response, we propose AWM-Fuse, a novel fusion method for adverse weather conditions, designed to handle multiple degradations through global and local text perception within a unified, shared weight architecture. In particular, a global feature perception module leverages BLIP-produced captions to extract overall scene features and identify primary degradation types, thus promoting generalization across various adverse weather conditions. Complementing this, the local module employs detailed scene descriptions produced by ChatGPT to concentrate on specific degradation effects through concrete textual cues, thereby capturing finer details. Furthermore, textual descriptions are used to constrain the generation of fusion images, effectively steering the network learning process toward better alignment with real semantic labels, thereby promoting the learning of more meaningful visual features. Extensive experiments demonstrate that AWM-Fuse outperforms current state-of-the-art methods in complex weather conditions and downstream tasks. Our code is available at https://github.com/Feecuin/AWM-Fuse.

Related papers

• Unleashing the Power of Text: Text-Guided Flow Matching for Image Fusion under Complex Degradations [50.60539317126614]
  Infrared-visible image fusion under realistic degradation scenarios is a challenging task.<n>Recent studies indicate that text can provide prior information about degradation characteristics.<n>We propose TGFusion, a text-guided latent-space flow matching framework.
  arXiv  Detail & Related papers  (2026-08-01T08:43:26Z)
• FB-CLIP: Fine-Grained Zero-Shot Anomaly Detection with Foreground-Background Disentanglement [21.260231526776128]
  We propose FB-CLIP, a framework that enhances anomaly localization via multi-strategy textual representations and foreground-background separation.<n>Experiments show that FB-CLIP effectively distinguishes anomalies from complex backgrounds, achieving accurate fine-grained anomaly detection and localization under zero-shot settings.
  arXiv  Detail & Related papers  (2026-03-20T03:25:56Z)
• Constrained Prompt Enhancement for Improving Zero-Shot Generalization of Vision-Language Models [57.357091028792325]
  Vision-language models (VLMs) pre-trained on web-scale data exhibit promising zero-shot generalization but often suffer from semantic misalignment.<n>We propose a novel constrained prompt enhancement (CPE) method to improve visual-textual alignment.<n>Our approach consists of two key components: Topology-Guided Synonymous Semantic Generation (TGSSG) and Category-Agnostic Discriminative Region Selection (CADRS)
  arXiv  Detail & Related papers  (2025-08-24T15:45:22Z)
• TeSG: Textual Semantic Guidance for Infrared and Visible Image Fusion [55.34830989105704]
  Infrared and visible image fusion (IVF) aims to combine complementary information from both image modalities.<n>We introduce textual semantics at two levels: the mask semantic level and the text semantic level.<n>We propose Textual Semantic Guidance for infrared and visible image fusion, which guides the image synthesis process.
  arXiv  Detail & Related papers  (2025-06-20T03:53:07Z)
• Natural Language Supervision for Low-light Image Enhancement [0.0]
  We introduce a Natural Language Supervision (NLS) strategy, which learns feature maps from text corresponding to images.<n>We also design a Textual Guidance Conditioning Mechanism (TCM) that incorporates the connections between image regions and sentence words.<n>In order to effectively identify and merge features from various levels of image and textual information, we design an Information Fusion Attention (IFA) module.
  arXiv  Detail & Related papers  (2025-01-11T13:53:10Z)
• AWRaCLe: All-Weather Image Restoration using Visual In-Context Learning [29.309503214127016]
  All-Weather Image Restoration (AWIR) under adverse weather conditions is a challenging task due to the presence of different types of degradations.<n>We propose AWRaCLe, a novel approach for AWIR that innovatively utilizes degradation-specific visual context information to steer the image restoration process.<n>AWRaCLe incorporates Degradation Context Extraction (DCE) and Context Fusion (CF) to seamlessly integrate degradation-specific features from the context into an image restoration network.
  arXiv  Detail & Related papers  (2024-08-30T21:35:25Z)
• GLoD: Composing Global Contexts and Local Details in Image Generation [0.0]
  Global-Local Diffusion (textitGLoD) is a novel framework which allows simultaneous control over the global contexts and the local details. It assigns multiple global and local prompts to corresponding layers and composes their noises to guide a denoising process. Our framework enables complex global-local compositions, conditioning objects in the global prompt with the local prompts while preserving other unspecified identities.
  arXiv  Detail & Related papers  (2024-04-23T18:39:57Z)
• TextCoT: Zoom In for Enhanced Multimodal Text-Rich Image Understanding [91.30065932213758]
  Large Multimodal Models (LMMs) have sparked a surge in research aimed at harnessing their remarkable reasoning abilities. We propose TextCoT, a novel Chain-of-Thought framework for text-rich image understanding. Our method is free of extra training, offering immediate plug-and-play functionality.
  arXiv  Detail & Related papers  (2024-04-15T13:54:35Z)
• All-weather Multi-Modality Image Fusion: Unified Framework and 100k Benchmark [42.49073228252726]
  Multi-modality image fusion (MMIF) combines complementary information from different image modalities to provide a more comprehensive and objective interpretation of scenes. Existing MMIF methods lack the ability to resist different weather interferences in real-world scenes, preventing them from being useful in practical applications such as autonomous driving. We propose an all-weather MMIF model to achieve effective multi-tasking in this context. Experimental results in both real-world and synthetic scenes show that the proposed algorithm excels in detail recovery and multi-modality feature extraction.
  arXiv  Detail & Related papers  (2024-02-03T09:02:46Z)
• Typhoon Intensity Prediction with Vision Transformer [51.84456610977905]
  We introduce "Typhoon Intensity Transformer" (Tint) to predict typhoon intensity accurately across space and time. Tint uses self-attention mechanisms with global receptive fields per layer. Experiments on a publicly available typhoon benchmark validate the efficacy of Tint.
  arXiv  Detail & Related papers  (2023-11-28T03:11:33Z)
• Image-Specific Information Suppression and Implicit Local Alignment for Text-based Person Search [61.24539128142504]
  Text-based person search (TBPS) is a challenging task that aims to search pedestrian images with the same identity from an image gallery given a query text. Most existing methods rely on explicitly generated local parts to model fine-grained correspondence between modalities. We propose an efficient joint Multi-level Alignment Network (MANet) for TBPS, which can learn aligned image/text feature representations between modalities at multiple levels.
  arXiv  Detail & Related papers  (2022-08-30T16:14:18Z)
• Good Visual Guidance Makes A Better Extractor: Hierarchical Visual Prefix for Multimodal Entity and Relation Extraction [88.6585431949086]
  We propose a novel Hierarchical Visual Prefix fusion NeTwork (HVPNeT) for visual-enhanced entity and relation extraction. We regard visual representation as pluggable visual prefix to guide the textual representation for error insensitive forecasting decision. Experiments on three benchmark datasets demonstrate the effectiveness of our method, and achieve state-of-the-art performance.
  arXiv  Detail & Related papers  (2022-05-07T02:10:55Z)
• Situational Perception Guided Image Matting [16.1897179939677]
  We propose a Situational Perception Guided Image Matting (SPG-IM) method that mitigates subjective bias of matting annotations. SPG-IM can better associate inter-objects and object-to-environment saliency, and compensate the subjective nature of image matting.
  arXiv  Detail & Related papers  (2022-04-20T07:35:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.