Seeing Roads Through Words: A Language-Guided Framework for RGB-T Driving Scene Segmentation

• URL: http://arxiv.org/abs/2602.07343v1
• Date: Sat, 07 Feb 2026 03:52:04 GMT
• Title: Seeing Roads Through Words: A Language-Guided Framework for RGB-T Driving Scene Segmentation
• Abstract summary: RGB-Thermal fusion is a standard approach, yet existing methods apply static fusion strategies uniformly across all conditions.<n>We propose CLARITY that dynamically adapts its fusion strategy to the detected scene condition.<n> Experiments on the MFNet dataset demonstrate that CLARITY establishes a new state-of-the-art (SOTA)
• Score: 5.030754278104693
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Robust semantic segmentation of road scenes under adverse illumination, lighting, and shadow conditions remain a core challenge for autonomous driving applications. RGB-Thermal fusion is a standard approach, yet existing methods apply static fusion strategies uniformly across all conditions, allowing modality-specific noise to propagate throughout the network. Hence, we propose CLARITY that dynamically adapts its fusion strategy to the detected scene condition. Guided by vision-language model (VLM) priors, the network learns to modulate each modality's contribution based on the illumination state while leveraging object embeddings for segmentation, rather than applying a fixed fusion policy. We further introduce two mechanisms, i.e., one which preserves valid dark-object semantics that prior noise-suppression methods incorrectly discard, and a hierarchical decoder that enforces structural consistency across scales to sharpen boundaries on thin objects. Experiments on the MFNet dataset demonstrate that CLARITY establishes a new state-of-the-art (SOTA), achieving 62.3% mIoU and 77.5% mAcc.

Related papers

• IAF-Net: Illumination-Adaptive Fusion for Low-Light Urban Road Segmentation [6.81209407358202]
  We propose an end-to-end framework with illumination-adaptive fusion for robust road segmentation across different lighting conditions.<n>IAF-Net dynamically adjusts fusion weights of RGB and geometric features via the core Illumination-Adaptive Fusion (IAF) module.<n>Experiments on nuScenes-NRS show state-of-the-art overall performance among compared methods, while CARLA-MWRS further validates robustness across adverse weather conditions.
  arXiv  Detail & Related papers  (2026-05-29T07:28:08Z)
• Frequency-Guided Fusion For RGB-Thermal Semantic Segmentation [0.6187780920448871]
  We propose a multi-modal fusion architecture built upon dual ConvNeXt V2 backbones that employs stage-wise, modality-adaptive fusion strategies.<n>Experiments on MFNet and PST900 demonstrate that our lightest variant achieves 61.73% and 86.24% mIoU, respectively, with only 35.43M parameters.
  arXiv  Detail & Related papers  (2026-05-25T18:57:22Z)
• Hierarchical Consistency Learning for Test-time Adaptation in Camouflage Perception [50.278200968044665]
  Camouflaged object detection (COD) aims to localize targets that exhibit minimal perceptual differences from backgrounds through physical attributes.<n>Existing methods, constrained by the static train-then-freeze paradigm, suffer from domain rigidity and dependency annotation.<n>We propose the hierarchical consistency learning framework, which integrates test-time adaptation for dynamic representation recalibration.
  arXiv  Detail & Related papers  (2026-05-25T09:57:46Z)
• Modality-Agnostic Prompt Learning for Multi-Modal Camouflaged Object Detection [61.36976558603528]
  We propose a novel framework that generates modality-agnostic multi-modal prompts for the Segment Anything Model (SAM)<n>Specifically, we model multi-modal learning through interactions between a data-driven content domain and a knowledge-driven prompt domain.<n>We further introduce a lightweight Mask Refine Module to calibrate coarse predictions by incorporating fine-grained prompt cues.
  arXiv  Detail & Related papers  (2026-04-14T07:13:28Z)
• Towards Controllable Low-Light Image Enhancement: A Continuous Multi-illumination Dataset and Efficient State Space Framework [2.977173671229414]
  We introduce Controllable Low-light Enhancement (CLE), a holistic framework supported by Light100.<n>To resolve the conflict between luminance control and chromatic fidelity, a noise-decoupled supervision strategy is employed.<n>Our approach achieves competitive performance and robust controllability, providing a real-world multi-illumination alternative.
  arXiv  Detail & Related papers  (2026-03-26T10:40:08Z)
• UniMotion: A Unified Framework for Motion-Text-Vision Understanding and Generation [47.383717641500844]
  UniMotion is a framework for simultaneous understanding and generation of human motion, natural language, and RGB images.<n>It treats motion as a first-class continuous modality on equal footing with RGB.<n>UniMotion achieves state-of-the-art performance across seven tasks spanning any-to-any understanding, generation, and editing.
  arXiv  Detail & Related papers  (2026-03-23T17:59:48Z)
• SLGNet: Synergizing Structural Priors and Language-Guided Modulation for Multimodal Object Detection [28.779870703756668]
  We propose SLGNet, a framework that synergizes hierarchical structural priors and language-guided modulation within a frozen Vision Transformer (ViT)-based foundation model.<n>SLGNet achieves an mAP of 66.1, while reducing trainable parameters by approximately 87% compared to traditional full fine-tuning.
  arXiv  Detail & Related papers  (2026-01-05T16:31:41Z)
• Toward Stable Semi-Supervised Remote Sensing Segmentation via Co-Guidance and Co-Fusion [31.189038928192648]
  Co2S is a semi-supervised RS segmentation framework that fuses priors from vision-language models and self-supervised models.<n>An explicit-implicit semantic co-guidance mechanism is introduced that utilizes text embeddings and learnable queries.<n>Experiments on six popular datasets demonstrate the superiority of the proposed method.
  arXiv  Detail & Related papers  (2025-12-28T18:24:19Z)
• UniMRSeg: Unified Modality-Relax Segmentation via Hierarchical Self-Supervised Compensation [104.59740403500132]
  Multi-modal image segmentation faces real-world deployment challenges from incomplete/corrupted modalities degrading performance.<n>We propose a unified modality-relax segmentation network (UniMRSeg) through hierarchical self-supervised compensation (HSSC)<n>Our approach hierarchically bridges representation gaps between complete and incomplete modalities across input, feature and output levels.
  arXiv  Detail & Related papers  (2025-09-19T17:29:25Z)
• Semantic-Enhanced Cross-Modal Place Recognition for Robust Robot Localization [1.2031796234206136]
  We introduce a framework we call Semantic-Enhanced Cross-Modal Place Recognition (SCM-PR)<n>SCM-PR combines high-level semantics utilizing RGB images for robust localization in LiDAR maps.<n>Our experimental work on the KITTI and KITTI-360 datasets show that SCM-PR achieves state-of-the-art performance.
  arXiv  Detail & Related papers  (2025-09-16T19:17:54Z)
• RUN: Reversible Unfolding Network for Concealed Object Segmentation [61.13528324971598]
  reversible strategies across both mask and RGB domains.<n>We propose the Reversible Unfolding Network (RUN), which applies reversible strategies across both mask and RGB domains.
  arXiv  Detail & Related papers  (2025-01-30T22:19:15Z)
• Multi-Modality Driven LoRA for Adverse Condition Depth Estimation [61.525312117638116]
  We propose Multi-Modality Driven LoRA (MMD-LoRA) for Adverse Condition Depth Estimation.<n>It consists of two core components: Prompt Driven Domain Alignment (PDDA) and Visual-Text Consistent Contrastive Learning (VTCCL)<n>It achieves state-of-the-art performance on the nuScenes and Oxford RobotCar datasets.
  arXiv  Detail & Related papers  (2024-12-28T14:23:58Z)
• Unleashing Network Potentials for Semantic Scene Completion [50.95486458217653]
  This paper proposes a novel SSC framework - Adrial Modality Modulation Network (AMMNet) AMMNet introduces two core modules: a cross-modal modulation enabling the interdependence of gradient flows between modalities, and a customized adversarial training scheme leveraging dynamic gradient competition. Extensive experimental results demonstrate that AMMNet outperforms state-of-the-art SSC methods by a large margin.
  arXiv  Detail & Related papers  (2024-03-12T11:48:49Z)
• Frequency Perception Network for Camouflaged Object Detection [51.26386921922031]
  We propose a novel learnable and separable frequency perception mechanism driven by the semantic hierarchy in the frequency domain.<n>Our entire network adopts a two-stage model, including a frequency-guided coarse localization stage and a detail-preserving fine localization stage.<n>Compared with the currently existing models, our proposed method achieves competitive performance in three popular benchmark datasets.
  arXiv  Detail & Related papers  (2023-08-17T11:30:46Z)
• Complementary Random Masking for RGB-Thermal Semantic Segmentation [63.93784265195356]
  RGB-thermal semantic segmentation is a potential solution to achieve reliable semantic scene understanding in adverse weather and lighting conditions. This paper proposes 1) a complementary random masking strategy of RGB-T images and 2) self-distillation loss between clean and masked input modalities. We achieve state-of-the-art performance over three RGB-T semantic segmentation benchmarks.
  arXiv  Detail & Related papers  (2023-03-30T13:57:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.