Thermal-Det: Language-Guided Cross-Modal Distillation for Open-Vocabulary Thermal Object Detection

• URL: http://arxiv.org/abs/2605.10130v1
• Date: Mon, 11 May 2026 07:41:37 GMT
• Title: Thermal-Det: Language-Guided Cross-Modal Distillation for Open-Vocabulary Thermal Object Detection
• Abstract summary: Existing open-vocabulary detectors focus on RGB images and fail to generalize to thermal imagery.<n>We present Thermal-Det, the first large language model (LLM) supervised open-vocabulary detector tailored for thermal images.
• Score: 41.947503656624285
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Existing open-vocabulary detectors focus on RGB images and fail to generalize to thermal imagery, where low texture and emissivity variations challenge RGB-based semantics. We present Thermal-Det, the first large language model (LLM) supervised open-vocabulary detector tailored for thermal images. To enable large-scale training, we develop a synthetic dataset by converting GroundingCap-1M into the thermal domain and filtering captions to remove RGB-specific terms, yielding over one million thermally aligned samples with bounding boxes, grounding texts, and detailed captions. Thermal-Det jointly optimizes detection, captioning, and cross-modal distillation objectives. A frozen RGB teacher provides geometric and semantic pseudo-supervision for paired but unlabeled RGB-thermal data, transferring open-vocabulary knowledge without manual annotation. The model further employs a Thermal-Text Alignment Head for text calibration and a Modality-Fused Cross-Attention module for dual-modality reasoning. Unlike prior domain-adaptation methods, the detector is fully fine-tuned to internalize thermal contrast patterns while preserving language alignment. Experiments on public benchmarks show consistent 2-4% AP gains over existing open-vocabulary detectors, establishing a strong foundation for scalable, language-driven thermal perception.

Related papers

• Text2Thermal: Physics-Aware Thermal Image Synthesis from Textual Priors [7.213400161126316]
  Text2Thermal is a framework for physics-aware thermal image synthesis from textual priors.<n>Text2Thermal synthesizes thermal imagery without requiring a registered RGB image at inference.<n>On M3FD and FLIR, Text2Thermal achieves state-of-the-art FID among thermal image synthesis methods.
  arXiv  Detail & Related papers  (2026-09-03T09:33:09Z)
• SC-Diff: Semantically Calibrated Diffusion for Visible-to-Infrared Image Translation [52.62632888060867]
  Visible-to-infrared image translation provides a practical way to expand infrared training data using abundant visible images.<n>Existing diffusion-based methods typically use semantic priors only as external conditions.<n>We propose SC-Diff, a semantically latent diffusion framework that uses semantic priors for both conditional guidance and internal self-attention calibration.
  arXiv  Detail & Related papers  (2026-08-09T07:57:40Z)
• Thermo-VL: Extending Vision-Language Models to Thermal Infrared Perception [45.307537865864994]
  Vision-language models (VLMs) often fail under low illumination because their visual grounding is learned predominantly from RGB imagery.<n>We present Thermo-VL, a wavelength-aware VLM that augments a frozen Molmo-7B backbone with a trainable thermal encoder and a text-guided dual-attention fusion module.<n>Experiments show strong gains on challenging thermal-only and RGB+thermal reasoning tasks, highlighting the value of prompt-conditioned multispectral fusion.
  arXiv  Detail & Related papers  (2026-05-21T01:43:05Z)
• TextLDM: Language Modeling with Continuous Latent Diffusion [89.69255520673248]
  Diffusion Transformers (DiT) trained with flow matching in a VAE latent space have unified visual generation across images and videos.<n>We propose TextLDM, which transfers the visual latent diffusion recipe to text generation with minimal architectural modification.
  arXiv  Detail & Related papers  (2026-05-08T13:54:34Z)
• Bridging the RGB-IR Gap: Consensus and Discrepancy Modeling for Text-Guided Multispectral Detection [45.62297680190076]
  We propose a semantic bridge fusion framework with bi-support modeling for multispectral object detection.<n>Specifically, text is used as a shared semantic bridge to align RGB and IR responses under a unified category condition.<n>We formulate RGB-IR interaction evidence into the regular consensus support and the complementary discrepancy support.
  arXiv  Detail & Related papers  (2026-04-13T09:41:01Z)
• TherA: Thermal-Aware Visual-Language Prompting for Controllable RGB-to-Thermal Infrared Translation [12.591408054941027]
  TherA is a controllable RGB-to-TIR translation framework that produces diverse and thermally plausible images at both scene and object level.<n>TherA achieves state-of-the-art translation performance, demonstrating improved zero-shot translation performance up to 33% increase averaged across all metrics.
  arXiv  Detail & Related papers  (2026-02-23T01:56:29Z)
• ThermEval: A Structured Benchmark for Evaluation of Vision-Language Models on Thermal Imagery [11.547362584832769]
  Vision language models (VLMs) achieve strong performance on RGB imagery, but they do not generalize to thermal images.<n> thermal sensing plays a critical role in settings where visible light fails, including nighttime surveillance, search and rescue, autonomous driving, and medical screening.<n>We introduce ThermEval-B, a benchmark to assess the foundational primitives required for thermal vision language understanding.
  arXiv  Detail & Related papers  (2026-02-16T18:16:19Z)
• ThermalGen: Style-Disentangled Flow-Based Generative Models for RGB-to-Thermal Image Translation [14.108149959967095]
  Paired RGB-thermal data is crucial for visual-thermal sensor fusion and cross-modality tasks.<n>To overcome this challenge, RGB-to-Thermal (RGB-T) image translation has emerged as a promising solution.<n>We propose ThermalGen, an adaptive flow-based generative model for RGB-T image translation.
  arXiv  Detail & Related papers  (2025-09-29T14:55:51Z)
• Alignment-Free RGBT Salient Object Detection: Semantics-guided Asymmetric Correlation Network and A Unified Benchmark [15.435695491233982]
  RGB and Thermal (RGBT) Salient Object Detection (SOD) aims to achieve high-quality saliency prediction. Existing methods are tailored for manually aligned image pairs, which are labor-intensive. We make the first attempt to address RGBT SOD for initially captured RGB and thermal image pairs without manual alignment.
  arXiv  Detail & Related papers  (2024-06-03T01:01:58Z)
• Does Thermal Really Always Matter for RGB-T Salient Object Detection? [153.17156598262656]
  This paper proposes a network named TNet to solve the RGB-T salient object detection (SOD) task. In this paper, we introduce a global illumination estimation module to predict the global illuminance score of the image. On the other hand, we introduce a two-stage localization and complementation module in the decoding phase to transfer object localization cue and internal integrity cue in thermal features to the RGB modality.
  arXiv  Detail & Related papers  (2022-10-09T13:50:12Z)
• Bi-directional Cross-Modality Feature Propagation with Separation-and-Aggregation Gate for RGB-D Semantic Segmentation [59.94819184452694]
  Depth information has proven to be a useful cue in the semantic segmentation of RGBD images for providing a geometric counterpart to the RGB representation. Most existing works simply assume that depth measurements are accurate and well-aligned with the RGB pixels and models the problem as a cross-modal feature fusion. In this paper, we propose a unified and efficient Crossmodality Guided to not only effectively recalibrate RGB feature responses, but also to distill accurate depth information via multiple stages and aggregate the two recalibrated representations alternatively.
  arXiv  Detail & Related papers  (2020-07-17T18:35:24Z)
• Exploring Thermal Images for Object Detection in Underexposure Regions for Autonomous Driving [67.69430435482127]
  Underexposure regions are vital to construct a complete perception of the surroundings for safe autonomous driving. The availability of thermal cameras has provided an essential alternate to explore regions where other optical sensors lack in capturing interpretable signals. This work proposes a domain adaptation framework which employs a style transfer technique for transfer learning from visible spectrum images to thermal images.
  arXiv  Detail & Related papers  (2020-06-01T09:59:09Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.