T-CLIP: Enabling Thermal Perception for Contrastive Language-Image Pretraining

• URL: http://arxiv.org/abs/2606.00673v1
• Date: Sat, 30 May 2026 11:03:58 GMT
• Title: T-CLIP: Enabling Thermal Perception for Contrastive Language-Image Pretraining
• Abstract summary: Thermal imaging offers a powerful alternative to visible-spectrum vision under challenging conditions such as low illumination and adverse weather.<n>Yet foundational vision-language models like CLIP fail to align thermal images with textual descriptions due to a fundamental thermal perception gap.<n>We introduce IR-Cap, the first physics-aware thermal captioning pipeline and dataset, and T-CLIP, a decoupled dual-LoRA framework that independently adapts CLIP for scene-level and object-level thermal understanding.
• Score: 10.818599367617919
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Thermal imaging offers a powerful alternative to visible-spectrum vision under challenging conditions such as low illumination and adverse weather, yet foundational vision-language models like CLIP fail to align thermal images with textual descriptions due to a fundamental thermal perception gap. We identify three major challenges: the lack of captioned thermal datasets, the inability of standard LLMs to reason about thermal phenomena, and a key representational challenge in thermal imaging where global scene context and object-level heat signatures conflict when learned together in a single embedding space. To address these, we introduce IR-Cap, the first physics-aware thermal captioning pipeline and dataset providing complementary global and fine-grained thermal descriptions across three public benchmarks, and T-CLIP, a decoupled dual-LoRA framework that independently adapts CLIP for scene-level and object-level thermal understanding. T-CLIP achieves consistent improvements over all baselines across three thermal benchmarks in cross-modal retrieval, and we provide an exploratory demonstration of its applicability to text-conditioned thermal image generation.

Related papers

• PL-LIT: A LiDAR-Inertial-Thermal SLAM Using Point-Line Features and Thermographic Mapping [39.002163918691465]
  Thermal imaging is resilient to adverse conditions, such as intense illumination, low-light operation, and fog.<n>Most thermal cameras employ automatic gain control (AGC), and thermal images often present low global contrast despite containing informative edge structures.<n>We propose a general LiDAR-Inertial-Thermal SLAM system that accommodates both visible-light and thermal cameras.
  arXiv  Detail & Related papers  (2026-06-28T08:13:43Z)
• Thermal-Det: Language-Guided Cross-Modal Distillation for Open-Vocabulary Thermal Object Detection [41.947503656624285]
  Existing open-vocabulary detectors focus on RGB images and fail to generalize to thermal imagery.<n>We present Thermal-Det, the first large language model (LLM) supervised open-vocabulary detector tailored for thermal images.
  arXiv  Detail & Related papers  (2026-05-11T07:41:37Z)
• ThermEval: A Structured Benchmark for Evaluation of Vision-Language Models on Thermal Imagery [11.547362584832769]
  Vision language models (VLMs) achieve strong performance on RGB imagery, but they do not generalize to thermal images.<n> thermal sensing plays a critical role in settings where visible light fails, including nighttime surveillance, search and rescue, autonomous driving, and medical screening.<n>We introduce ThermEval-B, a benchmark to assess the foundational primitives required for thermal vision language understanding.
  arXiv  Detail & Related papers  (2026-02-16T18:16:19Z)
• Learning to Reconstruct Temperature Field from Sparse Observations with Implicit Physics Priors [22.013633764284936]
  High cost of measurement acquisition and substantial distributional shifts in temperature field present challenges for developing reconstruction models.<n>We propose IPTR, an implicit physics-guided temperature field reconstruction framework.<n>We show that IPTR consistently outperforms existing methods, achieving state-of-the-art reconstruction accuracy and strong generalization capability.
  arXiv  Detail & Related papers  (2025-12-01T02:22:30Z)
• Light of Normals: Unified Feature Representation for Universal Photometric Stereo [69.95514862547174]
  Current encoders cannot guarantee that illumination and normal information are decoupled.<n>We introduce LINO UniPS with two key components: (i) Light Register Tokens with light alignment supervision to aggregate point, direction, and environment lights.<n>We also introduce PS-Verse, a large-scale synthetic dataset graded by geometric complexity and lighting diversity.
  arXiv  Detail & Related papers  (2025-06-23T17:53:11Z)
• ThermalLoc: A Vision Transformer-Based Approach for Robust Thermal Camera Relocalization in Large-Scale Environments [11.322440891972901]
  Thermal cameras capture environmental data through heat emission.<n>Traditional visual relocalization methods not directly applicable to thermal images.<n>We introduce ThermalLoc, a novel end-to-end deep learning method for thermal image relocalization.
  arXiv  Detail & Related papers  (2025-06-23T03:52:35Z)
• Does Thermal Really Always Matter for RGB-T Salient Object Detection? [153.17156598262656]
  This paper proposes a network named TNet to solve the RGB-T salient object detection (SOD) task. In this paper, we introduce a global illumination estimation module to predict the global illuminance score of the image. On the other hand, we introduce a two-stage localization and complementation module in the decoding phase to transfer object localization cue and internal integrity cue in thermal features to the RGB modality.
  arXiv  Detail & Related papers  (2022-10-09T13:50:12Z)
• Thermal to Visible Image Synthesis under Atmospheric Turbulence [67.99407460140263]
  In biometrics and surveillance, thermal imagining modalities are often used to capture images in low-light and nighttime conditions. Such imaging systems often suffer from atmospheric turbulence, which introduces severe blur and deformation artifacts to the captured images. An end-to-end reconstruction method is proposed which can directly transform thermal images into visible-spectrum images.
  arXiv  Detail & Related papers  (2022-04-06T19:47:41Z)
• A Synthesis-Based Approach for Thermal-to-Visible Face Verification [105.63410428506536]
  This paper presents an algorithm that achieves state-of-the-art performance on the ARL-VTF and TUFTS multi-spectral face datasets. We also present MILAB-VTF(B), a challenging multi-spectral face dataset composed of paired thermal and visible videos.
  arXiv  Detail & Related papers  (2021-08-21T17:59:56Z)
• A Large-Scale, Time-Synchronized Visible and Thermal Face Dataset [62.193924313292875]
  We present the DEVCOM Army Research Laboratory Visible-Thermal Face dataset (ARL-VTF) With over 500,000 images from 395 subjects, the ARL-VTF dataset represents to the best of our knowledge, the largest collection of paired visible and thermal face images to date. This paper presents benchmark results and analysis on thermal face landmark detection and thermal-to-visible face verification by evaluating state-of-the-art models on the ARL-VTF dataset.
  arXiv  Detail & Related papers  (2021-01-07T17:17:12Z)
• Exploring Thermal Images for Object Detection in Underexposure Regions for Autonomous Driving [67.69430435482127]
  Underexposure regions are vital to construct a complete perception of the surroundings for safe autonomous driving. The availability of thermal cameras has provided an essential alternate to explore regions where other optical sensors lack in capturing interpretable signals. This work proposes a domain adaptation framework which employs a style transfer technique for transfer learning from visible spectrum images to thermal images.
  arXiv  Detail & Related papers  (2020-06-01T09:59:09Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.