TeMo: Temperature Modulation for Multimodal Contrastive Learning

• URL: http://arxiv.org/abs/2609.07540v1
• Date: Mon, 07 Sep 2026 14:23:02 GMT
• Title: TeMo: Temperature Modulation for Multimodal Contrastive Learning
• Abstract summary: Contrastive learning approaches achieve strong performance by training models to bring similar samples closer while pushing dissimilar samples apart.<n>We introduce TeMo, Temperature Modulation framework, a similarity-based modulation approach that adaptively adjusts the temperature for each positive-negative pair according to their similarity.<n>Our approach seamlessly integrates temperature-modulated multimodal and unimodal losses with the standard multimodal contrastive loss by gradually transitioning between them.
• Score: 69.50376336775496
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Contrastive learning approaches achieve strong performance by training models to bring similar samples closer while pushing dissimilar samples apart. A crucial component of contrastive learning is the temperature hyperparameter $τ$, which controls the penalty strength applied to negative samples. However, most existing methods either fix this hyperparameter or learn a global value during training. In this paper, we introduce TeMo, Temperature Modulation framework, a similarity-based modulation approach that adaptively adjusts the temperature for each positive-negative pair according to their similarity, enabling more fine-grained multimodal contrastive learning. Our approach seamlessly integrates temperature-modulated multimodal and unimodal losses with the standard multimodal contrastive loss by gradually transitioning between them. This design allows the model to capture both coarse- and fine-grained semantics at different training stages. Extensive experiments demonstrate that each component of TeMo consistently enhances performance across diverse zero-shot retrieval and classification tasks, establishing new state-of-the-art results.

Related papers

• Diversify Diffusion with Temperature Sampling and Variance-Corrective Time Shifting [57.75750707482832]
  We introduce variance-corrective time shifting, a training-free fix that queries the network at a shifted timestep and scales the resulting score by $$, canceling the variance inflation while preserving the mode reweighting.<n>We demonstrate consistent gains at minimal cost to sample quality and condition fidelity across DiT, Stable Diffusion and Motion Diffusion models.
  arXiv  Detail & Related papers  (2026-07-12T17:32:29Z)
• MM-TS: Multi-Modal Temperature and Margin Schedules for Contrastive Learning with Long-Tail Data [64.78447637450937]
  We propose Multi-Modal Temperature and Margin Schedules (MM-TS), extending the concept of uni-modal temperature scheduling to multi-modal contrastive learning.<n>Our method dynamically adjusts the temperature in the contrastive loss during training, modulating the attraction and repulsion forces in the multi-modal setting.
  arXiv  Detail & Related papers  (2026-03-09T10:29:50Z)
• MoETTA: Test-Time Adaptation Under Mixed Distribution Shifts with MoE-LayerNorm [13.63833424954647]
  Test-Time adaptation (TTA) has proven effective in mitigating performance drops under single-domain distribution shifts.<n>We propose MoETTA, a novel entropy-based TTA framework that integrates the Mixture-of-Experts architecture.
  arXiv  Detail & Related papers  (2025-11-14T10:24:06Z)
• LLM-Oriented Token-Adaptive Knowledge Distillation [64.08412563818662]
  We propose a novel framework that adapts the distillation process to the real-time learning state of each token.<n>AdaKD consists of two synergistic modules driven by a unified token difficulty metric.<n>As a plug-and-play framework, AdaKD can consistently improve the performance of various distillation methods on multiple model architectures and benchmarks.
  arXiv  Detail & Related papers  (2025-10-13T16:55:07Z)
• Decoupled Contrastive Learning for Federated Learning [0.0]
  We introduce Decoupled Contrastive Learning for Federated Learning (DCFL)<n>DCFL is a novel framework that decouples the existing contrastive loss into two objectives.<n>Our results show that DCFL achieves stronger alignment between positive samples and greater uniformity between negative samples.
  arXiv  Detail & Related papers  (2025-08-06T01:39:54Z)
• A Robust Incomplete Multimodal Low-Rank Adaptation Approach for Emotion Recognition [17.332141776831513]
  Multimodal Emotion Recognition (MER) often encounters incomplete multimodality in practical applications.<n>We propose a unimodal decoupled dynamic low-rank adaptation method based on modality combinations, named MCULoRA.
  arXiv  Detail & Related papers  (2025-07-15T11:15:35Z)
• Classifier-guided Gradient Modulation for Enhanced Multimodal Learning [50.7008456698935]
  Gradient-Guided Modulation (CGGM) is a novel method to balance multimodal learning with gradients. We conduct extensive experiments on four multimodal datasets: UPMC-Food 101, CMU-MOSI, IEMOCAP and BraTS. CGGM outperforms all the baselines and other state-of-the-art methods consistently.
  arXiv  Detail & Related papers  (2024-11-03T02:38:43Z)
• Efficient and Effective Weight-Ensembling Mixture of Experts for Multi-Task Model Merging [111.8456671452411]
  Multi-task learning (MTL) leverages a shared model to accomplish multiple tasks and facilitate knowledge transfer. We propose a Weight-Ensembling Mixture of Experts (WEMoE) method for multi-task model merging. We show that WEMoE and E-WEMoE outperform state-of-the-art (SOTA) model merging methods in terms of MTL performance, generalization, and robustness.
  arXiv  Detail & Related papers  (2024-10-29T07:16:31Z)
• Dynamically Scaled Temperature in Self-Supervised Contrastive Learning [11.133502139934437]
  We focus on improving the performance of InfoNCE loss in self-supervised learning by proposing a novel cosine similarity dependent temperature scaling function. Experimental evidence shows that the proposed framework outperforms the contrastive loss-based SSL algorithms.
  arXiv  Detail & Related papers  (2023-08-02T13:31:41Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.