Contrastive Learning of Preferences with a Contextual InfoNCE Loss

• URL: http://arxiv.org/abs/2407.05898v1
• Date: Mon, 8 Jul 2024 13:05:08 GMT
• Title: Contrastive Learning of Preferences with a Contextual InfoNCE Loss
• Abstract summary: We show how one can solve this problem via a suitable adaptation of the CLIP framework. We empirically demonstrate the utility of our adapted version of the InfoNCE loss in the domain of collectable card games.
• Score: 4.336779198334903
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: A common problem in contextual preference ranking is that a single preferred action is compared against several choices, thereby blowing up the complexity and skewing the preference distribution. In this work, we show how one can solve this problem via a suitable adaptation of the CLIP framework.This adaptation is not entirely straight-forward, because although the InfoNCE loss used by CLIP has achieved great success in computer vision and multi-modal domains, its batch-construction technique requires the ability to compare arbitrary items, and is not well-defined if one item has multiple positive associations in the same batch. We empirically demonstrate the utility of our adapted version of the InfoNCE loss in the domain of collectable card games, where we aim to learn an embedding space that captures the associations between single cards and whole card pools based on human selections. Such selection data only exists for restricted choices, thus generating concrete preferences of one item over a set of other items rather than a perfect fit between the card and the pool. Our results show that vanilla CLIP does not perform well due to the aforementioned intuitive issues. However, by adapting CLIP to the problem, we receive a model outperforming previous work trained with the triplet loss, while also alleviating problems associated with mining triplets.

Related papers

• ConeSep: Cone-based Robust Noise-Unlearning Compositional Network for Composed Image Retrieval [60.051600134831226]
  This paper systematically investigates the Noisy Triplet Correspondence (NTC) problem introduced by annotations.<n>We propose a Cone-based robuSt noisE-unlearning comPositional network (ConeSep) to address these challenges.
  arXiv  Detail & Related papers  (2026-04-22T08:59:21Z)
• LLM Routing with Dueling Feedback [49.67815163970033]
  We study the problem of selecting the best model for each query while balancing user satisfaction, model expertise, and inference cost.<n>We formulate routing as contextual dueling bandits, learning from pairwise preference feedback rather than absolute scores.<n>We introduce Category-Calibrated Fine-Tuning (CCFT), a representation-learning method that derives model embeddings from offline data using contrastive fine-tuning with categorical weighting.
  arXiv  Detail & Related papers  (2025-10-01T12:52:25Z)
• SeMoBridge: Semantic Modality Bridge for Efficient Few-Shot Adaptation of CLIP [6.427791334895624]
  We introduce SeMoBridge, a lightweight yet powerful approach that directly addresses the misalignment.<n>Our method maps images into the text modality, while keeping their semantic content intact through what we call a Semantic Modality Bridge.<n> Experiments show that the trained version, SeMoBridge-T, requires only a fraction of the training time while overall outperforming other methods.
  arXiv  Detail & Related papers  (2025-09-30T10:12:15Z)
• CLIP Won't Learn Object-Attribute Binding from Natural Data and Here is Why [28.410388568106587]
  We show that low attribute density, incomplete captions, and the saliency bias have a detrimental effect on binding performance.<n>We find that neither scaling the batch size, i.e., implicitly adding more hard negatives, nor explicitly creating hard negatives enables CLIP to learn reliable binding.
  arXiv  Detail & Related papers  (2025-07-10T17:57:31Z)
• UPCORE: Utility-Preserving Coreset Selection for Balanced Unlearning [57.081646768835704]
  User specifications or legal frameworks often require information to be removed from pretrained models, including large language models (LLMs) This requires deleting or "forgetting" a set of data points from an already-trained model, which typically degrades its performance on other data points. We propose UPCORE, a method-agnostic data selection framework for mitigating collateral damage during unlearning.
  arXiv  Detail & Related papers  (2025-02-20T22:51:10Z)
• A Systematic Examination of Preference Learning through the Lens of Instruction-Following [83.71180850955679]
  We use a novel synthetic data generation pipeline to generate 48,000 instruction unique-following prompts. With our synthetic prompts, we use two preference dataset curation methods - rejection sampling (RS) and Monte Carlo Tree Search (MCTS) Experiments reveal that shared prefixes in preference pairs, as generated by MCTS, provide marginal but consistent improvements. High-contrast preference pairs generally outperform low-contrast pairs; however, combining both often yields the best performance.
  arXiv  Detail & Related papers  (2024-12-18T15:38:39Z)
• Probably Approximately Precision and Recall Learning [62.912015491907994]
  Precision and Recall are foundational metrics in machine learning. One-sided feedback--where only positive examples are observed during training--is inherent in many practical problems. We introduce a PAC learning framework where each hypothesis is represented by a graph, with edges indicating positive interactions.
  arXiv  Detail & Related papers  (2024-11-20T04:21:07Z)
• TripletCLIP: Improving Compositional Reasoning of CLIP via Synthetic Vision-Language Negatives [65.82577305915643]
  Contrastive Language-Image Pretraining (CLIP) models maximize the mutual information between text and visual modalities to learn representations. We show that generating hard'' negative captions via in-context learning and corresponding negative images with text-to-image generators offers a solution. We demonstrate that our method, named TripletCLIP, enhances the compositional capabilities of CLIP, resulting in an absolute improvement of over 9% on the SugarCrepe benchmark.
  arXiv  Detail & Related papers  (2024-11-04T19:24:59Z)
• CLIP Adaptation by Intra-modal Overlap Reduction [1.2277343096128712]
  We analyse the intra-modal overlap in image space in terms of embedding representation. We train a lightweight adapter on a generic set of samples from the Google Open Images dataset.
  arXiv  Detail & Related papers  (2024-09-17T16:40:58Z)
• CLIPCleaner: Cleaning Noisy Labels with CLIP [36.434849361479316]
  textitCLIPCleaner is a zero-shot classifier for efficient, offline, clean sample selection. textitCLIPCleaner offers a simple, single-step approach that achieves competitive or superior performance on benchmark datasets.
  arXiv  Detail & Related papers  (2024-08-19T14:05:58Z)
• CLIPLoss and Norm-Based Data Selection Methods for Multimodal Contrastive Learning [19.100022935748225]
  Data selection has emerged as a core issue for large-scale visual-language model pretaining (e.g., CLIP) Three main data selection approaches are: (1) leveraging external non-CLIP models to aid data selection, (2) training new CLIP-style embedding models that are more effective at selecting high-quality data, and (3) designing better metrics or strategies universally applicable to any CLIP embedding.
  arXiv  Detail & Related papers  (2024-05-29T22:19:57Z)
• SoftCLIP: Softer Cross-modal Alignment Makes CLIP Stronger [30.758184720183106]
  We propose SoftCLIP, a novel approach that relaxes the strict one-to-one constraint and achieves a soft cross-modal alignment. In particular, on ImageNet zero-shot classification task, using CC3M/CC12M as pre-training dataset, SoftCLIP brings a top-1 accuracy improvement of 6.8%/7.2%.
  arXiv  Detail & Related papers  (2023-03-30T17:27:22Z)
• CLIPood: Generalizing CLIP to Out-of-Distributions [73.86353105017076]
  Contrastive language-image pre-training (CLIP) models have shown impressive zero-shot ability, but the further adaptation of CLIP on downstream tasks undesirably degrades OOD performances. We propose CLIPood, a fine-tuning method that can adapt CLIP models to OOD situations where both domain shifts and open classes may occur on unseen test data. Experiments on diverse datasets with different OOD scenarios show that CLIPood consistently outperforms existing generalization techniques.
  arXiv  Detail & Related papers  (2023-02-02T04:27:54Z)
• Model Selection in Batch Policy Optimization [88.52887493684078]
  We study the problem of model selection in batch policy optimization. We identify three sources of error that any model selection algorithm should optimally trade-off in order to be competitive.
  arXiv  Detail & Related papers  (2021-12-23T02:31:50Z)
• A Comparison of Contextual and Non-Contextual Preference Ranking for Set Addition Problems [0.0]
  We study the problem of evaluating the addition of elements to a set. We model preferences based on the context of the decision. We show that the triplet approach achieves a better result than the twin network and that both outperform previous results on this task.
  arXiv  Detail & Related papers  (2021-07-09T13:33:16Z)
• Improving filling level classification with adversarial training [90.01594595780928]
  We investigate the problem of classifying - from a single image - the level of content in a cup or a drinking glass. We use adversarial training in a generic source dataset and then refine the training with a task-specific dataset. We show that transfer learning with adversarial training in the source domain consistently improves the classification accuracy on the test set.
  arXiv  Detail & Related papers  (2021-02-08T08:32:56Z)
• Contrastive Learning with Adversarial Examples [79.39156814887133]
  Contrastive learning (CL) is a popular technique for self-supervised learning (SSL) of visual representations. This paper introduces a new family of adversarial examples for constrastive learning and using these examples to define a new adversarial training algorithm for SSL, denoted as CLAE.
  arXiv  Detail & Related papers  (2020-10-22T20:45:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.