Supervised Contrastive Learning for Product Matching
- URL: http://arxiv.org/abs/2202.02098v1
- Date: Fri, 4 Feb 2022 12:16:38 GMT
- Title: Supervised Contrastive Learning for Product Matching
- Authors: Ralph Peeters, Christian Bizer
- Abstract summary: This poster is the first work that applies contrastive learning to the task of product matching in e-commerce.
We employ a supervised contrastive learning technique to pre-train a Transformer encoder which is afterwards fine-tuned for the matching problem.
We propose a source-aware sampling strategy which enables contrastive learning to be applied for use cases in which the training data does not contain product idenifiers.
- Score: 2.28438857884398
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Contrastive learning has seen increasing success in the fields of computer
vision and information retrieval in recent years. This poster is the first work
that applies contrastive learning to the task of product matching in e-commerce
using product offers from different e-shops. More specifically, we employ a
supervised contrastive learning technique to pre-train a Transformer encoder
which is afterwards fine-tuned for the matching problem using pair-wise
training data. We further propose a source-aware sampling strategy which
enables contrastive learning to be applied for use cases in which the training
data does not contain product idenifiers. We show that applying supervised
contrastive pre-training in combination with source-aware sampling
significantly improves the state-of-the art performance on several widely used
benchmark datasets: For Abt-Buy, we reach an F1 of 94.29 (+3.24 compared to the
previous state-of-the-art), for Amazon-Google 79.28 (+ 3.7). For WDC Computers
datasets, we reach improvements between +0.8 and +8.84 F1 depending on the
training set size. Further experiments with data augmentation and
self-supervised contrastive pre-training show, that the former can be helpful
for smaller training sets while the latter leads to a significant decline in
performance due to inherent label-noise. We thus conclude that contrastive
pre-training has a high potential for product matching use cases in which
explicit supervision is available.
Related papers
- Automatic Pair Construction for Contrastive Post-training [57.57149781848383]
In this paper, we propose an automatic way to construct contrastive data for large language models (LLMs)
We compare the contrastive techniques of SLiC and DPO to SFT baselines and find that DPO provides a step-function improvement even after continuing SFT saturates.
We also explore a data curriculum learning scheme for contrastive post-training, which starts by learning from "easier" pairs and transitioning to "harder" ones.
arXiv Detail & Related papers (2023-10-03T17:59:46Z) - Training with Product Digital Twins for AutoRetail Checkout [28.823850493539293]
We propose a training data optimization framework, i.e., training with digital twins (DtTrain)
These digital twins, inherit product labels and, when augmented, form the Digital Twin training set (DT set)
In our experiment, we show that DT set outperforms training sets created by existing dataset synthesis methods in terms of counting accuracy.
arXiv Detail & Related papers (2023-08-18T17:58:10Z) - Retrieval-Enhanced Contrastive Vision-Text Models [61.783728119255365]
We propose to equip vision-text models with the ability to refine their embedding with cross-modal retrieved information from a memory at inference time.
Remarkably, we show that this can be done with a light-weight, single-layer, fusion transformer on top of a frozen CLIP.
Our experiments validate that our retrieval-enhanced contrastive (RECO) training improves CLIP performance substantially on several challenging fine-grained tasks.
arXiv Detail & Related papers (2023-06-12T15:52:02Z) - Boosting Visual-Language Models by Exploiting Hard Samples [126.35125029639168]
HELIP is a cost-effective strategy tailored to enhance the performance of existing CLIP models.
Our method allows for effortless integration with existing models' training pipelines.
On comprehensive benchmarks, HELIP consistently boosts existing models to achieve leading performance.
arXiv Detail & Related papers (2023-05-09T07:00:17Z) - Time Series Contrastive Learning with Information-Aware Augmentations [57.45139904366001]
A key component of contrastive learning is to select appropriate augmentations imposing some priors to construct feasible positive samples.
How to find the desired augmentations of time series data that are meaningful for given contrastive learning tasks and datasets remains an open question.
We propose a new contrastive learning approach with information-aware augmentations, InfoTS, that adaptively selects optimal augmentations for time series representation learning.
arXiv Detail & Related papers (2023-03-21T15:02:50Z) - ColloSSL: Collaborative Self-Supervised Learning for Human Activity
Recognition [9.652822438412903]
A major bottleneck in training robust Human-Activity Recognition models (HAR) is the need for large-scale labeled sensor datasets.
Because labeling large amounts of sensor data is an expensive task, unsupervised and semi-supervised learning techniques have emerged.
We present a novel technique called Collaborative Self-Supervised Learning (ColloSSL) which leverages unlabeled data collected from multiple devices.
arXiv Detail & Related papers (2022-02-01T21:05:05Z) - Robust Pre-Training by Adversarial Contrastive Learning [120.33706897927391]
Recent work has shown that, when integrated with adversarial training, self-supervised pre-training can lead to state-of-the-art robustness.
We improve robustness-aware self-supervised pre-training by learning representations consistent under both data augmentations and adversarial perturbations.
arXiv Detail & Related papers (2020-10-26T04:44:43Z) - Supervision Accelerates Pre-training in Contrastive Semi-Supervised
Learning of Visual Representations [12.755943669814236]
We propose a semi-supervised loss, SuNCEt, that aims to distinguish examples of different classes in addition to self-supervised instance-wise pretext tasks.
On ImageNet, we find that SuNCEt can be used to match the semi-supervised learning accuracy of previous contrastive approaches.
Our main insight is that leveraging even a small amount of labeled data during pre-training, and not only during fine-tuning, provides an important signal.
arXiv Detail & Related papers (2020-06-18T18:44:13Z) - Omni-supervised Facial Expression Recognition via Distilled Data [120.11782405714234]
We propose omni-supervised learning to exploit reliable samples in a large amount of unlabeled data for network training.
We experimentally verify that the new dataset can significantly improve the ability of the learned FER model.
To tackle this, we propose to apply a dataset distillation strategy to compress the created dataset into several informative class-wise images.
arXiv Detail & Related papers (2020-05-18T09:36:51Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.