Improving Contrastive Learning of Sentence Embeddings from AI Feedback
- URL: http://arxiv.org/abs/2305.01918v3
- Date: Sat, 20 May 2023 09:39:29 GMT
- Title: Improving Contrastive Learning of Sentence Embeddings from AI Feedback
- Authors: Qinyuan Cheng, Xiaogui Yang, Tianxiang Sun, Linyang Li, Xipeng Qiu
- Abstract summary: Supervised contrastive learning can produce more accurate sample pairs with human feedback labels.
Our method utilizes AI feedback from large pre-trained language models to construct sample pairs with fine-grained sample similarity scores.
Experimental results show that our method achieves state-of-the-art performance on several semantic textual similarity tasks.
- Score: 43.56070504980024
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Contrastive learning has become a popular approach in natural language
processing, particularly for the learning of sentence embeddings. However, the
discrete nature of natural language makes it difficult to ensure the quality of
positive and negative sample pairs generated through data augmentation methods.
Although supervised contrastive learning can produce more accurate sample pairs
with human feedback labels, it still lacks fine-grained training signals. In
this paper, we propose to improve \textbf{C}ontrastive \textbf{L}earning of
sentence embeddings from \textbf{AI} \textbf{F}eedback \textbf{(CLAIF)}. Our
method utilizes AI feedback from large pre-trained language models (LLMs) to
construct sample pairs with fine-grained sample similarity scores to improve
contrastive learning. Besides, we combine human feedback and AI feedback to
provide better supervision signals for supervised contrastive learning of
sentence embeddings. Experimental results show that our method achieves
state-of-the-art performance on several semantic textual similarity (STS) and
transfer learning tasks compared to other unsupervised and supervised
contrastive learning methods.
Related papers
- DenoSent: A Denoising Objective for Self-Supervised Sentence
Representation Learning [59.4644086610381]
We propose a novel denoising objective that inherits from another perspective, i.e., the intra-sentence perspective.
By introducing both discrete and continuous noise, we generate noisy sentences and then train our model to restore them to their original form.
Our empirical evaluations demonstrate that this approach delivers competitive results on both semantic textual similarity (STS) and a wide range of transfer tasks.
arXiv Detail & Related papers (2024-01-24T17:48:45Z) - Differentiable Data Augmentation for Contrastive Sentence Representation
Learning [6.398022050054328]
The proposed method yields significant improvements over existing methods under both semi-supervised and supervised settings.
Our experiments under a low labeled data setting also show that our method is more label-efficient than the state-of-the-art contrastive learning methods.
arXiv Detail & Related papers (2022-10-29T08:57:45Z) - Improving Contrastive Learning of Sentence Embeddings with
Case-Augmented Positives and Retrieved Negatives [17.90820242798732]
Unsupervised contrastive learning methods still lag far behind the supervised counterparts.
We propose switch-case augmentation to flip the case of the first letter of randomly selected words in a sentence.
For negative samples, we sample hard negatives from the whole dataset based on a pre-trained language model.
arXiv Detail & Related papers (2022-06-06T09:46:12Z) - Generative or Contrastive? Phrase Reconstruction for Better Sentence
Representation Learning [86.01683892956144]
We propose a novel generative self-supervised learning objective based on phrase reconstruction.
Our generative learning may yield powerful enough sentence representation and achieve performance in Sentence Textual Similarity tasks on par with contrastive learning.
arXiv Detail & Related papers (2022-04-20T10:00:46Z) - PCL: Peer-Contrastive Learning with Diverse Augmentations for
Unsupervised Sentence Embeddings [69.87899694963251]
We propose a novel Peer-Contrastive Learning (PCL) with diverse augmentations.
PCL constructs diverse contrastive positives and negatives at the group level for unsupervised sentence embeddings.
PCL can perform peer-positive contrast as well as peer-network cooperation, which offers an inherent anti-bias ability.
arXiv Detail & Related papers (2022-01-28T13:02:41Z) - Simple Contrastive Representation Adversarial Learning for NLP Tasks [17.12062566060011]
Two novel frameworks, supervised contrastive adversarial learning (SCAL) and unsupervised SCAL (USCAL), are proposed.
We employ it to Transformer-based models for natural language understanding, sentence semantic textual similarity and adversarial learning tasks.
Experimental results on GLUE benchmark tasks show that our fine-tuned supervised method outperforms BERT$_base$ over 1.75%.
arXiv Detail & Related papers (2021-11-26T03:16:09Z) - Incremental False Negative Detection for Contrastive Learning [95.68120675114878]
We introduce a novel incremental false negative detection for self-supervised contrastive learning.
During contrastive learning, we discuss two strategies to explicitly remove the detected false negatives.
Our proposed method outperforms other self-supervised contrastive learning frameworks on multiple benchmarks within a limited compute.
arXiv Detail & Related papers (2021-06-07T15:29:14Z) - Constructing Contrastive samples via Summarization for Text
Classification with limited annotations [46.53641181501143]
We propose a novel approach to constructing contrastive samples for language tasks using text summarization.
We use these samples for supervised contrastive learning to gain better text representations with limited annotations.
Experiments on real-world text classification datasets (Amazon-5, Yelp-5, AG News) demonstrate the effectiveness of the proposed contrastive learning framework.
arXiv Detail & Related papers (2021-04-11T20:13:24Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.