Contrastive Fine-tuning Improves Robustness for Neural Rankers
- URL: http://arxiv.org/abs/2105.12932v1
- Date: Thu, 27 May 2021 04:00:22 GMT
- Title: Contrastive Fine-tuning Improves Robustness for Neural Rankers
- Authors: Xiaofei Ma, Cicero Nogueira dos Santos and Andrew O. Arnold
- Abstract summary: We present a novel method for fine-tuning neural rankers that can significantly improve their robustness to out-of-domain data and query perturbations.
We use relevance labels to denote similar/dissimilar pairs, which allows the model to learn the underlying matching semantics across different query-document pairs.
In experiments with four passage ranking datasets, the proposed contrastive fine-tuning method obtains improvements on robustness to query reformulations, noise perturbations, and zero-shot transfer for both BERT and BART based rankers.
- Score: 1.3868793694964396
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: The performance of state-of-the-art neural rankers can deteriorate
substantially when exposed to noisy inputs or applied to a new domain. In this
paper, we present a novel method for fine-tuning neural rankers that can
significantly improve their robustness to out-of-domain data and query
perturbations. Specifically, a contrastive loss that compares data points in
the representation space is combined with the standard ranking loss during
fine-tuning. We use relevance labels to denote similar/dissimilar pairs, which
allows the model to learn the underlying matching semantics across different
query-document pairs and leads to improved robustness. In experiments with four
passage ranking datasets, the proposed contrastive fine-tuning method obtains
improvements on robustness to query reformulations, noise perturbations, and
zero-shot transfer for both BERT and BART based rankers. Additionally, our
experiments show that contrastive fine-tuning outperforms data augmentation for
robustifying neural rankers.
Related papers
- Phase-aggregated Dual-branch Network for Efficient Fingerprint Dense Registration [34.16169623776737]
Fingerprint dense registration aims to finely align fingerprint pairs at the pixel level, thereby reducing intra-class differences caused by distortion.
Traditional methods exhibited subpar performance when dealing with low-quality fingerprints.
Deep learning based approaches shows significant improvement in these aspects, but their registration accuracy is still unsatisfactory.
arXiv Detail & Related papers (2024-04-26T05:06:53Z) - DiffImpute: Tabular Data Imputation With Denoising Diffusion Probabilistic Model [9.908561639396273]
We propose DiffImpute, a novel Denoising Diffusion Probabilistic Model (DDPM)
It produces credible imputations for missing entries without undermining the authenticity of the existing data.
It can be applied to various settings of Missing Completely At Random (MCAR) and Missing At Random (MAR)
arXiv Detail & Related papers (2024-03-20T08:45:31Z) - Noisy Correspondence Learning with Self-Reinforcing Errors Mitigation [63.180725016463974]
Cross-modal retrieval relies on well-matched large-scale datasets that are laborious in practice.
We introduce a novel noisy correspondence learning framework, namely textbfSelf-textbfReinforcing textbfErrors textbfMitigation (SREM)
arXiv Detail & Related papers (2023-12-27T09:03:43Z) - Noisy Self-Training with Synthetic Queries for Dense Retrieval [49.49928764695172]
We introduce a novel noisy self-training framework combined with synthetic queries.
Experimental results show that our method improves consistently over existing methods.
Our method is data efficient and outperforms competitive baselines.
arXiv Detail & Related papers (2023-11-27T06:19:50Z) - Binary Classification with Confidence Difference [100.08818204756093]
This paper delves into a novel weakly supervised binary classification problem called confidence-difference (ConfDiff) classification.
We propose a risk-consistent approach to tackle this problem and show that the estimation error bound the optimal convergence rate.
We also introduce a risk correction approach to mitigate overfitting problems, whose consistency and convergence rate are also proven.
arXiv Detail & Related papers (2023-10-09T11:44:50Z) - RoPDA: Robust Prompt-based Data Augmentation for Low-Resource Named
Entity Recognition [10.03246698225533]
Robust Prompt-based Data Augmentation (RoPDA) for low-resource NER
Based on pre-trained language models (PLMs) with continuous prompt, RoPDA performs entity augmentation and context augmentation.
Experiments on three benchmarks from different domains demonstrate that RoPDA significantly improves upon strong baselines.
arXiv Detail & Related papers (2023-07-11T14:44:14Z) - Modelling Adversarial Noise for Adversarial Defense [96.56200586800219]
adversarial defenses typically focus on exploiting adversarial examples to remove adversarial noise or train an adversarially robust target model.
Motivated by that the relationship between adversarial data and natural data can help infer clean data from adversarial data to obtain the final correct prediction.
We study to model adversarial noise to learn the transition relationship in the label space for using adversarial labels to improve adversarial accuracy.
arXiv Detail & Related papers (2021-09-21T01:13:26Z) - Exploiting Sample Uncertainty for Domain Adaptive Person
Re-Identification [137.9939571408506]
We estimate and exploit the credibility of the assigned pseudo-label of each sample to alleviate the influence of noisy labels.
Our uncertainty-guided optimization brings significant improvement and achieves the state-of-the-art performance on benchmark datasets.
arXiv Detail & Related papers (2020-12-16T04:09:04Z) - Pairwise Supervised Hashing with Bernoulli Variational Auto-Encoder and
Self-Control Gradient Estimator [62.26981903551382]
Variational auto-encoders (VAEs) with binary latent variables provide state-of-the-art performance in terms of precision for document retrieval.
We propose a pairwise loss function with discrete latent VAE to reward within-class similarity and between-class dissimilarity for supervised hashing.
This new semantic hashing framework achieves superior performance compared to the state-of-the-arts.
arXiv Detail & Related papers (2020-05-21T06:11:33Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.