Neural Passage Retrieval with Improved Negative Contrast

• URL: http://arxiv.org/abs/2010.12523v1
• Date: Fri, 23 Oct 2020 16:45:06 GMT
• Title: Neural Passage Retrieval with Improved Negative Contrast
• Abstract summary: We explore four negative sampling strategies that complement the straightforward random sampling of negatives, typically used to train dual encoder models. Our retrieval-based strategies are based on the semantic similarity and the lexical overlap between questions and passages. Our results establish a new state-of-the-art level of performance on two of the open-domain question answering datasets.
• Score: 21.415627458949356
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In this paper we explore the effects of negative sampling in dual encoder models used to retrieve passages for automatic question answering. We explore four negative sampling strategies that complement the straightforward random sampling of negatives, typically used to train dual encoder models. Out of the four strategies, three are based on retrieval and one on heuristics. Our retrieval-based strategies are based on the semantic similarity and the lexical overlap between questions and passages. We train the dual encoder models in two stages: pre-training with synthetic data and fine tuning with domain-specific data. We apply negative sampling to both stages. The approach is evaluated in two passage retrieval tasks. Even though it is not evident that there is one single sampling strategy that works best in all the tasks, it is clear that our strategies contribute to improving the contrast between the response and all the other passages. Furthermore, mixing the negatives from different strategies achieve performance on par with the best performing strategy in all tasks. Our results establish a new state-of-the-art level of performance on two of the open-domain question answering datasets that we evaluated.

Related papers

• TopoCurate:Modeling Interaction Topology for Tool-Use Agent Training [53.93696896939915]
  Training tool-use agents typically rely on Supervised Fine-Tuning (SFT) on successful trajectories and Reinforcement Learning (RL) on pass-rate-selected tasks.<n>We propose TopoCurate, an interaction-aware framework that projects multi-trial rollouts from the same task into a unified semantic quotient topology.<n>TopoCurate achieves consistent gains of 4.2% (SFT) and 6.9% (RL) over state-of-the-art baselines.
  arXiv  Detail & Related papers  (2026-03-02T10:38:54Z)
• Learning Binarized Representations with Pseudo-positive Sample Enhancement for Efficient Graph Collaborative Filtering [35.82405808653398]
  We study the problem of graph representation binarization for efficient collaborative filtering.<n>Our findings indicate that explicitly mitigating information loss at various stages of embedding binarization has a significant positive impact on performance.<n>Compared to its predecessor BiGeaR, BiGeaR++ introduces a fine-grained inference distillation mechanism and an effective embedding sample synthesis approach.
  arXiv  Detail & Related papers  (2025-06-03T11:11:43Z)
• Sampling Strategies for Efficient Training of Deep Learning Object Detection Algorithms [4.609829289649562]
  Two sampling strategies are investigated to enhance efficiency in training a deep learning object detection model.<n>The first strategy is uniform sampling which seeks to obtain samples evenly yet randomly through the state space of the object dynamics.<n>The second strategy of frame difference sampling is developed to explore the temporal redundancy among successive frames in a video.
  arXiv  Detail & Related papers  (2025-05-23T18:54:01Z)
• On Synthetic Data Strategies for Domain-Specific Generative Retrieval [23.906425329806456]
  We study the data strategies for a two-stage training framework. In the first stage, we learn to decode document identifiers from queries. In the second stage, we refine document ranking through preference learning.
  arXiv  Detail & Related papers  (2025-02-25T08:27:54Z)
• Exploring the Performance of Continuous-Time Dynamic Link Prediction Algorithms [14.82820088479196]
  Dynamic Link Prediction (DLP) addresses the prediction of future links in evolving networks. In this work, we contribute tools to perform such a comprehensive evaluation. We describe an exhaustive taxonomy of negative sampling methods that can be used at evaluation time.
  arXiv  Detail & Related papers  (2024-05-27T14:03:28Z)
• Forging the Forger: An Attempt to Improve Authorship Verification via Data Augmentation [52.72682366640554]
  Authorship Verification (AV) is a text classification task concerned with inferring whether a candidate text has been written by one specific author or by someone else. It has been shown that many AV systems are vulnerable to adversarial attacks, where a malicious author actively tries to fool the classifier by either concealing their writing style, or by imitating the style of another author.
  arXiv  Detail & Related papers  (2024-03-17T16:36:26Z)
• Relation-Aware Network with Attention-Based Loss for Few-Shot Knowledge Graph Completion [9.181270251524866]
  Current approaches randomly select one negative sample for each reference entity pair to minimize a margin-based ranking loss. We propose a novel Relation-Aware Network with Attention-Based Loss framework. Experiments demonstrate that RANA outperforms the state-of-the-art models on two benchmark datasets.
  arXiv  Detail & Related papers  (2023-06-15T21:41:43Z)
• Incorporating Relevance Feedback for Information-Seeking Retrieval using Few-Shot Document Re-Ranking [56.80065604034095]
  We introduce a kNN approach that re-ranks documents based on their similarity with the query and the documents the user considers relevant. To evaluate our different integration strategies, we transform four existing information retrieval datasets into the relevance feedback scenario.
  arXiv  Detail & Related papers  (2022-10-19T16:19:37Z)
• Disjoint Contrastive Regression Learning for Multi-Sourced Annotations [10.159313152511919]
  Large-scale datasets are important for the development of deep learning models. Multiple annotators may be employed to label different subsets of the data. The inconsistency and bias among different annotators are harmful to the model training.
  arXiv  Detail & Related papers  (2021-12-31T12:39:04Z)
• Jo-SRC: A Contrastive Approach for Combating Noisy Labels [58.867237220886885]
  We propose a noise-robust approach named Jo-SRC (Joint Sample Selection and Model Regularization based on Consistency) Specifically, we train the network in a contrastive learning manner. Predictions from two different views of each sample are used to estimate its "likelihood" of being clean or out-of-distribution.
  arXiv  Detail & Related papers  (2021-03-24T07:26:07Z)
• Doubly Contrastive Deep Clustering [135.7001508427597]
  We present a novel Doubly Contrastive Deep Clustering (DCDC) framework, which constructs contrastive loss over both sample and class views. Specifically, for the sample view, we set the class distribution of the original sample and its augmented version as positive sample pairs. For the class view, we build the positive and negative pairs from the sample distribution of the class. In this way, two contrastive losses successfully constrain the clustering results of mini-batch samples in both sample and class level.
  arXiv  Detail & Related papers  (2021-03-09T15:15:32Z)
• Semi-supervised Collaborative Filtering by Text-enhanced Domain Adaptation [32.93934837792708]
  We consider the problem of recommendation on sparse implicit feedbacks as a semi-supervised learning task. We focus on the most challenging case -- there is no user or item overlap. We adopt domain-invariant textual features as the anchor points to align the latent spaces.
  arXiv  Detail & Related papers  (2020-06-28T05:28:05Z)
• One-Shot Object Detection without Fine-Tuning [62.39210447209698]
  We introduce a two-stage model consisting of a first stage Matching-FCOS network and a second stage Structure-Aware Relation Module. We also propose novel training strategies that effectively improve detection performance. Our method exceeds the state-of-the-art one-shot performance consistently on multiple datasets.
  arXiv  Detail & Related papers  (2020-05-08T01:59:23Z)
• The Simulator: Understanding Adaptive Sampling in the Moderate-Confidence Regime [52.38455827779212]
  We propose a novel technique for analyzing adaptive sampling called the em Simulator. We prove the first instance-based lower bounds the top-k problem which incorporate the appropriate log-factors. Our new analysis inspires a simple and near-optimal for the best-arm and top-k identification, the first em practical of its kind for the latter problem.
  arXiv  Detail & Related papers  (2017-02-16T23:42:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.