Zero-shot Neural Passage Retrieval via Domain-targeted Synthetic
Question Generation
- URL: http://arxiv.org/abs/2004.14503v3
- Date: Wed, 27 Jan 2021 16:04:12 GMT
- Title: Zero-shot Neural Passage Retrieval via Domain-targeted Synthetic
Question Generation
- Authors: Ji Ma, Ivan Korotkov, Yinfei Yang, Keith Hall and Ryan McDonald
- Abstract summary: A major obstacle to the wide-spread adoption of neural retrieval models is that they require large supervised training sets to surpass traditional term-based techniques.
In this paper, we propose an approach to zero-shot learning for passage retrieval that uses synthetic question generation to close this gap.
- Score: 10.324402925019946
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: A major obstacle to the wide-spread adoption of neural retrieval models is
that they require large supervised training sets to surpass traditional
term-based techniques, which are constructed from raw corpora. In this paper,
we propose an approach to zero-shot learning for passage retrieval that uses
synthetic question generation to close this gap. The question generation system
is trained on general domain data, but is applied to documents in the targeted
domain. This allows us to create arbitrarily large, yet noisy, question-passage
relevance pairs that are domain specific. Furthermore, when this is coupled
with a simple hybrid term-neural model, first-stage retrieval performance can
be improved further. Empirically, we show that this is an effective strategy
for building neural passage retrieval models in the absence of large training
corpora. Depending on the domain, this technique can even approach the accuracy
of supervised models.
Related papers
- Transferable Post-training via Inverse Value Learning [83.75002867411263]
We propose modeling changes at the logits level during post-training using a separate neural network (i.e., the value network)
After training this network on a small base model using demonstrations, this network can be seamlessly integrated with other pre-trained models during inference.
We demonstrate that the resulting value network has broad transferability across pre-trained models of different parameter sizes.
arXiv Detail & Related papers (2024-10-28T13:48:43Z) - State-Space Modeling in Long Sequence Processing: A Survey on Recurrence in the Transformer Era [59.279784235147254]
This survey provides an in-depth summary of the latest approaches that are based on recurrent models for sequential data processing.
The emerging picture suggests that there is room for thinking of novel routes, constituted by learning algorithms which depart from the standard Backpropagation Through Time.
arXiv Detail & Related papers (2024-06-13T12:51:22Z) - On the Resurgence of Recurrent Models for Long Sequences -- Survey and
Research Opportunities in the Transformer Era [59.279784235147254]
This survey is aimed at providing an overview of these trends framed under the unifying umbrella of Recurrence.
It emphasizes novel research opportunities that become prominent when abandoning the idea of processing long sequences.
arXiv Detail & Related papers (2024-02-12T23:55:55Z) - GPT-ST: Generative Pre-Training of Spatio-Temporal Graph Neural Networks [24.323017830938394]
This work aims to address challenges by introducing a pre-training framework that seamlessly integrates with baselines and enhances their performance.
The framework is built upon two key designs: (i) We propose a.
apple-to-apple mask autoencoder as a pre-training model for learning-temporal dependencies.
These modules are specifically designed to capture intra-temporal customized representations and semantic- and inter-cluster relationships.
arXiv Detail & Related papers (2023-11-07T02:36:24Z) - Unsupervised Non-transferable Text Classification [8.077841946617472]
We propose a novel unsupervised non-transferable learning method for the text classification task.
We introduce a secret key component in our approach for recovering the access to the target domain.
arXiv Detail & Related papers (2022-10-23T08:15:43Z) - Goal-driven Self-Attentive Recurrent Networks for Trajectory Prediction [31.02081143697431]
Human trajectory forecasting is a key component of autonomous vehicles, social-aware robots and video-surveillance applications.
We propose a lightweight attention-based recurrent backbone that acts solely on past observed positions.
We employ a common goal module, based on a U-Net architecture, which additionally extracts semantic information to predict scene-compliant destinations.
arXiv Detail & Related papers (2022-04-25T11:12:37Z) - Ultrasound Signal Processing: From Models to Deep Learning [64.56774869055826]
Medical ultrasound imaging relies heavily on high-quality signal processing to provide reliable and interpretable image reconstructions.
Deep learning based methods, which are optimized in a data-driven fashion, have gained popularity.
A relatively new paradigm combines the power of the two: leveraging data-driven deep learning, as well as exploiting domain knowledge.
arXiv Detail & Related papers (2022-04-09T13:04:36Z) - SAGE: Generating Symbolic Goals for Myopic Models in Deep Reinforcement
Learning [18.37286885057802]
We propose an algorithm combining learning and planning to exploit a previously unusable class of incomplete models.
This combines the strengths of symbolic planning and neural learning approaches in a novel way that outperforms competing methods on variations of taxi world and Minecraft.
arXiv Detail & Related papers (2022-03-09T22:55:53Z) - TraND: Transferable Neighborhood Discovery for Unsupervised Cross-domain
Gait Recognition [77.77786072373942]
This paper proposes a Transferable Neighborhood Discovery (TraND) framework to bridge the domain gap for unsupervised cross-domain gait recognition.
We design an end-to-end trainable approach to automatically discover the confident neighborhoods of unlabeled samples in the latent space.
Our method achieves state-of-the-art results on two public datasets, i.e., CASIA-B and OU-LP.
arXiv Detail & Related papers (2021-02-09T03:07:07Z) - Latent Space Conditioning on Generative Adversarial Networks [3.823356975862006]
We introduce a novel framework that benefits from two popular learning techniques, adversarial training and representation learning.
In particular, our approach exploits the structure of a latent space (learned by the representation learning) and employs it to condition the generative model.
arXiv Detail & Related papers (2020-12-16T08:58:10Z) - Unsupervised Intra-domain Adaptation for Semantic Segmentation through
Self-Supervision [73.76277367528657]
Convolutional neural network-based approaches have achieved remarkable progress in semantic segmentation.
To cope with this limitation, automatically annotated data generated from graphic engines are used to train segmentation models.
We propose a two-step self-supervised domain adaptation approach to minimize the inter-domain and intra-domain gap together.
arXiv Detail & Related papers (2020-04-16T15:24:11Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.