C-MORE: Pretraining to Answer Open-Domain Questions by Consulting Millions of References

• URL: http://arxiv.org/abs/2203.08928v1
• Date: Wed, 16 Mar 2022 20:30:05 GMT
• Title: C-MORE: Pretraining to Answer Open-Domain Questions by Consulting Millions of References
• Abstract summary: Key challenge is how to construct a large amount of high-quality question-answer-context triplets without task-specific annotations. In this work, we automatically construct a large-scale corpus that meets all three criteria by consulting millions of references cited within Wikipedia. Our pretrained retriever leads to 2%-10% absolute gains in top-20 accuracy. And with our pretrained reader, the entire system improves by up to 4% in exact match.
• Score: 47.20656650130518
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We consider the problem of pretraining a two-stage open-domain question answering (QA) system (retriever + reader) with strong transfer capabilities. The key challenge is how to construct a large amount of high-quality question-answer-context triplets without task-specific annotations. Specifically, the triplets should align well with downstream tasks by: (i) covering a wide range of domains (for open-domain applications), (ii) linking a question to its semantically relevant context with supporting evidence (for training the retriever), and (iii) identifying the correct answer in the context (for training the reader). Previous pretraining approaches generally fall short of one or more of these requirements. In this work, we automatically construct a large-scale corpus that meets all three criteria by consulting millions of references cited within Wikipedia. The well-aligned pretraining signals benefit both the retriever and the reader significantly. Our pretrained retriever leads to 2%-10% absolute gains in top-20 accuracy. And with our pretrained reader, the entire system improves by up to 4% in exact match.

Related papers

• Inferential Question Answering [67.54465021408724]
  We introduce Inferential QA -- a new task that challenges models to infer answers from answer-supporting passages which provide only clues.<n>To study this problem, we construct QUIT (QUestions requiring Inference from Texts) dataset, comprising 7,401 questions and 2.4M passages.<n>We show that methods effective on traditional QA tasks struggle in inferential QA: retrievers underperform, rerankers offer limited gains, and fine-tuning provides inconsistent improvements.
  arXiv  Detail & Related papers  (2026-02-01T14:02:43Z)
• Clarify or Answer: Reinforcement Learning for Agentic VQA with Context Under-specification [10.412378789389544]
  We propose CoA, an ask-or-answer agent that separately models the decision to ask or answer, and what to ask if needed.<n>CoA first determines whether clarification is necessary; if so, it asks a single focused question and then incorporates the response to produce the final answer.<n>CoA achieves consistent improvements at both the module and system levels, improving end-to-end VQA accuracy by an average of +15.3 points (83%) over prompting-based baselines.
  arXiv  Detail & Related papers  (2026-01-23T02:12:33Z)
• Beyond Prompting: An Efficient Embedding Framework for Open-Domain Question Answering [15.04887070246276]
  Large language models have recently pushed open domain question answering to new frontiers. prevailing retriever-reader pipelines often depend on multiple rounds of prompt level instructions. We propose EmbQA, an embedding-level framework that enhances both the retriever and the reader.
  arXiv  Detail & Related papers  (2025-03-03T14:41:35Z)
• Pre-training, Fine-tuning and Re-ranking: A Three-Stage Framework for Legal Question Answering [20.948737566388036]
  Legal question answering (QA) has attracted increasing attention from people seeking legal advice. Previous methods mainly use a dual-encoder architecture to learn dense representations of both questions and answers. We propose a three-stage framework for underlinepre-training, underlinefine-tuning and underlinere-ranking.
  arXiv  Detail & Related papers  (2024-12-27T06:33:42Z)
• Do RAG Systems Cover What Matters? Evaluating and Optimizing Responses with Sub-Question Coverage [74.70255719194819]
  We introduce a novel framework based on sub-question coverage, which measures how well a RAG system addresses different facets of a question. We use this framework to evaluate three commercial generative answer engines: You.com, Perplexity AI, and Bing Chat. We find that while all answer engines cover core sub-questions more often than background or follow-up ones, they still miss around 50% of core sub-questions.
  arXiv  Detail & Related papers  (2024-10-20T22:59:34Z)
• Open-domain Question Answering via Chain of Reasoning over Heterogeneous Knowledge [82.5582220249183]
  We propose a novel open-domain question answering (ODQA) framework for answering single/multi-hop questions across heterogeneous knowledge sources. Unlike previous methods that solely rely on the retriever for gathering all evidence in isolation, our intermediary performs a chain of reasoning over the retrieved set. Our system achieves competitive performance on two ODQA datasets, OTT-QA and NQ, against tables and passages from Wikipedia.
  arXiv  Detail & Related papers  (2022-10-22T03:21:32Z)
• Neural Retriever and Go Beyond: A Thesis Proposal [1.082365064737981]
  Information Retriever (IR) aims to find the relevant documents to a given query at large scale. Recent neural-based algorithms (termed as neural retrievers) have gained more attention which can mitigate the limitations of traditional methods.
  arXiv  Detail & Related papers  (2022-05-31T17:59:30Z)
• Multifaceted Improvements for Conversational Open-Domain Question Answering [54.913313912927045]
  We propose a framework with Multifaceted Improvements for Conversational open-domain Question Answering (MICQA) Firstly, the proposed KL-divergence based regularization is able to lead to a better question understanding for retrieval and answer reading. Second, the added post-ranker module can push more relevant passages to the top placements and be selected for reader with a two-aspect constrains. Third, the well designed curriculum learning strategy effectively narrows the gap between the golden passage settings of training and inference, and encourages the reader to find true answer without the golden passage assistance.
  arXiv  Detail & Related papers  (2022-04-01T07:54:27Z)
• End-to-End Training of Multi-Document Reader and Retriever for Open-Domain Question Answering [36.80395759543162]
  We present an end-to-end differentiable training method for retrieval-augmented open-domain question answering systems. We model retrieval decisions as latent variables over sets of relevant documents. Our proposed method outperforms all existing approaches of comparable size by 2-3% exact match points.
  arXiv  Detail & Related papers  (2021-06-09T19:25:37Z)
• ComQA:Compositional Question Answering via Hierarchical Graph Neural Networks [47.12013005600986]
  We present a large-scale compositional question answering dataset containing more than 120k human-labeled questions. To tackle the ComQA problem, we proposed a hierarchical graph neural networks, which represents the document from the low-level word to the high-level sentence. Our proposed model achieves a significant improvement over previous machine reading comprehension methods and pre-training methods.
  arXiv  Detail & Related papers  (2021-01-16T08:23:27Z)
• Tradeoffs in Sentence Selection Techniques for Open-Domain Question Answering [54.541952928070344]
  We describe two groups of models for sentence selection: QA-based approaches, which run a full-fledged QA system to identify answer candidates, and retrieval-based models, which find parts of each passage specifically related to each question. We show that very lightweight QA models can do well at this task, but retrieval-based models are faster still.
  arXiv  Detail & Related papers  (2020-09-18T23:39:15Z)
• Open-Domain Question Answering with Pre-Constructed Question Spaces [70.13619499853756]
  Open-domain question answering aims at solving the task of locating the answers to user-generated questions in massive collections of documents. There are two families of solutions available: retriever-readers, and knowledge-graph-based approaches. We propose a novel algorithm with a reader-retriever structure that differs from both families.
  arXiv  Detail & Related papers  (2020-06-02T04:31:09Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.