HiQA: A Hierarchical Contextual Augmentation RAG for Massive Documents
QA
- URL: http://arxiv.org/abs/2402.01767v1
- Date: Thu, 1 Feb 2024 02:24:15 GMT
- Title: HiQA: A Hierarchical Contextual Augmentation RAG for Massive Documents
QA
- Authors: Xinyue Chen, Pengyu Gao, Jiangjiang Song, Xiaoyang Tan
- Abstract summary: HiQA integrates cascading metadata into content as well as a multi-route retrieval mechanism.
We release a benchmark called MasQA to evaluate and research in MDQA.
- Score: 14.20201554222619
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: As language model agents leveraging external tools rapidly evolve,
significant progress has been made in question-answering(QA) methodologies
utilizing supplementary documents and the Retrieval-Augmented Generation (RAG)
approach. This advancement has improved the response quality of language models
and alleviates the appearance of hallucination. However, these methods exhibit
limited retrieval accuracy when faced with massive indistinguishable documents,
presenting notable challenges in their practical application. In response to
these emerging challenges, we present HiQA, an advanced framework for
multi-document question-answering (MDQA) that integrates cascading metadata
into content as well as a multi-route retrieval mechanism. We also release a
benchmark called MasQA to evaluate and research in MDQA. Finally, HiQA
demonstrates the state-of-the-art performance in multi-document environments.
Related papers
- KaPQA: Knowledge-Augmented Product Question-Answering [59.096607961704656]
We introduce two product question-answering (QA) datasets focused on Adobe Acrobat and Photoshop products.
We also propose a novel knowledge-driven RAG-QA framework to enhance the performance of the models in the product QA task.
arXiv Detail & Related papers (2024-07-22T22:14:56Z) - Visual Haystacks: Answering Harder Questions About Sets of Images [63.296342841358815]
This paper explores the task of Multi-Image Visual Question Answering (MIQA)
Given a large set of images and a natural language query, the task is to generate a relevant and grounded response.
We introduce MIRAGE, a novel retrieval/QA framework tailored for Large Multimodal Models (LMMs)
arXiv Detail & Related papers (2024-07-18T17:59:30Z) - DEXTER: A Benchmark for open-domain Complex Question Answering using LLMs [3.24692739098077]
Open-domain complex Question Answering (QA) is a difficult task with challenges in evidence retrieval and reasoning.
We evaluate state-of-the-art pre-trained dense and sparse retrieval models in an open-domain setting.
We observe that late interaction models and surprisingly lexical models like BM25 perform well compared to other pre-trained dense retrieval models.
arXiv Detail & Related papers (2024-06-24T22:09:50Z) - QontSum: On Contrasting Salient Content for Query-focused Summarization [22.738731393540633]
Query-focused summarization (QFS) is a challenging task in natural language processing that generates summaries to address specific queries.
This paper highlights the role of QFS in Grounded Answer Generation (GAR)
We propose QontSum, a novel approach for QFS that leverages contrastive learning to help the model attend to the most relevant regions of the input document.
arXiv Detail & Related papers (2023-07-14T19:25:35Z) - Peek Across: Improving Multi-Document Modeling via Cross-Document
Question-Answering [49.85790367128085]
We pre-training a generic multi-document model from a novel cross-document question answering pre-training objective.
This novel multi-document QA formulation directs the model to better recover cross-text informational relations.
Unlike prior multi-document models that focus on either classification or summarization tasks, our pre-training objective formulation enables the model to perform tasks that involve both short text generation and long text generation.
arXiv Detail & Related papers (2023-05-24T17:48:40Z) - RoMQA: A Benchmark for Robust, Multi-evidence, Multi-answer Question
Answering [87.18962441714976]
We introduce RoMQA, the first benchmark for robust, multi-evidence, multi-answer question answering (QA)
We evaluate state-of-the-art large language models in zero-shot, few-shot, and fine-tuning settings, and find that RoMQA is challenging.
Our results show that RoMQA is a challenging benchmark for large language models, and provides a quantifiable test to build more robust QA methods.
arXiv Detail & Related papers (2022-10-25T21:39:36Z) - Narrative Question Answering with Cutting-Edge Open-Domain QA
Techniques: A Comprehensive Study [45.9120218818558]
We benchmark the research on the NarrativeQA dataset with experiments with cutting-edge ODQA techniques.
This quantifies the challenges Book QA poses, as well as advances the published state-of-the-art with a $sim$7% absolute improvement on Rouge-L.
Our findings indicate that the event-centric questions dominate this task, which exemplifies the inability of existing QA models to handle event-oriented scenarios.
arXiv Detail & Related papers (2021-06-07T17:46:09Z) - Retrieving and Reading: A Comprehensive Survey on Open-domain Question
Answering [62.88322725956294]
We review the latest research trends in OpenQA, with particular attention to systems that incorporate neural MRC techniques.
We introduce modern OpenQA architecture named Retriever-Reader'' and analyze the various systems that follow this architecture.
We then discuss key challenges to developing OpenQA systems and offer an analysis of benchmarks that are commonly used.
arXiv Detail & Related papers (2021-01-04T04:47:46Z) - Template-Based Question Generation from Retrieved Sentences for Improved
Unsupervised Question Answering [98.48363619128108]
We propose an unsupervised approach to training QA models with generated pseudo-training data.
We show that generating questions for QA training by applying a simple template on a related, retrieved sentence rather than the original context sentence improves downstream QA performance.
arXiv Detail & Related papers (2020-04-24T17:57:45Z) - Towards Automatic Generation of Questions from Long Answers [11.198653485869935]
We propose a novel evaluation benchmark to assess the performance of existing AQG systems for long-text answers.
We empirically demonstrate that the performance of existing AQG methods significantly degrades as the length of the answer increases.
Transformer-based methods outperform other existing AQG methods on long answers in terms of automatic as well as human evaluation.
arXiv Detail & Related papers (2020-04-10T16:45:08Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.