Retrieval Augmentation for Commonsense Reasoning: A Unified Approach

• URL: http://arxiv.org/abs/2210.12887v1
• Date: Sun, 23 Oct 2022 23:49:08 GMT
• Title: Retrieval Augmentation for Commonsense Reasoning: A Unified Approach
• Authors: Wenhao Yu, Chenguang Zhu, Zhihan Zhang, Shuohang Wang, Zhuosheng Zhang, Yuwei Fang, Meng Jiang
• Abstract summary: We propose a unified framework of retrieval-augmented commonsense reasoning (called RACo) Our proposed RACo can significantly outperform other knowledge-enhanced method counterparts.
• Score: 64.63071051375289
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: A common thread of retrieval-augmented methods in the existing literature focuses on retrieving encyclopedic knowledge, such as Wikipedia, which facilitates well-defined entity and relation spaces that can be modeled. However, applying such methods to commonsense reasoning tasks faces two unique challenges, i.e., the lack of a general large-scale corpus for retrieval and a corresponding effective commonsense retriever. In this paper, we systematically investigate how to leverage commonsense knowledge retrieval to improve commonsense reasoning tasks. We proposed a unified framework of retrieval-augmented commonsense reasoning (called RACo), including a newly constructed commonsense corpus with over 20 million documents and novel strategies for training a commonsense retriever. We conducted experiments on four different commonsense reasoning tasks. Extensive evaluation results showed that our proposed RACo can significantly outperform other knowledge-enhanced method counterparts, achieving new SoTA performance on the CommonGen and CREAK leaderboards.

Related papers

• REAR: A Relevance-Aware Retrieval-Augmented Framework for Open-Domain Question Answering [122.62012375722124]
  In existing methods, large language models (LLMs) cannot precisely assess the relevance of retrieved documents. We propose REAR, a RElevance-Aware Retrieval-augmented approach for open-domain question answering (QA)
  arXiv  Detail & Related papers  (2024-02-27T13:22:51Z)
• RAPTOR: Recursive Abstractive Processing for Tree-Organized Retrieval [26.527911244587134]
  We introduce the novel approach of embedding, clustering, and summarizing chunks of text, constructing a tree with differing levels of summarization from the bottom up. At inference time, our RAPTOR model retrieves from this tree, integrating information across lengthy documents at different levels of abstraction.
  arXiv  Detail & Related papers  (2024-01-31T18:30:21Z)
• Modeling Uncertainty and Using Post-fusion as Fallback Improves Retrieval Augmented Generation with LLMs [80.74263278847063]
  The integration of retrieved passages and large language models (LLMs) has significantly contributed to improving open-domain question answering. This paper investigates different methods of combining retrieved passages with LLMs to enhance answer generation.
  arXiv  Detail & Related papers  (2023-08-24T05:26:54Z)
• On the Importance of Exploration for Generalization in Reinforcement Learning [89.63074327328765]
  We propose EDE: Exploration via Distributional Ensemble, a method that encourages exploration of states with high uncertainty. Our algorithm is the first value-based approach to achieve state-of-the-art on both Procgen and Crafter.
  arXiv  Detail & Related papers  (2023-06-08T18:07:02Z)
• Coarse-to-Fine Knowledge Selection for Document Grounded Dialogs [11.63334863772068]
  Multi-document grounded dialogue systems (DGDS) answer users' requests by finding supporting knowledge from a collection of documents. This paper proposes Re3G, which aims to optimize both coarse-grained knowledge retrieval and fine-grained knowledge extraction in a unified framework.
  arXiv  Detail & Related papers  (2023-02-23T08:28:29Z)
• UnifieR: A Unified Retriever for Large-Scale Retrieval [84.61239936314597]
  Large-scale retrieval is to recall relevant documents from a huge collection given a query. Recent retrieval methods based on pre-trained language models (PLM) can be coarsely categorized into either dense-vector or lexicon-based paradigms. We propose a new learning framework, UnifieR which unifies dense-vector and lexicon-based retrieval in one model with a dual-representing capability.
  arXiv  Detail & Related papers  (2022-05-23T11:01:59Z)
• Building an Efficient and Effective Retrieval-based Dialogue System via Mutual Learning [27.04857039060308]
  We propose to combine the best of both worlds to build a retrieval system. We employ a fast bi-encoder to replace the traditional feature-based pre-retrieval model. We train the pre-retrieval model and the re-ranking model at the same time via mutual learning.
  arXiv  Detail & Related papers  (2021-10-01T01:32:33Z)
• Retrieval-Free Knowledge-Grounded Dialogue Response Generation with Adapters [52.725200145600624]
  We propose KnowExpert to bypass the retrieval process by injecting prior knowledge into the pre-trained language models with lightweight adapters. Experimental results show that KnowExpert performs comparably with the retrieval-based baselines.
  arXiv  Detail & Related papers  (2021-05-13T12:33:23Z)
• A Replication Study of Dense Passage Retriever [32.192420072129636]
  We study the dense passage retriever (DPR) technique proposed by Karpukhin et al. ( 2020) for end-to-end open-domain question answering. We present a replication study of this work, starting with model checkpoints provided by the authors. We are able to improve end-to-end question answering effectiveness using exactly the same models as in the original work.
  arXiv  Detail & Related papers  (2021-04-12T18:10:39Z)
• Balancing Reinforcement Learning Training Experiences in Interactive Information Retrieval [19.723551683930776]
  Interactive Information Retrieval (IIR) and Reinforcement Learning (RL) share many commonalities, including an agent who learns while interacting. To successfully apply RL methods to IIR, one challenge is to obtain sufficient relevance labels to train the RL agents. Our paper addresses this issue by using domain randomization to synthesize more relevant documents for the training.
  arXiv  Detail & Related papers  (2020-06-05T00:38:39Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.