Retrieval Augmentation for Commonsense Reasoning: A Unified Approach

• URL: http://arxiv.org/abs/2210.12887v1
• Date: Sun, 23 Oct 2022 23:49:08 GMT
• Title: Retrieval Augmentation for Commonsense Reasoning: A Unified Approach
• Authors: Wenhao Yu, Chenguang Zhu, Zhihan Zhang, Shuohang Wang, Zhuosheng Zhang, Yuwei Fang, Meng Jiang
• Abstract summary: We propose a unified framework of retrieval-augmented commonsense reasoning (called RACo) Our proposed RACo can significantly outperform other knowledge-enhanced method counterparts.
• Score: 64.63071051375289
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: A common thread of retrieval-augmented methods in the existing literature focuses on retrieving encyclopedic knowledge, such as Wikipedia, which facilitates well-defined entity and relation spaces that can be modeled. However, applying such methods to commonsense reasoning tasks faces two unique challenges, i.e., the lack of a general large-scale corpus for retrieval and a corresponding effective commonsense retriever. In this paper, we systematically investigate how to leverage commonsense knowledge retrieval to improve commonsense reasoning tasks. We proposed a unified framework of retrieval-augmented commonsense reasoning (called RACo), including a newly constructed commonsense corpus with over 20 million documents and novel strategies for training a commonsense retriever. We conducted experiments on four different commonsense reasoning tasks. Extensive evaluation results showed that our proposed RACo can significantly outperform other knowledge-enhanced method counterparts, achieving new SoTA performance on the CommonGen and CREAK leaderboards.

Related papers

• Bridging Search and Recommendation in Generative Retrieval: Does One Task Help the Other? [9.215695600542249]
  Generative retrieval for search and recommendation is a promising paradigm for retrieving items. These generative systems can play a crucial role in centralizing a variety of Information Retrieval (IR) tasks in a single model. This paper investigates whether and when such a unified approach can outperform task-specific models in the IR tasks of search and recommendation.
  arXiv  Detail & Related papers  (2024-10-22T08:49:43Z)
• Unified Active Retrieval for Retrieval Augmented Generation [69.63003043712696]
  In Retrieval-Augmented Generation (RAG), retrieval is not always helpful and applying it to every instruction is sub-optimal. Existing active retrieval methods face two challenges: 1. They usually rely on a single criterion, which struggles with handling various types of instructions. They depend on specialized and highly differentiated procedures, and thus combining them makes the RAG system more complicated.
  arXiv  Detail & Related papers  (2024-06-18T12:09:02Z)
• RAPTOR: Recursive Abstractive Processing for Tree-Organized Retrieval [26.527911244587134]
  We introduce the novel approach of embedding, clustering, and summarizing chunks of text, constructing a tree with differing levels of summarization from the bottom up. At inference time, our RAPTOR model retrieves from this tree, integrating information across lengthy documents at different levels of abstraction.
  arXiv  Detail & Related papers  (2024-01-31T18:30:21Z)
• Modeling Uncertainty and Using Post-fusion as Fallback Improves Retrieval Augmented Generation with LLMs [80.74263278847063]
  The integration of retrieved passages and large language models (LLMs) has significantly contributed to improving open-domain question answering. This paper investigates different methods of combining retrieved passages with LLMs to enhance answer generation.
  arXiv  Detail & Related papers  (2023-08-24T05:26:54Z)
• On the Importance of Exploration for Generalization in Reinforcement Learning [89.63074327328765]
  We propose EDE: Exploration via Distributional Ensemble, a method that encourages exploration of states with high uncertainty. Our algorithm is the first value-based approach to achieve state-of-the-art on both Procgen and Crafter.
  arXiv  Detail & Related papers  (2023-06-08T18:07:02Z)
• Coarse-to-Fine Knowledge Selection for Document Grounded Dialogs [11.63334863772068]
  Multi-document grounded dialogue systems (DGDS) answer users' requests by finding supporting knowledge from a collection of documents. This paper proposes Re3G, which aims to optimize both coarse-grained knowledge retrieval and fine-grained knowledge extraction in a unified framework.
  arXiv  Detail & Related papers  (2023-02-23T08:28:29Z)
• UnifieR: A Unified Retriever for Large-Scale Retrieval [84.61239936314597]
  Large-scale retrieval is to recall relevant documents from a huge collection given a query. Recent retrieval methods based on pre-trained language models (PLM) can be coarsely categorized into either dense-vector or lexicon-based paradigms. We propose a new learning framework, UnifieR which unifies dense-vector and lexicon-based retrieval in one model with a dual-representing capability.
  arXiv  Detail & Related papers  (2022-05-23T11:01:59Z)
• Building an Efficient and Effective Retrieval-based Dialogue System via Mutual Learning [27.04857039060308]
  We propose to combine the best of both worlds to build a retrieval system. We employ a fast bi-encoder to replace the traditional feature-based pre-retrieval model. We train the pre-retrieval model and the re-ranking model at the same time via mutual learning.
  arXiv  Detail & Related papers  (2021-10-01T01:32:33Z)
• Retrieval-Free Knowledge-Grounded Dialogue Response Generation with Adapters [52.725200145600624]
  We propose KnowExpert to bypass the retrieval process by injecting prior knowledge into the pre-trained language models with lightweight adapters. Experimental results show that KnowExpert performs comparably with the retrieval-based baselines.
  arXiv  Detail & Related papers  (2021-05-13T12:33:23Z)
• A Replication Study of Dense Passage Retriever [32.192420072129636]
  We study the dense passage retriever (DPR) technique proposed by Karpukhin et al. ( 2020) for end-to-end open-domain question answering. We present a replication study of this work, starting with model checkpoints provided by the authors. We are able to improve end-to-end question answering effectiveness using exactly the same models as in the original work.
  arXiv  Detail & Related papers  (2021-04-12T18:10:39Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.