Answering Questions by Meta-Reasoning over Multiple Chains of Thought

• URL: http://arxiv.org/abs/2304.13007v4
• Date: Fri, 2 Aug 2024 14:18:51 GMT
• Title: Answering Questions by Meta-Reasoning over Multiple Chains of Thought
• Authors: Ori Yoran, Tomer Wolfson, Ben Bogin, Uri Katz, Daniel Deutch, Jonathan Berant,
• Abstract summary: We introduce Multi-Chain Reasoning (MCR), an approach which prompts large language models to meta-reason over multiple chains of thought. MCR examines different reasoning chains, mixes information between them and selects the most relevant facts in generating an explanation and predicting the answer.
• Score: 53.55653437903948
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Modern systems for multi-hop question answering (QA) typically break questions into a sequence of reasoning steps, termed chain-of-thought (CoT), before arriving at a final answer. Often, multiple chains are sampled and aggregated through a voting mechanism over the final answers, but the intermediate steps themselves are discarded. While such approaches improve performance, they do not consider the relations between intermediate steps across chains and do not provide a unified explanation for the predicted answer. We introduce Multi-Chain Reasoning (MCR), an approach which prompts large language models to meta-reason over multiple chains of thought, rather than aggregating their answers. MCR examines different reasoning chains, mixes information between them and selects the most relevant facts in generating an explanation and predicting the answer. MCR outperforms strong baselines on 7 multi-hop QA datasets. Moreover, our analysis reveals that MCR explanations exhibit high quality, enabling humans to verify its answers.

Related papers

• STOC-TOT: Stochastic Tree-of-Thought with Constrained Decoding for Complex Reasoning in Multi-Hop Question Answering [8.525847131940031]
  Multi-hop question answering (MHQA) requires a model to retrieve and integrate information from multiple passages to answer a complex question. Recent systems leverage the power of large language models and integrate evidence retrieval with reasoning prompts. We propose STOC-TOT, a tree-of-thought reasoning prompting method with constrained decoding for MHQA.
  arXiv  Detail & Related papers  (2024-07-04T07:17:53Z)
• Chain-of-Action: Faithful and Multimodal Question Answering through Large Language Models [17.60243337898751]
  We present a Chain-of-Action framework for multimodal and retrieval-augmented Question-Answering (QA) Compared to the literature, CoA overcomes two major challenges of current QA applications: (i) unfaithful hallucination that is inconsistent with real-time or domain facts and (ii) weak reasoning performance over compositional information.
  arXiv  Detail & Related papers  (2024-03-26T03:51:01Z)
• How Many Answers Should I Give? An Empirical Study of Multi-Answer Reading Comprehension [64.76737510530184]
  We design a taxonomy to categorize commonly-seen multi-answer MRC instances. We analyze how well different paradigms of current multi-answer MRC models deal with different types of multi-answer instances.
  arXiv  Detail & Related papers  (2023-06-01T08:22:21Z)
• Complexity-Based Prompting for Multi-Step Reasoning [72.0057198610614]
  We study the task of prompting large-scale language models to perform multi-step reasoning. A central question is which reasoning examples make the most effective prompts. We propose complexity-based prompting, a simple and effective example selection scheme for multi-step reasoning.
  arXiv  Detail & Related papers  (2022-10-03T05:33:27Z)
• Interpretable AMR-Based Question Decomposition for Multi-hop Question Answering [12.35571328854374]
  We propose a Question Decomposition method based on Abstract Meaning Representation (QDAMR) for multi-hop QA. We decompose a multi-hop question into simpler sub-questions and answer them in order. Experimental results on HotpotQA demonstrate that our approach is competitive for interpretable reasoning.
  arXiv  Detail & Related papers  (2022-06-16T23:46:33Z)
• Modeling Multi-hop Question Answering as Single Sequence Prediction [88.72621430714985]
  We propose a simple generative approach (PathFid) that extends the task beyond just answer generation. PathFid explicitly models the reasoning process to resolve the answer for multi-hop questions. Our experiments demonstrate that PathFid leads to strong performance gains on two multi-hop QA datasets.
  arXiv  Detail & Related papers  (2022-05-18T21:57:59Z)
• Ask to Understand: Question Generation for Multi-hop Question Answering [11.626390908264872]
  Multi-hop Question Answering (QA) requires the machine to answer complex questions by finding scattering clues and reasoning from multiple documents. We propose a novel method to complete multi-hop QA from the perspective of Question Generation (QG)
  arXiv  Detail & Related papers  (2022-03-17T04:02:29Z)
• Multi-Perspective Abstractive Answer Summarization [76.10437565615138]
  Community Question Answering forums contain a rich resource of answers to a wide range of questions. The goal of multi-perspective answer summarization is to produce a summary that includes all perspectives of the answer. This work introduces a novel dataset creation method to automatically create multi-perspective, bullet-point abstractive summaries.
  arXiv  Detail & Related papers  (2021-04-17T13:15:29Z)
• Unsupervised Question Decomposition for Question Answering [102.56966847404287]
  We propose an algorithm for One-to-N Unsupervised Sequence Sequence (ONUS) that learns to map one hard, multi-hop question to many simpler, single-hop sub-questions. We show large QA improvements on HotpotQA over a strong baseline on the original, out-of-domain, and multi-hop dev sets.
  arXiv  Detail & Related papers  (2020-02-22T19:40:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.