Plan, Verify and Switch: Integrated Reasoning with Diverse X-of-Thoughts
- URL: http://arxiv.org/abs/2310.14628v2
- Date: Wed, 27 Dec 2023 13:54:48 GMT
- Title: Plan, Verify and Switch: Integrated Reasoning with Diverse X-of-Thoughts
- Authors: Tengxiao Liu, Qipeng Guo, Yuqing Yang, Xiangkun Hu, Yue Zhang, Xipeng
Qiu, Zheng Zhang
- Abstract summary: We propose XoT, an integrated problem solving framework by prompting LLMs with diverse reasoning thoughts.
For each question, XoT always begins with selecting the most suitable method then executes each method iteratively.
Within each iteration, XoT actively checks the validity of the generated answer and incorporates the feedback from external executors.
- Score: 65.15322403136238
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: As large language models (LLMs) have shown effectiveness with different
prompting methods, such as Chain of Thought, Program of Thought, we find that
these methods have formed a great complementarity to each other on math
reasoning tasks. In this work, we propose XoT, an integrated problem solving
framework by prompting LLMs with diverse reasoning thoughts. For each question,
XoT always begins with selecting the most suitable method then executes each
method iteratively. Within each iteration, XoT actively checks the validity of
the generated answer and incorporates the feedback from external executors,
allowing it to dynamically switch among different prompting methods. Through
extensive experiments on 10 popular math reasoning datasets, we demonstrate the
effectiveness of our proposed approach and thoroughly analyze the strengths of
each module. Moreover, empirical results suggest that our framework is
orthogonal to recent work that makes improvements on single reasoning methods
and can further generalise to logical reasoning domain. By allowing method
switching, XoT provides a fresh perspective on the collaborative integration of
diverse reasoning thoughts in a unified framework. The code is available at
https://github.com/tengxiaoliu/XoT.
Related papers
- Thought-Path Contrastive Learning via Premise-Oriented Data Augmentation for Logical Reading Comprehension [9.67774998354062]
Previous research has primarily focused on enhancing logical reasoning capabilities through Chain-of-Thought (CoT) or data augmentation.
We propose a Premise-Oriented Data Augmentation (PODA) framework to generate CoT rationales including analyses for both correct and incorrect options.
We also introduce a novel thought-path contrastive learning method that compares reasoning paths between the original and counterfactual samples.
arXiv Detail & Related papers (2024-09-22T15:44:43Z) - Iteration of Thought: Leveraging Inner Dialogue for Autonomous Large Language Model Reasoning [0.0]
Iterative human engagement is a common and effective means of leveraging the advanced language processing power of large language models (LLMs)
We propose the Iteration of Thought (IoT) framework for enhancing LLM responses by generating "thought"-provoking prompts.
Unlike static or semi-static approaches, IoT adapts its reasoning path dynamically, based on evolving context.
arXiv Detail & Related papers (2024-09-19T09:44:17Z) - Generating Chain-of-Thoughts with a Pairwise-Comparison Approach to Searching for the Most Promising Intermediate Thought [70.30423016640749]
Chain-of-thoughts (CoT) methods were proposed to guide large language models to reason step-by-step, enabling problem solving from simple to complex.
The evaluation from the large language model (LLMs) is typically noisy and unreliable, potentially misleading the generation process in selecting promising intermediate thoughts.
In this paper, motivated by Vapnik's principle, we use pairwise-comparison evaluation instead of point-wise scoring to search for promising intermediate thoughts.
arXiv Detail & Related papers (2024-02-10T09:51:03Z) - Diversity of Thought Improves Reasoning Abilities of LLMs [26.149914503910235]
Large language models (LLMs) are documented to struggle in settings that require complex reasoning.
We discuss how one can create and leverage variations of the input prompt as a means of diversity of thought.
arXiv Detail & Related papers (2023-10-11T00:01:41Z) - SCREWS: A Modular Framework for Reasoning with Revisions [58.698199183147935]
We present SCREWS, a modular framework for reasoning with revisions.
We show that SCREWS unifies several previous approaches under a common framework.
We evaluate our framework with state-of-the-art LLMs on a diverse set of reasoning tasks.
arXiv Detail & Related papers (2023-09-20T15:59:54Z) - Self-Polish: Enhance Reasoning in Large Language Models via Problem Refinement [50.62461749446111]
Self-Polish (SP) is a novel method that facilitates the model's reasoning by guiding it to progressively refine the given problems to be more comprehensible and solvable.
SP is to all other prompting methods of answer/reasoning side like CoT, allowing for seamless integration with state-of-the-art techniques for further improvement.
arXiv Detail & Related papers (2023-05-23T19:58:30Z) - Multimodal Chain-of-Thought Reasoning in Language Models [94.70184390935661]
We propose Multimodal-CoT that incorporates language (text) and vision (images) modalities into a two-stage framework.
Experimental results on ScienceQA and A-OKVQA benchmark datasets show the effectiveness of our proposed approach.
arXiv Detail & Related papers (2023-02-02T07:51:19Z) - Taxonomy of Dual Block-Coordinate Ascent Methods for Discrete Energy
Minimization [96.1052289276254]
We consider the maximum-a-posteriori inference problem in discrete graphical models and study solvers based on the dual block-coordinate ascent rule.
We map all existing solvers in a single framework, allowing for a better understanding of their design principles.
arXiv Detail & Related papers (2020-04-16T15:49:13Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.