Latent Skill Discovery for Chain-of-Thought Reasoning
- URL: http://arxiv.org/abs/2312.04684v1
- Date: Thu, 7 Dec 2023 20:36:10 GMT
- Title: Latent Skill Discovery for Chain-of-Thought Reasoning
- Authors: Zifan Xu, Haozhu Wang, Dmitriy Bespalov, Peter Stone, Yanjun Qi
- Abstract summary: We introduce Reasoning Skill Discovery (RSD) that use unsupervised learning to create a latent space representation of rationales.
This can then guide the selection of examples that demonstrate the required reasoning skills.
Empirically, RSD outperforms existing methods by up to 6% in terms of the answer accuracy across multiple reasoning tasks.
- Score: 44.30408985857306
- License: http://creativecommons.org/licenses/by-nc-sa/4.0/
- Abstract: Recent advances in Large Language Models (LLMs) have led to an emergent
ability of chain-of-thought (CoT) prompting, a prompt reasoning strategy that
adds intermediate rationale steps between questions and answers to construct
prompts. Conditioned on these prompts, LLMs can effectively learn in context to
generate rationales that lead to more accurate answers than when answering the
same question directly. To design LLM prompts, one important setting, called
demonstration selection, considers selecting demonstrations from an example
bank. Existing methods use various heuristics for this selection, but for CoT
prompting, which involves unique rationales, it is essential to base the
selection upon the intrinsic skills that CoT rationales need, for instance, the
skills of addition or subtraction for math word problems.
To address this requirement, we introduce a novel approach named Reasoning
Skill Discovery (RSD) that use unsupervised learning to create a latent space
representation of rationales, called a reasoning skill. Simultaneously, RSD
learns a reasoning policy to determine the required reasoning skill for a given
question. This can then guide the selection of examples that demonstrate the
required reasoning skills. Our approach offers several desirable properties: it
is (1) theoretically grounded, (2) sample-efficient, requiring no LLM inference
or manual prompt design, and (3) LLM-agnostic. Empirically, RSD outperforms
existing methods by up to 6% in terms of the answer accuracy across multiple
reasoning tasks.
Related papers
- Reasoning with Large Language Models, a Survey [2.831296564800826]
This paper reviews the rapidly expanding field of prompt-based reasoning with LLMs.
Our taxonomy identifies different ways to generate, evaluate, and control multi-step reasoning.
We find that self-improvement, self-reflection, and some meta abilities of the reasoning processes are possible through the judicious use of prompts.
arXiv Detail & Related papers (2024-07-16T08:49:35Z) - Combinatorial Reasoning: Selecting Reasons in Generative AI Pipelines via Combinatorial Optimization [2.090904951468026]
Large Language Models (LLMs) have demonstrated impressive capabilities at tasks that require human intelligence.
Yet the reasoning capability of LLMs is a matter of significant debate.
We introduce a framework for what we call Combinatorial Reasoning (CR), a fully-automated prompting method.
arXiv Detail & Related papers (2024-06-19T16:47:44Z) - LogicBench: Towards Systematic Evaluation of Logical Reasoning Ability of Large Language Models [52.03659714625452]
Recently developed large language models (LLMs) have been shown to perform remarkably well on a wide range of language understanding tasks.
But, can they really "reason" over the natural language?
This question has been receiving significant research attention and many reasoning skills such as commonsense, numerical, and qualitative have been studied.
arXiv Detail & Related papers (2024-04-23T21:08:49Z) - DetermLR: Augmenting LLM-based Logical Reasoning from Indeterminacy to Determinacy [76.58614128865652]
We propose DetermLR, a novel perspective that rethinks the reasoning process as an evolution from indeterminacy to determinacy.
First, we categorize known conditions into two types: determinate and indeterminate premises This provides an oveall direction for the reasoning process and guides LLMs in converting indeterminate data into progressively determinate insights.
We automate the storage and extraction of available premises and reasoning paths with reasoning memory, preserving historical reasoning details for subsequent reasoning steps.
arXiv Detail & Related papers (2023-10-28T10:05:51Z) - Towards LogiGLUE: A Brief Survey and A Benchmark for Analyzing Logical Reasoning Capabilities of Language Models [56.34029644009297]
Large language models (LLMs) have demonstrated the ability to overcome various limitations of formal Knowledge Representation (KR) systems.
LLMs excel most in abductive reasoning, followed by deductive reasoning, while they are least effective at inductive reasoning.
We study single-task training, multi-task training, and "chain-of-thought" knowledge distillation fine-tuning technique to assess the performance of model.
arXiv Detail & Related papers (2023-10-02T01:00:50Z) - Boosting Language Models Reasoning with Chain-of-Knowledge Prompting [18.326858925174605]
Chain-of-Knowledge (CoK) prompting aims at eliciting explicit pieces of knowledge evidence in the form of structure triple.
Benefiting from CoK, we additionally introduce a F2-Verification method to estimate the reliability of the reasoning chains.
Extensive experiments demonstrate that our method can further improve the performance of commonsense, factual, symbolic, and arithmetic reasoning tasks.
arXiv Detail & Related papers (2023-06-10T12:42:36Z) - Active Prompting with Chain-of-Thought for Large Language Models [26.5029080638055]
This paper proposes a new method, Active-Prompt, to adapt large language models to different tasks.
By borrowing ideas from the related problem of uncertainty-based active learning, we introduce several metrics to characterize the uncertainty.
Experimental results demonstrate the superiority of our proposed method, achieving state-of-the-art on eight complex reasoning tasks.
arXiv Detail & Related papers (2023-02-23T18:58:59Z) - Towards Understanding Chain-of-Thought Prompting: An Empirical Study of
What Matters [82.84696222087396]
Chain-of-Thought (CoT) prompting can dramatically improve the multi-step reasoning abilities of large language models (LLMs)
We show that CoT reasoning is possible even with invalid demonstrations.
arXiv Detail & Related papers (2022-12-20T05:20:54Z) - Rationale-Augmented Ensembles in Language Models [53.45015291520658]
We reconsider rationale-augmented prompting for few-shot in-context learning.
We identify rationale sampling in the output space as the key component to robustly improve performance.
We demonstrate that rationale-augmented ensembles achieve more accurate and interpretable results than existing prompting approaches.
arXiv Detail & Related papers (2022-07-02T06:20:57Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.