TypedThinker: Typed Thinking Improves Large Language Model Reasoning

• URL: http://arxiv.org/abs/2410.01952v1
• Date: Wed, 2 Oct 2024 18:54:45 GMT
• Title: TypedThinker: Typed Thinking Improves Large Language Model Reasoning
• Authors: Danqing Wang, Jianxin Ma, Fei Fang, Lei Li,
• Abstract summary: We propose TypedThinker, a framework that enhances Large Language Models' problem-solving abilities. TypedThinker addresses two key challenges: selecting appropriate reasoning types for given problems and effectively implementing specific reasoning types. Experimental results demonstrate significant improvements over baseline models, with accuracy increases of 3.4% for Mistral 7B and 16.7% for LLaMA3 8B.
• Score: 44.8904486513791
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Despite significant advancements in the reasoning capabilities of Large Language Models (LLMs), the lack of diverse reasoning solutions often makes them trapped in a limited solution search area. In this paper, we propose TypedThinker, a novel framework that enhances LLMs' problem-solving abilities by incorporating multiple reasoning types (deductive, inductive, abductive, and analogical). Our analysis across four benchmarks reveals that different reasoning types uniquely solve distinct sets of problems, highlighting the importance of diverse thinking approaches. TypedThinker addresses two key challenges: selecting appropriate reasoning types for given problems and effectively implementing specific reasoning types. Through self-training on successful experiences, TypedThinker learns an implicit policy for reasoning type selection and application. Experimental results demonstrate significant improvements over baseline models, with accuracy increases of 3.4% for Mistral 7B and 16.7% for LLaMA3 8B across four reasoning benchmarks. Notably, TypedThinker shows effective generalization to new benchmarks and can further enhance the reasoning capability of powerful models like GPT-4o. The code is released at https://github.com/dqwang122/ThinkHub.

Related papers

• Why Reasoning Matters? A Survey of Advancements in Multimodal Reasoning (v1) [66.51642638034822]
  Reasoning is central to human intelligence, enabling structured problem-solving across diverse tasks. Recent advances in large language models (LLMs) have greatly enhanced their reasoning abilities in arithmetic, commonsense, and symbolic domains. This paper offers a concise yet insightful overview of reasoning techniques in both textual and multimodal LLMs.
  arXiv  Detail & Related papers  (2025-04-04T04:04:56Z)
• Meta-Reasoner: Dynamic Guidance for Optimized Inference-time Reasoning in Large Language Models [31.556646366268286]
  Large Language Models increasingly rely on prolonged reasoning chains to solve complex tasks. This trial-and-error approach often leads to high computational overhead and error propagation. We introduce Meta-Reasoner, a framework that dynamically optimize inference-time reasoning.
  arXiv  Detail & Related papers  (2025-02-27T09:40:13Z)
• Large Language Models and Mathematical Reasoning Failures [1.6114012813668932]
  This paper investigates the mathematical reasoning capabilities of large language models (LLMs) using 50 newly constructed high-school-level word problems. We rigorously analyze both final answers and solution steps to identify reasoning failures. We find that while newer models (e.g., o3-mini, deepseek-r1) achieve higher accuracy, all models exhibit errors in spatial reasoning, strategic planning, and arithmetic.
  arXiv  Detail & Related papers  (2025-02-17T09:07:32Z)
• Advancing Reasoning in Large Language Models: Promising Methods and Approaches [0.0]
  Large Language Models (LLMs) have succeeded remarkably in various natural language processing (NLP) tasks. Their ability to perform complex reasoning-spanning logical deduction, mathematical problem-solving, commonsense inference, and multi-step reasoning-often falls short of human expectations. This survey provides a comprehensive review of emerging techniques enhancing reasoning in LLMs.
  arXiv  Detail & Related papers  (2025-02-05T23:31:39Z)
• LlamaV-o1: Rethinking Step-by-step Visual Reasoning in LLMs [103.0226977561914]
  We propose a comprehensive framework for advancing step-by-step visual reasoning in large language models. We introduce a visual reasoning benchmark specifically designed to evaluate multi-step reasoning tasks. Second, we propose a novel metric that assesses visual reasoning quality at the granularity of individual steps. Third, we present a new multimodal visual reasoning model, named LlamaV-o1, trained using a multi-step curriculum learning approach.
  arXiv  Detail & Related papers  (2025-01-10T18:59:51Z)
• Seemingly Plausible Distractors in Multi-Hop Reasoning: Are Large Language Models Attentive Readers? [6.525065859315515]
  We investigate whether Large Language Models (LLMs) are prone to exploiting simplifying cues in multi-hop reasoning benchmarks. Motivated by this finding, we propose a challenging multi-hop reasoning benchmark, by generating seemingly plausible multi-hop reasoning chains. We find that their performance to perform multi-hop reasoning is affected, as indicated by up to 45% relative decrease in F1 score when presented with such seemingly plausible alternatives.
  arXiv  Detail & Related papers  (2024-09-08T19:22:58Z)
• MR-Ben: A Meta-Reasoning Benchmark for Evaluating System-2 Thinking in LLMs [55.20845457594977]
  Large language models (LLMs) have shown increasing capability in problem-solving and decision-making. We present a process-based benchmark MR-Ben that demands a meta-reasoning skill. Our meta-reasoning paradigm is especially suited for system-2 slow thinking.
  arXiv  Detail & Related papers  (2024-06-20T03:50:23Z)
• Conceptual and Unbiased Reasoning in Language Models [98.90677711523645]
  We propose a novel conceptualization framework that forces models to perform conceptual reasoning on abstract questions. We show that existing large language models fall short on conceptual reasoning, dropping 9% to 28% on various benchmarks. We then discuss how models can improve since high-level abstract reasoning is key to unbiased and generalizable decision-making.
  arXiv  Detail & Related papers  (2024-03-30T00:53:53Z)
• LogicAsker: Evaluating and Improving the Logical Reasoning Ability of Large Language Models [63.14196038655506]
  We introduce LogicAsker, a novel approach for evaluating and enhancing the logical reasoning capabilities of large language models (LLMs) Our methodology reveals significant gaps in LLMs' learning of logical rules, with identified reasoning failures ranging from 29% to 90% across different models. We leverage these findings to construct targeted demonstration examples and fine-tune data, notably enhancing logical reasoning in models like GPT-4o by up to 5%.
  arXiv  Detail & Related papers  (2024-01-01T13:53:53Z)
• Large Language Models as Analogical Reasoners [155.9617224350088]
  Chain-of-thought (CoT) prompting for language models demonstrates impressive performance across reasoning tasks. We introduce a new prompting approach, analogical prompting, designed to automatically guide the reasoning process of large language models.
  arXiv  Detail & Related papers  (2023-10-03T00:57:26Z)
• Towards LogiGLUE: A Brief Survey and A Benchmark for Analyzing Logical Reasoning Capabilities of Language Models [56.34029644009297]
  Large language models (LLMs) have demonstrated the ability to overcome various limitations of formal Knowledge Representation (KR) systems. LLMs excel most in abductive reasoning, followed by deductive reasoning, while they are least effective at inductive reasoning. We study single-task training, multi-task training, and "chain-of-thought" knowledge distillation fine-tuning technique to assess the performance of model.
  arXiv  Detail & Related papers  (2023-10-02T01:00:50Z)
• Investigating the Efficacy of Large Language Models in Reflective Assessment Methods through Chain of Thoughts Prompting [0.2552922646705803]
  Chain of Thought(CoT) prompting method has been proposed as a means to enhance LLMs' proficiency in complex reasoning tasks. The primary aim of this research is to assess how well four language models can grade reflective essays of third-year medical students.
  arXiv  Detail & Related papers  (2023-09-30T06:25:27Z)
• Tree of Thoughts: Deliberate Problem Solving with Large Language Models [52.31950122881687]
  We introduce a new framework for language model inference, Tree of Thoughts (ToT) ToT generalizes over the popular Chain of Thought approach to prompting language models. Our experiments show that ToT significantly enhances language models' problem-solving abilities.
  arXiv  Detail & Related papers  (2023-05-17T23:16:17Z)
• IRRGN: An Implicit Relational Reasoning Graph Network for Multi-turn Response Selection [4.471148909362883]
  Implicit Reasoning to Graph Network aims to implicitly extract between utterances, as well as utterances and options. Model surpasses human performance for the first time on the MuTual dataset.
  arXiv  Detail & Related papers  (2022-12-01T13:17:25Z)
• Making Large Language Models Better Reasoners with Step-Aware Verifier [49.16750018427259]
  DIVERSE (Diverse Verifier on Reasoning Step) is a novel approach that further enhances the reasoning capability of language models. We evaluate DIVERSE on the latest language model code-davinci and show that it achieves new state-of-the-art results on six of eight reasoning benchmarks.
  arXiv  Detail & Related papers  (2022-06-06T03:38:36Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.