Exploring the Hidden Reasoning Process of Large Language Models by Misleading Them

• URL: http://arxiv.org/abs/2503.16401v1
• Date: Thu, 20 Mar 2025 17:54:42 GMT
• Title: Exploring the Hidden Reasoning Process of Large Language Models by Misleading Them
• Authors: Guanyu Chen, Peiyang Wang, Tianren Zhang, Feng Chen,
• Abstract summary: Large language models (LLMs) and Vision language models (VLMs) have been able to perform various forms of reasoning tasks.<n>We propose a novel experimental approach, Misleading Fine-Tuning (MisFT), to examine whether LLMs/VLMs perform abstract reasoning.
• Score: 5.4908640334628345
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Large language models (LLMs) and Vision language models (VLMs) have been able to perform various forms of reasoning tasks in a wide range of scenarios, but are they truly engaging in task abstraction and rule-based reasoning beyond mere memorization and pattern matching? To answer this question, we propose a novel experimental approach, Misleading Fine-Tuning (MisFT), to examine whether LLMs/VLMs perform abstract reasoning by altering their original understanding of fundamental rules. In particular, by constructing a dataset with math expressions that contradict correct operation principles, we fine-tune the model to learn those contradictory rules and assess its generalization ability on different test domains. Through a series of experiments, we find that current LLMs/VLMs are capable of effectively applying contradictory rules to solve practical math word problems and math expressions represented by images, implying the presence of an internal mechanism that abstracts before reasoning.

Related papers

• Hypothesis-Driven Theory-of-Mind Reasoning for Large Language Models [76.6028674686018]
  We introduce thought-tracing, an inference-time reasoning algorithm to trace the mental states of agents.<n>Our algorithm is modeled after the Bayesian theory-of-mind framework.<n>We evaluate thought-tracing on diverse theory-of-mind benchmarks, demonstrating significant performance improvements.
  arXiv  Detail & Related papers  (2025-02-17T15:08:50Z)
• LogiDynamics: Unraveling the Dynamics of Logical Inference in Large Language Model Reasoning [49.58786377307728]
  This paper adopts an exploratory approach by introducing a controlled evaluation environment for analogical reasoning. We analyze the comparative dynamics of inductive, abductive, and deductive inference pipelines. We investigate advanced paradigms such as hypothesis selection, verification, and refinement, revealing their potential to scale up logical inference.
  arXiv  Detail & Related papers  (2025-02-16T15:54:53Z)
• The Inherent Limits of Pretrained LLMs: The Unexpected Convergence of Instruction Tuning and In-Context Learning Capabilities [51.594836904623534]
  We investigate whether instruction-tuned models possess fundamentally different capabilities from base models that are prompted using in-context examples. We show that the performance of instruction-tuned models is significantly correlated with the in-context performance of their base counterparts. Specifically, we extend this understanding to instruction-tuned models, suggesting that their pretraining data similarly sets a limiting boundary on the tasks they can solve.
  arXiv  Detail & Related papers  (2025-01-15T10:57:55Z)
• Argumentation Computation with Large Language Models : A Benchmark Study [6.0682923348298194]
  Large language models (LLMs) have made significant advancements in neuro-symbolic computing. We aim to investigate the capability of LLMs in determining the extensions of various abstract argumentation semantics.
  arXiv  Detail & Related papers  (2024-12-21T18:23:06Z)
• MIRAGE: Evaluating and Explaining Inductive Reasoning Process in Language Models [19.81485079689837]
  We evaluate large language models' capabilities in inductive and deductive stages.<n>We find that the models tend to consistently conduct correct deduction without correct inductive rules.<n>In the inductive reasoning process, the model tends to focus on observed facts that are close to the current test example in feature space.
  arXiv  Detail & Related papers  (2024-10-12T14:12:36Z)
• Can LLMs Reason with Rules? Logic Scaffolding for Stress-Testing and Improving LLMs [87.34281749422756]
  Large language models (LLMs) have achieved impressive human-like performance across various reasoning tasks. However, their mastery of underlying inferential rules still falls short of human capabilities. We propose a logic scaffolding inferential rule generation framework, to construct an inferential rule base, ULogic.
  arXiv  Detail & Related papers  (2024-02-18T03:38:51Z)
• Phenomenal Yet Puzzling: Testing Inductive Reasoning Capabilities of Language Models with Hypothesis Refinement [92.61557711360652]
  Language models (LMs) often fall short on inductive reasoning, despite achieving impressive success on research benchmarks. We conduct a systematic study of the inductive reasoning capabilities of LMs through iterative hypothesis refinement. We reveal several discrepancies between the inductive reasoning processes of LMs and humans, shedding light on both the potentials and limitations of using LMs in inductive reasoning tasks.
  arXiv  Detail & Related papers  (2023-10-12T17:51:10Z)
• Large Language Models can Learn Rules [106.40747309894236]
  We present Hypotheses-to-Theories (HtT), a framework that learns a rule library for reasoning with large language models (LLMs)<n> Experiments on relational reasoning, numerical reasoning and concept learning problems show that HtT improves existing prompting methods.<n>The learned rules are also transferable to different models and to different forms of the same problem.
  arXiv  Detail & Related papers  (2023-10-10T23:07:01Z)
• ThinkSum: Probabilistic reasoning over sets using large language models [18.123895485602244]
  We propose a two-stage probabilistic inference paradigm, ThinkSum, which reasons over sets of objects or facts in a structured manner. We demonstrate the possibilities and advantages of ThinkSum on the BIG-bench suite of LLM evaluation tasks.
  arXiv  Detail & Related papers  (2022-10-04T00:34:01Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.