CSCE: Boosting LLM Reasoning by Simultaneous Enhancing of Casual Significance and Consistency

• URL: http://arxiv.org/abs/2409.17174v1
• Date: Fri, 20 Sep 2024 08:28:23 GMT
• Title: CSCE: Boosting LLM Reasoning by Simultaneous Enhancing of Casual Significance and Consistency
• Authors: Kangsheng Wang, Xiao Zhang, Zizheng Guo, Tianyu Hu, Huimin Ma,
• Abstract summary: Chain-based reasoning methods like chain of thought (CoT) play a rising role in solving reasoning tasks for large language models (LLMs) This paper proposes a non-chain-based reasoning framework for simultaneous consideration of causal significance and consistency.
• Score: 12.961692839965115
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Chain-based reasoning methods like chain of thought (CoT) play a rising role in solving reasoning tasks for large language models (LLMs). However, the causal illusions between \textit{a step of reasoning} and \textit{corresponding state transitions} are becoming a significant obstacle to advancing LLMs' reasoning capabilities, especially in long-range reasoning tasks. This paper proposes a non-chain-based reasoning framework for simultaneous consideration of causal significance and consistency, i.e., the Causal Significance and Consistency Enhancer (CSCE). We customize LLM's loss function utilizing treatment effect assessments to enhance its reasoning ability from two aspects: causal significance and consistency. This ensures that the model captures essential causal relationships and maintains robust and consistent performance across various scenarios. Additionally, we transform the reasoning process from the cascading multiple one-step reasoning commonly used in Chain-Based methods, like CoT, to a causal-enhanced method that outputs the entire reasoning process in one go, further improving the model's reasoning efficiency. Extensive experiments show that our method improves both the reasoning success rate and speed. These improvements further demonstrate that non-chain-based methods can also aid LLMs in completing reasoning tasks.

Related papers

• Make LLMs better zero-shot reasoners: Structure-orientated autonomous reasoning [52.83539473110143]
  We introduce a novel structure-oriented analysis method to help Large Language Models (LLMs) better understand a question. To further improve the reliability in complex question-answering tasks, we propose a multi-agent reasoning system, Structure-oriented Autonomous Reasoning Agents (SARA) Extensive experiments verify the effectiveness of the proposed reasoning system. Surprisingly, in some cases, the system even surpasses few-shot methods.
  arXiv  Detail & Related papers  (2024-10-18T05:30:33Z)
• CreDes: Causal Reasoning Enhancement and Dual-End Searching for Solving Long-Range Reasoning Problems using LLMs [13.977459316171013]
  Large language models (LLMs) have demonstrated limitations in handling optimization problems involving long-range reasoning. This paper introduces the Causal Relationship Enhancement (CRE) mechanism combining cause-effect interventions and the Individual Treatment Effect (ITE) to guarantee the solid causal rightness. Experiments demonstrate that CreDes significantly outperforms existing State-Of-The-Art (SOTA) solutions in terms of both accuracy and time efficiency.
  arXiv  Detail & Related papers  (2024-10-02T16:05:01Z)
• Learning From Correctness Without Prompting Makes LLM Efficient Reasoner [30.203952806009717]
  Large language models (LLMs) have demonstrated outstanding performance across various tasks, yet they still exhibit limitations such as hallucination, unfaithful reasoning, and toxic content. We introduce an intrinsic self-correct reasoning framework for LLMs that eliminates the need for human feedback, external tools, and handcraft prompts.
  arXiv  Detail & Related papers  (2024-03-28T02:12:49Z)
• How Likely Do LLMs with CoT Mimic Human Reasoning? [31.86489714330338]
  Chain-of-thought (CoT) emerges as a promising technique to elicit reasoning capabilities from Large Language Models (LLMs) In this paper, we diagnose the underlying mechanism by comparing the reasoning process of LLMs with humans. Our empirical study reveals that LLMs often deviate from a causal chain, resulting in spurious correlations and potential consistency errors.
  arXiv  Detail & Related papers  (2024-02-25T10:13:04Z)
• Learning Planning-based Reasoning by Trajectories Collection and Process Reward Synthesizing [61.98556945939045]
  We propose a framework to learn planning-based reasoning through Direct Preference Optimization (DPO) on collected trajectories. Our results on challenging logical reasoning benchmarks demonstrate the effectiveness of our learning framework.
  arXiv  Detail & Related papers  (2024-02-01T15:18:33Z)
• From Heuristic to Analytic: Cognitively Motivated Strategies for Coherent Physical Commonsense Reasoning [66.98861219674039]
  Heuristic-Analytic Reasoning (HAR) strategies drastically improve the coherence of rationalizations for model decisions. Our findings suggest that human-like reasoning strategies can effectively improve the coherence and reliability of PLM reasoning.
  arXiv  Detail & Related papers  (2023-10-24T19:46:04Z)
• Concise and Organized Perception Facilitates Reasoning in Large Language Models [32.71672086718057]
  We show that large language models (LLMs) exhibit failure patterns akin to human-like cognitive biases when dealing with disordered and irrelevant content in reasoning tasks. We propose a novel reasoning approach named Concise and Organized Perception (COP) COP carefully analyzes the given statements to identify the most pertinent information while eliminating redundancy efficiently.
  arXiv  Detail & Related papers  (2023-10-05T04:47:49Z)
• Towards CausalGPT: A Multi-Agent Approach for Faithful Knowledge Reasoning via Promoting Causal Consistency in LLMs [60.244412212130264]
  Causal-Consistency Chain-of-Thought harnesses multi-agent collaboration to bolster the faithfulness and causality of foundation models. Our framework demonstrates significant superiority over state-of-the-art methods through extensive and comprehensive evaluations.
  arXiv  Detail & Related papers  (2023-08-23T04:59:21Z)
• Question Decomposition Improves the Faithfulness of Model-Generated Reasoning [23.34325378824462]
  Large language models (LLMs) are difficult to verify the correctness and safety of their behavior. One approach is to prompt LLMs to externalize their reasoning, by having them generate step-by-step reasoning as they answer a question. This approach relies on the stated reasoning faithfully reflecting the model's actual reasoning, which is not always the case. Decomposition-based methods achieve strong performance on question-answering tasks, sometimes approaching that of CoT.
  arXiv  Detail & Related papers  (2023-07-17T00:54:10Z)
• Enhancing Chain-of-Thoughts Prompting with Iterative Bootstrapping in Large Language Models [81.01397924280612]
  Large language models (LLMs) can achieve highly effective performance on various reasoning tasks by incorporating step-by-step chain-of-thought (CoT) prompting as demonstrations. We introduce Iter-CoT (Iterative bootstrapping in Chain-of-Thoughts Prompting), an iterative bootstrapping approach for selecting exemplars and generating reasoning chains.
  arXiv  Detail & Related papers  (2023-04-23T13:54:39Z)
• Towards Understanding Chain-of-Thought Prompting: An Empirical Study of What Matters [82.84696222087396]
  Chain-of-Thought (CoT) prompting can dramatically improve the multi-step reasoning abilities of large language models (LLMs) We show that CoT reasoning is possible even with invalid demonstrations.
  arXiv  Detail & Related papers  (2022-12-20T05:20:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.