Enhancing Language Model Rationality with Bi-Directional Deliberation Reasoning

• URL: http://arxiv.org/abs/2407.06112v1
• Date: Mon, 8 Jul 2024 16:48:48 GMT
• Title: Enhancing Language Model Rationality with Bi-Directional Deliberation Reasoning
• Authors: Yadong Zhang, Shaoguang Mao, Wenshan Wu, Yan Xia, Tao Ge, Man Lan, Furu Wei,
• Abstract summary: This paper introduces BI-Directional DEliberation Reasoning (BIDDER) to enhance the decision rationality of language models. Our approach involves three key processes:. Inferring hidden states to represent uncertain information in the decision-making process from historical data;. Using hidden states to predict future potential states and potential outcomes;. Integrating historical information (past contexts) and long-term outcomes (future contexts) to inform reasoning.
• Score: 73.77288647011295
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: This paper introduces BI-Directional DEliberation Reasoning (BIDDER), a novel reasoning approach to enhance the decision rationality of language models. Traditional reasoning methods typically rely on historical information and employ uni-directional (left-to-right) reasoning strategy. This lack of bi-directional deliberation reasoning results in limited awareness of potential future outcomes and insufficient integration of historical context, leading to suboptimal decisions. BIDDER addresses this gap by incorporating principles of rational decision-making, specifically managing uncertainty and predicting expected utility. Our approach involves three key processes: Inferring hidden states to represent uncertain information in the decision-making process from historical data; Using these hidden states to predict future potential states and potential outcomes; Integrating historical information (past contexts) and long-term outcomes (future contexts) to inform reasoning. By leveraging bi-directional reasoning, BIDDER ensures thorough exploration of both past and future contexts, leading to more informed and rational decisions. We tested BIDDER's effectiveness in two well-defined scenarios: Poker (Limit Texas Hold'em) and Negotiation. Our experiments demonstrate that BIDDER significantly improves the decision-making capabilities of LLMs and LLM agents.

Related papers

• Making Large Language Models Better Planners with Reasoning-Decision Alignment [70.5381163219608]
  We motivate an end-to-end decision-making model based on multimodality-augmented LLM. We propose a reasoning-decision alignment constraint between the paired CoTs and planning results. We dub our proposed large language planners with reasoning-decision alignment as RDA-Driver.
  arXiv  Detail & Related papers  (2024-08-25T16:43:47Z)
• PresAIse, A Prescriptive AI Solution for Enterprises [6.523929486550928]
  This paper outlines an initiative from IBM Research, aiming to address some of these challenges by offering a suite of prescriptive AI solutions. The solution suite includes scalable causal inference methods, interpretable decision-making approaches, and the integration of large language models. A proof-of-concept, PresAIse, demonstrates the solutions' potential by enabling non-ML experts to interact with prescriptive AI models via a natural language interface.
  arXiv  Detail & Related papers  (2024-02-03T03:23:08Z)
• DetermLR: Augmenting LLM-based Logical Reasoning from Indeterminacy to Determinacy [76.58614128865652]
  We propose DetermLR, a novel perspective that rethinks the reasoning process as an evolution from indeterminacy to determinacy. First, we categorize known conditions into two types: determinate and indeterminate premises This provides an oveall direction for the reasoning process and guides LLMs in converting indeterminate data into progressively determinate insights. We automate the storage and extraction of available premises and reasoning paths with reasoning memory, preserving historical reasoning details for subsequent reasoning steps.
  arXiv  Detail & Related papers  (2023-10-28T10:05:51Z)
• From Heuristic to Analytic: Cognitively Motivated Strategies for Coherent Physical Commonsense Reasoning [66.98861219674039]
  Heuristic-Analytic Reasoning (HAR) strategies drastically improve the coherence of rationalizations for model decisions. Our findings suggest that human-like reasoning strategies can effectively improve the coherence and reliability of PLM reasoning.
  arXiv  Detail & Related papers  (2023-10-24T19:46:04Z)
• Bridging the Gap between Decision and Logits in Decision-based Knowledge Distillation for Pre-trained Language Models [16.115386424278213]
  We propose a novel method to estimate logits from the decision distributions. Our method significantly outperforms strong baselines on both natural language understanding and machine reading comprehension datasets.
  arXiv  Detail & Related papers  (2023-06-15T07:23:44Z)
• Explainability's Gain is Optimality's Loss? -- How Explanations Bias Decision-making [0.0]
  Explanations help to facilitate communication between the algorithm and the human decision-maker. Feature-based explanations' semantics of causal models induce leakage from the decision-maker's prior beliefs. Such differences can lead to sub-optimal and biased decision outcomes.
  arXiv  Detail & Related papers  (2022-06-17T11:43:42Z)
• AR-LSAT: Investigating Analytical Reasoning of Text [57.1542673852013]
  We study the challenge of analytical reasoning of text and introduce a new dataset consisting of questions from the Law School Admission Test from 1991 to 2016. We analyze what knowledge understanding and reasoning abilities are required to do well on this task.
  arXiv  Detail & Related papers  (2021-04-14T02:53:32Z)
• On a notion of independence proposed by Teddy Seidenfeld [2.5585152083052574]
  Teddy Seidenfeld has been arguing for quite a long time that binary models are not powerful enough to deal with crucial aspects of imprecision and indeterminacy. We use this approach here to explore an interesting notion of irrelevance (and independence) We show that the consequences of making such an irrelevance or independence assessment are very strong.
  arXiv  Detail & Related papers  (2021-02-20T13:15:28Z)
• Leveraging Expert Consistency to Improve Algorithmic Decision Support [62.61153549123407]
  We explore the use of historical expert decisions as a rich source of information that can be combined with observed outcomes to narrow the construct gap. We propose an influence function-based methodology to estimate expert consistency indirectly when each case in the data is assessed by a single expert. Our empirical evaluation, using simulations in a clinical setting and real-world data from the child welfare domain, indicates that the proposed approach successfully narrows the construct gap.
  arXiv  Detail & Related papers  (2021-01-24T05:40:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.