DeFine: Enhancing LLM Decision-Making with Factor Profiles and Analogical Reasoning

• URL: http://arxiv.org/abs/2410.01772v1
• Date: Wed, 2 Oct 2024 17:29:34 GMT
• Title: DeFine: Enhancing LLM Decision-Making with Factor Profiles and Analogical Reasoning
• Authors: Yebowen Hu, Xiaoyang Wang, Wenlin Yao, Yiming Lu, Daoan Zhang, Hassan Foroosh, Dong Yu, Fei Liu,
• Abstract summary: We introduce DeFine, a new framework that constructs probabilistic factor profiles from complex scenarios. DeFine then integrates these profiles with analogical reasoning, leveraging insights from similar past experiences. This approach is particularly useful in fields such as medical consultations, negotiations, and political debates.
• Score: 35.9909472797192
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: LLMs are ideal for decision-making due to their ability to reason over long contexts and identify critical factors. However, challenges arise when processing transcripts of spoken speech describing complex scenarios. These transcripts often contain ungrammatical or incomplete sentences, repetitions, hedging, and vagueness. For example, during a company's earnings call, an executive might project a positive revenue outlook to reassure investors, despite significant uncertainty regarding future earnings. It is crucial for LLMs to incorporate this uncertainty systematically when making decisions. In this paper, we introduce DeFine, a new framework that constructs probabilistic factor profiles from complex scenarios. DeFine then integrates these profiles with analogical reasoning, leveraging insights from similar past experiences to guide LLMs in making critical decisions in novel situations. Our framework separates the tasks of quantifying uncertainty in complex scenarios and incorporating it into LLM decision-making. This approach is particularly useful in fields such as medical consultations, negotiations, and political debates, where making decisions under uncertainty is vital.

Related papers

• STRUX: An LLM for Decision-Making with Structured Explanations [17.518955158367305]
  We introduce a new framework called STRUX, which enhances LLM decision-making by providing structured explanations. STRUX begins by distilling lengthy information into a concise table of key facts. It then employs a series of self-reflection steps to determine which of these facts are pivotal, categorizing them as either favorable or adverse in relation to a specific decision.
  arXiv  Detail & Related papers  (2024-10-16T14:01:22Z)
• Understanding the Relationship between Prompts and Response Uncertainty in Large Language Models [55.332004960574004]
  Large language models (LLMs) are widely used in decision-making, but their reliability, especially in critical tasks like healthcare, is not well-established. This paper investigates how the uncertainty of responses generated by LLMs relates to the information provided in the input prompt. We propose a prompt-response concept model that explains how LLMs generate responses and helps understand the relationship between prompts and response uncertainty.
  arXiv  Detail & Related papers  (2024-07-20T11:19:58Z)
• Argumentative Large Language Models for Explainable and Contestable Decision-Making [13.045050015831903]
  Large language models (LLMs) are a promising candidate for use in decision-making. They are limited by their inability to reliably provide outputs which are explainable and contestable. We introduce argumentative LLMs, a method utilising LLMs to construct argumentation frameworks. We demonstrate the effectiveness of argumentative LLMs experimentally in the decision-making task of claim verification.
  arXiv  Detail & Related papers  (2024-05-03T13:12:28Z)
• Evaluating Interventional Reasoning Capabilities of Large Language Models [58.52919374786108]
  Large language models (LLMs) can estimate causal effects under interventions on different parts of a system. We conduct empirical analyses to evaluate whether LLMs can accurately update their knowledge of a data-generating process in response to an intervention. We create benchmarks that span diverse causal graphs (e.g., confounding, mediation) and variable types, and enable a study of intervention-based reasoning.
  arXiv  Detail & Related papers  (2024-04-08T14:15:56Z)
• FinLlama: Financial Sentiment Classification for Algorithmic Trading Applications [2.2661367844871854]
  Large Language Models (LLMs) can be used in this context, but they are not finance-specific and tend to require significant computational resources. We introduce a novel approach based on the Llama 2 7B foundational model, in order to benefit from its generative nature and comprehensive language manipulation. This is achieved by fine-tuning the Llama2 7B model on a small portion of supervised financial sentiment analysis data.
  arXiv  Detail & Related papers  (2024-03-18T22:11:00Z)
• Benchmarking Large Language Model Volatility [4.660822118740283]
  The impact of non-deterministic outputs from Large Language Models (LLMs) is not well examined for financial text understanding tasks. Through a compelling case study on investing in the US equity market via news sentiment analysis, we uncover substantial variability in sentence-level sentiment classification results. These uncertainties cascade downstream, leading to more significant variations in portfolio construction and return.
  arXiv  Detail & Related papers  (2023-11-26T03:54:03Z)
• Sentiment Analysis through LLM Negotiations [58.67939611291001]
  A standard paradigm for sentiment analysis is to rely on a singular LLM and makes the decision in a single round. This paper introduces a multi-LLM negotiation framework for sentiment analysis.
  arXiv  Detail & Related papers  (2023-11-03T12:35:29Z)
• A Comprehensive Evaluation of Large Language Models on Legal Judgment Prediction [60.70089334782383]
  Large language models (LLMs) have demonstrated great potential for domain-specific applications. Recent disputes over GPT-4's law evaluation raise questions concerning their performance in real-world legal tasks. We design practical baseline solutions based on LLMs and test on the task of legal judgment prediction.
  arXiv  Detail & Related papers  (2023-10-18T07:38:04Z)
• Survey on Factuality in Large Language Models: Knowledge, Retrieval and Domain-Specificity [61.54815512469125]
  This survey addresses the crucial issue of factuality in Large Language Models (LLMs) As LLMs find applications across diverse domains, the reliability and accuracy of their outputs become vital.
  arXiv  Detail & Related papers  (2023-10-11T14:18:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.