DeFine: Decision-Making with Analogical Reasoning over Factor Profiles

• URL: http://arxiv.org/abs/2410.01772v2
• Date: Thu, 17 Jul 2025 17:58:50 GMT
• Title: DeFine: Decision-Making with Analogical Reasoning over Factor Profiles
• Authors: Yebowen Hu, Xiaoyang Wang, Wenlin Yao, Yiming Lu, Daoan Zhang, Hassan Foroosh, Dong Yu, Fei Liu,
• Abstract summary: textscDeFine is a modular framework that constructs probabilistic factor profiles from complex scenarios.<n>It then integrates these profiles with analogical reasoning to guide LLMs in making critical decisions in new situations.<n>This approach is particularly useful in areas such as consulting and financial deliberation, where making decisions under uncertainty is vital.
• Score: 35.9909472797192
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: LLMs are ideal for decision-making thanks to their ability to reason over long contexts. However, challenges arise when processing speech transcripts that describe complex scenarios, as they are verbose and include repetition, hedging, and vagueness. E.g., during a company's earnings call, an executive might project a positive revenue outlook to reassure investors, despite uncertainty regarding future earnings. It is crucial for LLMs to incorporate this uncertainty systematically when making decisions. In this paper, we introduce \textsc{DeFine}, a modular framework that constructs probabilistic factor profiles from complex scenarios. It then integrates these profiles with analogical reasoning, leveraging insights from similar past experiences to guide LLMs in making critical decisions in new situations. Our framework separates the tasks of quantifying uncertainty and incorporating it into LLM decision-making. This approach is particularly useful in areas such as consulting and financial deliberation, where making decisions under uncertainty is vital.

Related papers

• Your AI, Not Your View: The Bias of LLMs in Investment Analysis [55.328782443604986]
  Large Language Models (LLMs) face frequent knowledge conflicts due to discrepancies between pre-trained parametric knowledge and real-time market data.<n>This paper offers the first quantitative analysis of confirmation bias in LLM-based investment analysis.<n>We observe a consistent preference for large-cap stocks and contrarian strategies across most models.
  arXiv  Detail & Related papers  (2025-07-28T16:09:38Z)
• UProp: Investigating the Uncertainty Propagation of LLMs in Multi-Step Agentic Decision-Making [47.64013151246807]
  Large Language Models (LLMs) are integrated into safety-critical applications involving sequential decision-making.<n>Existing LLM Uncertainty Quantification (UQ) methods are primarily designed for single-turn question-answering formats.<n>We introduce a principled, information-theoretic framework that decomposes LLM sequential decision uncertainty into two parts.
  arXiv  Detail & Related papers  (2025-06-20T18:34:04Z)
• Extending Epistemic Uncertainty Beyond Parameters Would Assist in Designing Reliable LLMs [40.7342896954488]
  We advocate for the adoption of a framework that provides a coherent foundation to reason about uncertainty and clarify the reducibility of uncertainty.<n>By supporting active resolution rather than passive avoidance, it opens the door to more reliable, transparent, and broadly applicable LLM systems.
  arXiv  Detail & Related papers  (2025-06-09T05:52:03Z)
• DecisionFlow: Advancing Large Language Model as Principled Decision Maker [48.654276010223384]
  DecisionFlow is a novel decision modeling framework that guides models to reason over structured representations of actions, attributes, and constraints.<n>Rather than predicting answers directly from prompts, DecisionFlow builds a semantically grounded decision space and infers a latent utility function.<n> Empirical results show that DecisionFlow achieves up to 30% accuracy gains over strong prompting baselines.
  arXiv  Detail & Related papers  (2025-05-27T16:23:53Z)
• Distributive Fairness in Large Language Models: Evaluating Alignment with Human Values [13.798198972161657]
  A number of societal problems involve the distribution of resources, where fairness, along with economic efficiency, play a critical role in the desirability of outcomes. This paper examines whether large language models (LLMs) adhere to fundamental fairness concepts and investigate their alignment with human preferences.
  arXiv  Detail & Related papers  (2025-02-01T04:24:47Z)
• STRUX: An LLM for Decision-Making with Structured Explanations [17.518955158367305]
  We introduce a new framework called STRUX, which enhances LLM decision-making by providing structured explanations. STRUX begins by distilling lengthy information into a concise table of key facts. It then employs a series of self-reflection steps to determine which of these facts are pivotal, categorizing them as either favorable or adverse in relation to a specific decision.
  arXiv  Detail & Related papers  (2024-10-16T14:01:22Z)
• Understanding the Relationship between Prompts and Response Uncertainty in Large Language Models [55.332004960574004]
  Large language models (LLMs) are widely used in decision-making, but their reliability, especially in critical tasks like healthcare, is not well-established. This paper investigates how the uncertainty of responses generated by LLMs relates to the information provided in the input prompt. We propose a prompt-response concept model that explains how LLMs generate responses and helps understand the relationship between prompts and response uncertainty.
  arXiv  Detail & Related papers  (2024-07-20T11:19:58Z)
• The Economic Implications of Large Language Model Selection on Earnings and Return on Investment: A Decision Theoretic Model [0.0]
  We use a decision-theoretic approach to compare the financial impact of different language models. The study reveals how the superior accuracy of more expensive models can, under certain conditions, justify a greater investment. This article provides a framework for companies looking to optimize their technology choices.
  arXiv  Detail & Related papers  (2024-05-27T20:08:41Z)
• Argumentative Large Language Models for Explainable and Contestable Decision-Making [13.045050015831903]
  Large language models (LLMs) are a promising candidate for use in decision-making. They are limited by their inability to reliably provide outputs which are explainable and contestable. We introduce argumentative LLMs, a method utilising LLMs to construct argumentation frameworks. We demonstrate the effectiveness of argumentative LLMs experimentally in the decision-making task of claim verification.
  arXiv  Detail & Related papers  (2024-05-03T13:12:28Z)
• Evaluating Interventional Reasoning Capabilities of Large Language Models [58.52919374786108]
  Large language models (LLMs) can estimate causal effects under interventions on different parts of a system. We conduct empirical analyses to evaluate whether LLMs can accurately update their knowledge of a data-generating process in response to an intervention. We create benchmarks that span diverse causal graphs (e.g., confounding, mediation) and variable types, and enable a study of intervention-based reasoning.
  arXiv  Detail & Related papers  (2024-04-08T14:15:56Z)
• FinLlama: Financial Sentiment Classification for Algorithmic Trading Applications [2.2661367844871854]
  Large Language Models (LLMs) can be used in this context, but they are not finance-specific and tend to require significant computational resources. We introduce a novel approach based on the Llama 2 7B foundational model, in order to benefit from its generative nature and comprehensive language manipulation. This is achieved by fine-tuning the Llama2 7B model on a small portion of supervised financial sentiment analysis data.
  arXiv  Detail & Related papers  (2024-03-18T22:11:00Z)
• Benchmarking Large Language Model Volatility [4.660822118740283]
  The impact of non-deterministic outputs from Large Language Models (LLMs) is not well examined for financial text understanding tasks. Through a compelling case study on investing in the US equity market via news sentiment analysis, we uncover substantial variability in sentence-level sentiment classification results. These uncertainties cascade downstream, leading to more significant variations in portfolio construction and return.
  arXiv  Detail & Related papers  (2023-11-26T03:54:03Z)
• Sentiment Analysis through LLM Negotiations [58.67939611291001]
  A standard paradigm for sentiment analysis is to rely on a singular LLM and makes the decision in a single round. This paper introduces a multi-LLM negotiation framework for sentiment analysis.
  arXiv  Detail & Related papers  (2023-11-03T12:35:29Z)
• A Comprehensive Evaluation of Large Language Models on Legal Judgment Prediction [60.70089334782383]
  Large language models (LLMs) have demonstrated great potential for domain-specific applications. Recent disputes over GPT-4's law evaluation raise questions concerning their performance in real-world legal tasks. We design practical baseline solutions based on LLMs and test on the task of legal judgment prediction.
  arXiv  Detail & Related papers  (2023-10-18T07:38:04Z)
• Survey on Factuality in Large Language Models: Knowledge, Retrieval and Domain-Specificity [61.54815512469125]
  This survey addresses the crucial issue of factuality in Large Language Models (LLMs) As LLMs find applications across diverse domains, the reliability and accuracy of their outputs become vital.
  arXiv  Detail & Related papers  (2023-10-11T14:18:03Z)
• Concise and Organized Perception Facilitates Reasoning in Large Language Models [31.238220405009617]
  Exploiting large language models (LLMs) to tackle reasoning has garnered growing attention. It still remains highly challenging to achieve satisfactory results in complex logical problems, characterized by plenty of premises within the context and requiring multi-hop reasoning. In this work, we first examine the mechanism from the perspective of information flow and reveal that LLMs confront difficulties akin to human-like cognitive biases when dealing with disordered and irrelevant content in reasoning tasks.
  arXiv  Detail & Related papers  (2023-10-05T04:47:49Z)
• On solving decision and risk management problems subject to uncertainty [91.3755431537592]
  Uncertainty is a pervasive challenge in decision and risk management. This paper develops a systematic understanding of such strategies, determine their range of application, and develop a framework to better employ them.
  arXiv  Detail & Related papers  (2023-01-18T19:16:23Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.