Retrieval Augmented Decision-Making: A Requirements-Driven, Multi-Criteria Framework for Structured Decision Support

• URL: http://arxiv.org/abs/2505.18483v1
• Date: Sat, 24 May 2025 03:13:29 GMT
• Title: Retrieval Augmented Decision-Making: A Requirements-Driven, Multi-Criteria Framework for Structured Decision Support
• Abstract summary: This paper proposes the RAD method, which integrates Multi-Criteria Decision Making with the semantic understanding capabilities of LLMs.<n>The method automatically extracts key criteria from industry documents, builds a weighted hierarchical decision model, and generates structured reports under model guidance.<n> Experiments show that in various decision-making tasks, the decision reports generated by RAD significantly outperform existing methods in terms of detail, rationality, and structure.
• Score: 8.585671505840637
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Various industries have produced a large number of documents such as industrial plans, technical guidelines, and regulations that are structurally complex and content-wise fragmented. This poses significant challenges for experts and decision-makers in terms of retrieval and understanding. Although existing LLM-based Retrieval-Augmented Generation methods can provide context-related suggestions, they lack quantitative weighting and traceable reasoning paths, making it difficult to offer multi-level and transparent decision support. To address this issue, this paper proposes the RAD method, which integrates Multi-Criteria Decision Making with the semantic understanding capabilities of LLMs. The method automatically extracts key criteria from industry documents, builds a weighted hierarchical decision model, and generates structured reports under model guidance. The RAD framework introduces explicit weight assignment and reasoning chains in decision generation to ensure accuracy, completeness, and traceability. Experiments show that in various decision-making tasks, the decision reports generated by RAD significantly outperform existing methods in terms of detail, rationality, and structure, demonstrating its application value and potential in complex decision support scenarios.

Related papers

• Integrating the Analytic Hierarchy Process with Large Language Models for Transparent Multi-Criteria Decision-Making [4.86435280701716]
  This study examines the decision-making capabilities of LLMs through the Analytic Hierarchy Process (AHP)<n>We construct a new annotated benchmark based on AHP and propose the first end-to-end approach that enables LLMs to perform the complete AHP workflow.<n> Experiments in real-world decision problems in the legal and higher-education ranking domains show that our method significantly improves alignment with expert judgments.
  arXiv  Detail & Related papers  (2026-09-15T07:47:04Z)
• Uncertainty-Aware Generation and Decision-Making Under Ambiguity [57.15357080072861]
  We present and evaluate various uncertainty-aware decision-making algorithms based on Bayesian decision theory and risk-averse decision making.<n>We find empirically that these algorithms can improve the utility of the generations but need to be carefully implemented when ambiguity is high.
  arXiv  Detail & Related papers  (2026-06-29T17:20:43Z)
• Rethinking Build vs. Buy Decisions in Enterprise Software: Navigating Trade-offs through a Structured Decision-Support Approach [2.4872412262736825]
  This paper presents a structured decision-support approach designed to augment build-versus-buy decision-making.<n>The approach is grounded in an ontology of decision factors spanning strategic considerations, application characteristics, cost and budget constraints, and risk dimensions.<n>The results suggest that making decision criteria explicit and systematically comparable can improve the quality, transparency, and auditability of build-versus-buy decisions in enterprise settings.
  arXiv  Detail & Related papers  (2026-06-29T05:49:40Z)
• Enhancing Decision-Making with Large Language Models through Multi-Agent Fictitious Play [53.56274149236814]
  Multi-Agent Fictitious Play (MAFP) is a novel MAS paradigm that represents stakeholder stances as agents.<n>MAFP iteratively updates each agent's decision by best responding to the empirical mixture of other agents' past decisions.<n>We evaluate MAFP on challenging decision-making tasks that test the capability of deciding strategies for competitive scenarios prior to acting.
  arXiv  Detail & Related papers  (2026-06-17T17:31:06Z)
• Survey of Various Fuzzy and Uncertain Decision-Making Methods [2.2667044928324747]
  Decision-making in real applications is often affected by vagueness, incomplete information, heterogeneous data, and conflicting expert opinions.<n>This survey reviews uncertainty-aware multi-criteria decision-making (MCDM) and organizes the field into a concise, task-oriented taxonomy.
  arXiv  Detail & Related papers  (2026-03-16T13:45:09Z)
• Doc2AHP: Inferring Structured Multi-Criteria Decision Models via Semantic Trees with LLMs [7.026862437055361]
  We propose Doc2AHP, a novel structured inference framework guided by AHP principles.<n>We introduce a multi-agent weighting mechanism coupled with an adaptive consistency optimization strategy to ensure the numerical consistency of weight allocation.<n> Empirical results demonstrate that Doc2AHP not only empowers non-expert users to construct high-quality decision models from scratch but also significantly outperforms direct generative baselines in both logical completeness and downstream task accuracy.
  arXiv  Detail & Related papers  (2026-01-23T06:20:23Z)
• Advancing Adaptive Multi-Stage Video Anomaly Reasoning: A Benchmark Dataset and Method [96.63801368613177]
  We present a new task that elevates video anomaly analysis from descriptive understanding to structured, multi-stage reasoning.<n>We present a new dataset with 8,641 videos, totaling more than 50,000 samples, making it one of the largest datasets for video anomaly understanding.<n>Building upon the proposed task and dataset, we develop an end-to-end MLLM-based VAR model termed Vad-R1-Plus, which supports adaptive hierarchical reasoning and risk-aware decision making.
  arXiv  Detail & Related papers  (2026-01-15T08:09:04Z)
• Increasing AI Explainability by LLM Driven Standard Processes [0.0]
  This paper introduces an approach to increasing the explainability of artificial intelligence (AI) systems by embedding Large Language Models (LLMs) within standardized analytical processes.<n>A layered architecture is presented that separates the reasoning space of the LLM from the explainable process space above it.<n> Empirical evaluations show that the system can reproduce human-level decision logic in decentralized governance, systems analysis, and strategic reasoning contexts.
  arXiv  Detail & Related papers  (2025-11-10T13:16:10Z)
• A Structured Evaluation Framework for Low-Code Platform Selection: A Multi-Criteria Decision Model for Enterprise Digital Transformation [0.0]
  This paper presents a comprehensive evaluation framework based on five key criteria.<n>We propose a weighted scoring model that allows organizations to quantitatively assess and compare different low-code platforms.
  arXiv  Detail & Related papers  (2025-10-21T12:42:11Z)
• How Good are Foundation Models in Step-by-Step Embodied Reasoning? [79.15268080287505]
  Embodied agents must make decisions that are safe, spatially coherent, and grounded in context.<n>Recent advances in large multimodal models have shown promising capabilities in visual understanding and language generation.<n>Our benchmark includes over 1.1k samples with detailed step-by-step reasoning across 10 tasks and 8 embodiments.
  arXiv  Detail & Related papers  (2025-09-18T17:56:30Z)
• A New Approach for Multicriteria Assessment in the Ranking of Alternatives Using Cardinal and Ordinal Data [0.0]
  We propose a novel MCA approach that combines two Virtual Gap Analysis (VGA) models.<n>The VGA framework, rooted in linear programming, is pivotal in the MCA methodology.
  arXiv  Detail & Related papers  (2025-07-10T04:00:48Z)
• MEXA: Towards General Multimodal Reasoning with Dynamic Multi-Expert Aggregation [64.85885900375483]
  MEXA is a training-free framework that performs modality- and task-aware aggregation of expert models.<n>We evaluate our approach on diverse multimodal benchmarks, including Video Reasoning, Audio Reasoning, 3D Understanding, and Medical QA.
  arXiv  Detail & Related papers  (2025-06-20T16:14:13Z)
• DecisionFlow: Advancing Large Language Model as Principled Decision Maker [48.654276010223384]
  DecisionFlow is a novel decision modeling framework that guides models to reason over structured representations of actions, attributes, and constraints.<n>Rather than predicting answers directly from prompts, DecisionFlow builds a semantically grounded decision space and infers a latent utility function.<n> Empirical results show that DecisionFlow achieves up to 30% accuracy gains over strong prompting baselines.
  arXiv  Detail & Related papers  (2025-05-27T16:23:53Z)
• Lost in Models? Structuring Managerial Decision Support in Process Mining with Multi-criteria Decision Making [0.0]
  This paper explores a multi-criteria decision-making (MCDM) approach to evaluate and prioritize process models.<n>Initial insights suggest that the MCDM approach enhances context-sensitive decision-making.
  arXiv  Detail & Related papers  (2025-05-15T12:49:56Z)
• Agentic LLM Framework for Adaptive Decision Discourse [2.4919169815423743]
  This study introduces a real-world inspired agentic Large Language Models (LLMs) framework.<n>Unlike traditional decision-support tools, the framework emphasizes dialogue, trade-off exploration, and the emergent synergies generated by interactions among agents.<n>Results reveal how the breadth-first exploration of alternatives fosters robust and equitable recommendation pathways.
  arXiv  Detail & Related papers  (2025-02-16T03:46:37Z)
• Decision Making in Changing Environments: Robustness, Query-Based Learning, and Differential Privacy [59.64384863882473]
  We study the problem of interactive decision making in which the underlying environment changes over time subject to given constraints.<n>We propose a framework, which provides an complexity between the complexity and adversarial settings of decision making.
  arXiv  Detail & Related papers  (2025-01-24T21:31:50Z)
• Cognitive LLMs: Towards Integrating Cognitive Architectures and Large Language Models for Manufacturing Decision-making [51.737762570776006]
  LLM-ACTR is a novel neuro-symbolic architecture that provides human-aligned and versatile decision-making. Our framework extracts and embeds knowledge of ACT-R's internal decision-making process as latent neural representations. Our experiments on novel Design for Manufacturing tasks show both improved task performance as well as improved grounded decision-making capability.
  arXiv  Detail & Related papers  (2024-08-17T11:49:53Z)
• Multi-step Inference over Unstructured Data [2.169874047093392]
  High-stakes decision-making tasks in fields such as medical, legal and finance require a level of precision, comprehensiveness, and logical consistency. We have developed a neuro-symbolic AI platform to tackle these problems. The platform integrates fine-tuned LLMs for knowledge extraction and alignment with a robust symbolic reasoning engine.
  arXiv  Detail & Related papers  (2024-06-26T00:00:45Z)
• MR-Ben: A Meta-Reasoning Benchmark for Evaluating System-2 Thinking in LLMs [55.20845457594977]
  Large language models (LLMs) have shown increasing capability in problem-solving and decision-making.<n>We present a process-based benchmark MR-Ben that demands a meta-reasoning skill.<n>Our meta-reasoning paradigm is especially suited for system-2 slow thinking.
  arXiv  Detail & Related papers  (2024-06-20T03:50:23Z)
• DeLLMa: Decision Making Under Uncertainty with Large Language Models [31.77731889916652]
  DeLLMa is a framework designed to enhance decision-making accuracy in uncertain environments. We show that DeLLMa can consistently enhance the decision-making performance of leading language models, and achieve up to a 40% increase in accuracy over competing methods.
  arXiv  Detail & Related papers  (2024-02-04T08:11:45Z)
• Solution-oriented Agent-based Models Generation with Verifier-assisted Iterative In-context Learning [10.67134969207797]
  Agent-based models (ABMs) stand as an essential paradigm for proposing and validating hypothetical solutions or policies. Large language models (LLMs) encapsulating cross-domain knowledge and programming proficiency could potentially alleviate the difficulty of this process. We present SAGE, a general solution-oriented ABM generation framework designed for automatic modeling and generating solutions for targeted problems.
  arXiv  Detail & Related papers  (2024-02-04T07:59:06Z)
• On strategies for risk management and decision making under uncertainty shared across multiple fields [55.2480439325792]
  The paper finds more than 110 examples of such strategies and this approach to risk is termed RDOT: Risk-reducing Design and Operations Toolkit.<n>RDOT strategies fall into six broad categories: structural, reactive, formal, adversarial, multi-stage and positive.<n>Overall, RDOT represents an overlooked class of versatile responses to uncertainty.
  arXiv  Detail & Related papers  (2023-09-06T16:14:32Z)
• Rational Decision-Making Agent with Internalized Utility Judgment [91.80700126895927]
  Large language models (LLMs) have demonstrated remarkable advancements and have attracted significant efforts to develop LLMs into agents capable of executing intricate multi-step decision-making tasks beyond traditional NLP applications. This paper proposes RadAgent, which fosters the development of its rationality through an iterative framework involving Experience Exploration and Utility Learning. Experimental results on the ToolBench dataset demonstrate RadAgent's superiority over baselines, achieving over 10% improvement in Pass Rate on diverse tasks.
  arXiv  Detail & Related papers  (2023-08-24T03:11:45Z)
• Leveraging Expert Consistency to Improve Algorithmic Decision Support [62.61153549123407]
  We explore the use of historical expert decisions as a rich source of information that can be combined with observed outcomes to narrow the construct gap. We propose an influence function-based methodology to estimate expert consistency indirectly when each case in the data is assessed by a single expert. Our empirical evaluation, using simulations in a clinical setting and real-world data from the child welfare domain, indicates that the proposed approach successfully narrows the construct gap.
  arXiv  Detail & Related papers  (2021-01-24T05:40:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.