Can LLMs Be CEOs? Benchmarking Strategic Resource Reallocation with Multi-Role Agent Simulation

• URL: http://arxiv.org/abs/2606.17459v1
• Date: Tue, 16 Jun 2026 03:20:04 GMT
• Title: Can LLMs Be CEOs? Benchmarking Strategic Resource Reallocation with Multi-Role Agent Simulation
• Abstract summary: evaluating the decision-making capabilities of large language models (LLMs) is a growing research priority.<n>We introduce textscCEO-Bench, a benchmark that evaluates LLMs on CEO-level strategic resource reallocation.<n> Experiments across five frontier models on 13 scenarios reveal that all models achieve high structural validity but diverge sharply on strategic calibration.
• Score: 5.488097487012986
• License: http://creativecommons.org/publicdomain/zero/1.0/
• Abstract: Evaluating the decision-making capabilities of large language models (LLMs) is a growing research priority, yet existing benchmarks focus on isolated cognitive tasks such as reasoning, knowledge retrieval, and economic rationality in stylized settings. These evaluations overlook the defining challenge of real executive decision-making: integrating conflicting recommendations from specialized stakeholders under information asymmetry, organizational constraints, and temporal dependencies. We introduce \textsc{CEO-Bench}, a multi-agent benchmark that evaluates LLMs on CEO-level strategic resource reallocation -- the process of redirecting capital across business units in a multi-round, constraint-rich organizational environment. In \textsc{CEO-Bench}, LLM agents receive conflicting advice from four role-conditioned C-suite advisors (CFO, CTO, COO, CMO), each with private signals and distinct priorities, and must synthesize these into a concrete allocation plan evaluated along four dimensions: role integration, conditional boldness, history-sensitive judgment, and plan validity. Experiments across five frontier models on 13 scenarios reveal that all models achieve high structural validity but diverge sharply on strategic calibration -- the hardest capability layer. We identify systematic failure modes including single-advisor capture, conservative default under ambiguity, and historical amnesia, and uncover a structural integration-boldness tradeoff: models that engage more deeply with conflicting perspectives tend to produce less decisive action. These findings delineate the current capability boundary of LLMs as organizational decision-makers and inform the design of future AI-assisted executive systems.

Related papers

• Seeing Is Not Deciding: Can Multimodal LLMs Act as Effective CEOs? [53.05042283845008]
  We introduce C-SUITEBENCH, a controlled multimodal benchmark for executive business decisions.<n>We place nine models in the role of a chief executive officer and evaluate their decision-making ability.<n>We uncover a multimodal integration paradox: adding visual business information degrades constrained resource allocation for all nine models.
  arXiv  Detail & Related papers  (2026-08-06T10:43:33Z)
• Enhancing Decision-Making with Large Language Models through Multi-Agent Fictitious Play [53.56274149236814]
  Multi-Agent Fictitious Play (MAFP) is a novel MAS paradigm that represents stakeholder stances as agents.<n>MAFP iteratively updates each agent's decision by best responding to the empirical mixture of other agents' past decisions.<n>We evaluate MAFP on challenging decision-making tasks that test the capability of deciding strategies for competitive scenarios prior to acting.
  arXiv  Detail & Related papers  (2026-06-17T17:31:06Z)
• Position: agentic AI orchestration should be Bayes-consistent [92.04373485710285]
  This paper argues that coherent decision-making requires Bayesian principles at the orchestration level of the agentic system.<n>It provides examples and design patterns to illustrate how beliefs and utility-aware policies can improve agentic AI orchestration.
  arXiv  Detail & Related papers  (2026-05-01T15:43:43Z)
• Foresight Optimization for Strategic Reasoning in Large Language Models [24.13855510359357]
  We introduce Foresight Policy Optimization (FoPO) to enhance strategic reasoning in large language models (LLMs)<n>FoPO integrates opponent modeling principles into policy optimization, thereby enabling explicit consideration of both self-interest and counterpart influence.<n>Experiments demonstrate that FoPO significantly enhances strategic reasoning across LLMs of varying sizes and origins.
  arXiv  Detail & Related papers  (2026-04-15T07:55:41Z)
• Can LLM Agents Be CFOs? A Benchmark for Resource Allocation in Dynamic Enterprise Environments [39.051947374027435]
  We introduce EnterpriseArena, the first benchmark for evaluating agents on long-horizon enterprise resource allocation.<n>It instantiates CFO-style decision-making in a 132-month enterprise simulator combining firm-level financial data, anonymized business documents, macroeconomic and industry signals, and expert-validated operating rules.<n>Experiments show that only 16% of runs survive the full horizon, and larger models do not reliably outperform smaller ones.
  arXiv  Detail & Related papers  (2026-03-24T18:25:00Z)
• Generative AI in Managerial Decision-Making: Redefining Boundaries through Ambiguity Resolution and Sycophancy Analysis [0.45880283710344055]
  This study compares various models on ambiguity detection, evaluating how a systematic resolution process enhances response quality.<n>Using a novel four-dimensional business ambiguity taxonomy, we conducted a human-in-the-loop experiment across strategic, tactical, and operational scenarios.<n>Results reveal distinct performance capabilities. While models excel in detecting internal contradictions and contextual ambiguities, they struggle with structural linguistic nuances.
  arXiv  Detail & Related papers  (2026-03-04T12:10:56Z)
• Responsible LLM Deployment for High-Stake Decisions by Decentralized Technologies and Human-AI Interactions [1.2464143741310136]
  This paper proposes a framework for responsible deployment of LLM-based decision-support systems through active human involvement.<n>It integrates interactive collaboration between human experts and developers through multiple iterations at the pre-deployment stage.<n>It was tested on Bert-large-uncased, MistLaMA 2 and LLaMA 3 models to assess the capability to support responsible financial decisions on business lending.
  arXiv  Detail & Related papers  (2025-11-28T18:10:52Z)
• Making LLMs Reliable When It Matters Most: A Five-Layer Architecture for High-Stakes Decisions [51.56484100374058]
  Current large language models (LLMs) excel in verifiable domains where outputs can be checked before action but prove less reliable for high-stakes strategic decisions with uncertain outcomes.<n>This gap, driven by mutually cognitive biases in both humans and artificial intelligence (AI) systems, threatens the defensibility of valuations and sustainability of investments in the sector.<n>This report describes a framework emerging from systematic qualitative assessment across 7 frontier-grade LLMs and 3 market-facing venture vignettes under time pressure.
  arXiv  Detail & Related papers  (2025-11-10T22:24:21Z)
• Maestro: Learning to Collaborate via Conditional Listwise Policy Optimization for Multi-Agent LLMs [23.590034731179824]
  We present Through Role Orchestration (Maestro), a principled paradigm for collaboration that structurally decouples cognitive modes.<n>Maestro uses a collective of parallel Execution Agents for diverse exploration and a specialized Central Agent for convergent, evaluative synthesis.<n>Experiments on mathematical reasoning and general problem-solving benchmarks demonstrate that Maestro, coupled with CLPO, consistently outperforms existing state-of-the-art multi-agent approaches.
  arXiv  Detail & Related papers  (2025-11-08T21:01:27Z)
• How Good are Foundation Models in Step-by-Step Embodied Reasoning? [79.15268080287505]
  Embodied agents must make decisions that are safe, spatially coherent, and grounded in context.<n>Recent advances in large multimodal models have shown promising capabilities in visual understanding and language generation.<n>Our benchmark includes over 1.1k samples with detailed step-by-step reasoning across 10 tasks and 8 embodiments.
  arXiv  Detail & Related papers  (2025-09-18T17:56:30Z)
• How Strategic Agents Respond: Comparing Analytical Models with LLM-Generated Responses in Strategic Classification [11.614944245315186]
  We use Strategic Classification to study the interaction between agents and decision-makers.<n>This shift prompts two questions: (i) Can LLMs generate effective and socially responsible strategies in SC settings?<n>We show that even without access to the decision policy, LLMs can generate effective strategies that improve both agents' scores and qualification.
  arXiv  Detail & Related papers  (2025-01-20T01:39:03Z)
• LLM as a Mastermind: A Survey of Strategic Reasoning with Large Language Models [75.89014602596673]
  Strategic reasoning requires understanding and predicting adversary actions in multi-agent settings while adjusting strategies accordingly. We explore the scopes, applications, methodologies, and evaluation metrics related to strategic reasoning with Large Language Models. It underscores the importance of strategic reasoning as a critical cognitive capability and offers insights into future research directions and potential improvements.
  arXiv  Detail & Related papers  (2024-04-01T16:50:54Z)
• K-Level Reasoning: Establishing Higher Order Beliefs in Large Language Models for Strategic Reasoning [76.3114831562989]
  It requires Large Language Model (LLM) agents to adapt their strategies dynamically in multi-agent environments. We propose a novel framework: "K-Level Reasoning with Large Language Models (K-R)"
  arXiv  Detail & Related papers  (2024-02-02T16:07:05Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.