Let's Have a Conversation: Designing and Evaluating LLM Agents for Interactive Optimization

• URL: http://arxiv.org/abs/2604.02666v1
• Date: Fri, 03 Apr 2026 02:55:24 GMT
• Title: Let's Have a Conversation: Designing and Evaluating LLM Agents for Interactive Optimization
• Abstract summary: This paper proposes a scalable and replicable methodology for evaluating optimization agents through conversations.<n>We generate thousands of conversations in a school scheduling case study.<n>Results show that one-shot evaluation is severely limiting: the same optimization agent converges to much higher-quality solutions through conversations.
• Score: 42.990074666128145
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Optimization is as much about modeling the right problem as solving it. Identifying the right objectives, constraints, and trade-offs demands extensive interaction between researchers and stakeholders. Large language models can empower decision-makers with optimization capabilities through interactive optimization agents that can propose, interpret and refine solutions. However, it is fundamentally harder to evaluate a conversation-based interaction than traditional one-shot approaches. This paper proposes a scalable and replicable methodology for evaluating optimization agents through conversations. We build LLM-powered decision agents that role-play diverse stakeholders, each governed by an internal utility function but communicating like a real decision-maker. We generate thousands of conversations in a school scheduling case study. Results show that one-shot evaluation is severely limiting: the same optimization agent converges to much higher-quality solutions through conversations. Then, this paper uses this methodology to demonstrate that tailored optimization agents, endowed with domain-specific prompts and structured tools, can lead to significant improvements in solution quality in fewer interactions, as compared to general-purpose chatbots. These findings provide evidence of the benefits of emerging solutions at the AI-optimization interface to expand the reach of optimization technologies in practice. They also uncover the impact of operations research expertise to facilitate interactive deployments through the design of effective and reliable optimization agents.

Related papers

• Large Language Models as Optimizers: A Survey of Direct vs. Tool-Augmented Approaches and Their Performance Frontiers [0.0]
  Large Language Models (LLMs) are increasingly involved in complex mathematical optimization.<n>This paper argues for trade-offs between the future potential of direct optimization and the auditability of tool-augmented optimization.
  arXiv  Detail & Related papers  (2026-04-09T17:35:31Z)
• Agentic Conversational Search with Contextualized Reasoning via Reinforcement Learning [66.52010873968383]
  We introduce a conversational agent that interleaves search and reasoning across turns, enabling exploratory and adaptive behaviors learned through reinforcement learning (RL) training.<n>The experimental results across four widely used conversational benchmarks demonstrate the effectiveness of our methods.
  arXiv  Detail & Related papers  (2026-01-19T14:55:54Z)
• ORMind: A Cognitive-Inspired End-to-End Reasoning Framework for Operations Research [56.961539386979354]
  We introduce ORMind, a cognitive-inspired framework that enhances optimization through counterfactual reasoning.<n>Our approach emulates human cognition, implementing an end-to-end workflow that transforms requirements into mathematical models and executable code.<n>It is currently being tested internally in Lenovo's AI Assistant, with plans to enhance optimization capabilities for both business and consumer customers.
  arXiv  Detail & Related papers  (2025-06-02T05:11:21Z)
• A Survey on the Optimization of Large Language Model-based Agents [16.733092886211097]
  Large Language Models (LLMs) have been widely adopted in various fields, becoming essential for autonomous decision-making and interactive tasks.<n>However, current work typically relies on prompt design or fine-tuning strategies applied to vanilla LLMs.<n>We provide a comprehensive review of LLM-based agent optimization approaches, categorizing them into parameter-driven and parameter-free methods.
  arXiv  Detail & Related papers  (2025-03-16T10:09:10Z)
• Optima: Optimizing Effectiveness and Efficiency for LLM-Based Multi-Agent System [75.25394449773052]
  Large Language Model (LLM) based multi-agent systems (MAS) show remarkable potential in collaborative problem-solving.<n>Yet they still face critical challenges: low communication efficiency, poor scalability, and a lack of effective parameter-updating optimization methods.<n>We present Optima, a novel framework that addresses these issues by significantly enhancing both communication efficiency and task effectiveness.
  arXiv  Detail & Related papers  (2024-10-10T17:00:06Z)
• Learning Joint Models of Prediction and Optimization [56.04498536842065]
  Predict-Then-Then framework uses machine learning models to predict unknown parameters of an optimization problem from features before solving. This paper proposes an alternative method, in which optimal solutions are learned directly from the observable features by joint predictive models.
  arXiv  Detail & Related papers  (2024-09-07T19:52:14Z)
• Enhancing Decision-Making in Optimization through LLM-Assisted Inference: A Neural Networks Perspective [1.0420394952839245]
  This paper explores the seamless integration of Generative AI (GenAI) and Evolutionary Algorithms (EAs) Focusing on the transformative role of Large Language Models (LLMs), our study investigates the potential of LLM-Assisted Inference to automate and enhance decision-making processes.
  arXiv  Detail & Related papers  (2024-05-12T08:22:53Z)
• From Large Language Models and Optimization to Decision Optimization CoPilot: A Research Manifesto [2.4981381729038743]
  We propose research at the intersection of Large Language Models and optimization to create a Decision Optimization CoPilot (DOCP) DOCP is an AI tool designed to assist any decision maker, interacting in natural language to grasp the business problem, subsequently formulating and solving the corresponding optimization model. We show that a) LLMs already provide substantial novel capabilities relevant to a DOCP, and b. major research challenges remain to be addressed.
  arXiv  Detail & Related papers  (2024-02-26T03:10:11Z)
• Benchmarking PtO and PnO Methods in the Predictive Combinatorial Optimization Regime [59.27851754647913]
  Predictive optimization is the precise modeling of many real-world applications, including energy cost-aware scheduling and budget allocation on advertising. We develop a modular framework to benchmark 11 existing PtO/PnO methods on 8 problems, including a new industrial dataset for advertising. Our study shows that PnO approaches are better than PtO on 7 out of 8 benchmarks, but there is no silver bullet found for the specific design choices of PnO.
  arXiv  Detail & Related papers  (2023-11-13T13:19:34Z)
• AgentCF: Collaborative Learning with Autonomous Language Agents for Recommender Systems [112.76941157194544]
  We propose AgentCF for simulating user-item interactions in recommender systems through agent-based collaborative filtering. We creatively consider not only users but also items as agents, and develop a collaborative learning approach that optimize both kinds of agents together. Overall, the optimized agents exhibit diverse interaction behaviors within our framework, including user-item, user-user, item-item, and collective interactions.
  arXiv  Detail & Related papers  (2023-10-13T16:37:14Z)
• Introducing Interactions in Multi-Objective Optimization of Software Architectures [3.483484684588142]
  This study investigates the impact of designer interactions on software architecture optimization.<n>By directing the search towards regions of interest, the interaction uncovers architectures that remain unexplored in the fully automated process.
  arXiv  Detail & Related papers  (2023-08-29T07:49:46Z)
• An Interactive Knowledge-based Multi-objective Evolutionary Algorithm Framework for Practical Optimization Problems [5.387300498478744]
  This paper proposes an interactive knowledge-based evolutionary multi-objective optimization (IK-EMO) framework. It extracts hidden variable-wise relationships as knowledge from evolving high-performing solutions, shares them with users to receive feedback, and applies them back to the optimization process to improve its effectiveness. The working of the proposed IK-EMO is demonstrated on three large-scale real-world engineering design problems.
  arXiv  Detail & Related papers  (2022-09-18T16:51:01Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.