Solving General Natural-Language-Description Optimization Problems with Large Language Models

• URL: http://arxiv.org/abs/2407.07924v1
• Date: Tue, 9 Jul 2024 07:11:10 GMT
• Title: Solving General Natural-Language-Description Optimization Problems with Large Language Models
• Authors: Jihai Zhang, Wei Wang, Siyan Guo, Li Wang, Fangquan Lin, Cheng Yang, Wotao Yin,
• Abstract summary: We propose a novel framework called OptLLM that augments LLMs with external solvers. OptLLM accepts user queries in natural language, convert them into mathematical formulations and programming codes, and calls the solvers to calculate the results. Some features of OptLLM framework have been available for trial since June 2023.
• Score: 34.50671063271608
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Optimization problems seek to find the best solution to an objective under a set of constraints, and have been widely investigated in real-world applications. Modeling and solving optimization problems in a specific domain typically require a combination of domain knowledge, mathematical skills, and programming ability, making it difficult for general users and even domain professionals. In this paper, we propose a novel framework called OptLLM that augments LLMs with external solvers. Specifically, OptLLM accepts user queries in natural language, convert them into mathematical formulations and programming codes, and calls the solvers to calculate the results for decision-making. In addition, OptLLM supports multi-round dialogues to gradually refine the modeling and solving of optimization problems. To illustrate the effectiveness of OptLLM, we provide tutorials on three typical optimization applications and conduct experiments on both prompt-based GPT models and a fine-tuned Qwen model using a large-scale selfdeveloped optimization dataset. Experimental results show that OptLLM works with various LLMs, and the fine-tuned model achieves an accuracy boost compared to the promptbased models. Some features of OptLLM framework have been available for trial since June 2023 (https://opt.alibabacloud.com/chat or https://opt.aliyun.com/chat).

Related papers

• Autoformulation of Mathematical Optimization Models Using LLMs [50.030647274271516]
  We develop an automated approach to creating optimization models from natural language descriptions for commercial solvers. We identify the three core challenges of autoformulation: (1) defining the vast, problem-dependent hypothesis space, (2) efficiently searching this space under uncertainty, and (3) evaluating formulation correctness.
  arXiv  Detail & Related papers  (2024-11-03T20:41:38Z)
• LLMOPT: Learning to Define and Solve General Optimization Problems from Scratch [16.174567164068037]
  We propose a unified learning-based framework called LLMOPT to boost optimization generalization. LLMOPT constructs the introduced five-element formulation as a universal model for learning to define diverse optimization problem types. We evaluate the optimization generalization ability of LLMOPT and compared methods across six real-world datasets.
  arXiv  Detail & Related papers  (2024-10-17T04:37:37Z)
• LLaMA-Berry: Pairwise Optimization for O1-like Olympiad-Level Mathematical Reasoning [56.273799410256075]
  The framework combines Monte Carlo Tree Search (MCTS) with iterative Self-Refine to optimize the reasoning path. The framework has been tested on general and advanced benchmarks, showing superior performance in terms of search efficiency and problem-solving capability.
  arXiv  Detail & Related papers  (2024-10-03T18:12:29Z)
• Learning Joint Models of Prediction and Optimization [56.04498536842065]
  Predict-Then-Then framework uses machine learning models to predict unknown parameters of an optimization problem from features before solving. This paper proposes an alternative method, in which optimal solutions are learned directly from the observable features by joint predictive models.
  arXiv  Detail & Related papers  (2024-09-07T19:52:14Z)
• OptiMUS-0.3: Using Large Language Models to Model and Solve Optimization Problems at Scale [16.33736498565436]
  We introduce a Large Language Model (LLM)-based system designed to formulate and solve linear programming problems from their natural language descriptions. Our system is capable of developing mathematical models, writing and debugning solver code, evaluating the generated solutions, and improving efficiency and correctness of its model and code. Experiments demonstrate that OptiMUS-0.3 outperforms existing state-of-the-art methods on easy datasets by more than 12% and on hard datasets by more than 8%.
  arXiv  Detail & Related papers  (2024-07-29T01:31:45Z)
• OptiBench Meets ReSocratic: Measure and Improve LLMs for Optimization Modeling [62.19438812624467]
  Large language models (LLMs) have exhibited their problem-solving abilities in mathematical reasoning. We propose OptiBench, a benchmark for End-to-end optimization problem-solving with human-readable inputs and outputs.
  arXiv  Detail & Related papers  (2024-07-13T13:27:57Z)
• Large Language Model-Based Evolutionary Optimizer: Reasoning with elitism [1.1463861912335864]
  Large Language Models (LLMs) have demonstrated remarkable reasoning abilities. This paper asserts that LLMs possess the capability for zero-shot optimization across diverse scenarios. We introduce a novel population-based method for numerical optimization using LLMs.
  arXiv  Detail & Related papers  (2024-03-04T13:57:37Z)
• OptiMUS: Scalable Optimization Modeling with (MI)LP Solvers and Large Language Models [21.519880445683107]
  This paper introduces OptiMUS, a Large Language Model (LL)M-based agent designed to formulate and solve (mixed integer) linear programming problems from their natural language descriptions. OptiMUS can develop mathematical models, write and debug solver code, evaluate the generated solutions, and improve its model and code based on these evaluations. Experiments demonstrate that OptiMUS outperforms existing state-of-the-art methods on easy datasets by more than $20%$ and on hard datasets by more than $30%$.
  arXiv  Detail & Related papers  (2024-02-15T18:19:18Z)
• CoLLiE: Collaborative Training of Large Language Models in an Efficient Way [59.09824823710863]
  CoLLiE is an efficient library that facilitates collaborative training of large language models. With its modular design and comprehensive functionality, CoLLiE offers a balanced blend of efficiency, ease of use, and customization.
  arXiv  Detail & Related papers  (2023-12-01T08:02:16Z)
• Large Language Models as Optimizers [106.52386531624532]
  We propose Optimization by PROmpting (OPRO), a simple and effective approach to leverage large language models (LLMs) as prompts. In each optimization step, the LLM generates new solutions from the prompt that contains previously generated solutions with their values. We demonstrate that the best prompts optimized by OPRO outperform human-designed prompts by up to 8% on GSM8K, and by up to 50% on Big-Bench Hard tasks.
  arXiv  Detail & Related papers  (2023-09-07T00:07:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.