Large Language Models as Particle Swarm Optimizers

• URL: http://arxiv.org/abs/2504.09247v1
• Date: Sat, 12 Apr 2025 15:04:13 GMT
• Title: Large Language Models as Particle Swarm Optimizers
• Authors: Yamato Shinohara, Jinglue Xu, Tianshui Li, Hitoshi Iba,
• Abstract summary: In LMPSO, the velocity of each particle is represented as a prompt that generates the next candidate solution.<n>The proposed LMPSO approach is evaluated across multiple problem domains, including the Traveling Salesman Problem (TSP)<n> Experimental results demonstrate that LMPSO is particularly effective for solving problems where solutions are represented as structured sequences.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Optimization problems often require domain-specific expertise to design problem-dependent methodologies. Recently, several approaches have gained attention by integrating large language models (LLMs) into genetic algorithms. Building on this trend, we introduce Language Model Particle Swarm Optimization (LMPSO), a novel method that incorporates an LLM into the swarm intelligence framework of Particle Swarm Optimization (PSO). In LMPSO, the velocity of each particle is represented as a prompt that generates the next candidate solution, leveraging the capabilities of an LLM to produce solutions in accordance with the PSO paradigm. This integration enables an LLM-driven search process that adheres to the foundational principles of PSO. The proposed LMPSO approach is evaluated across multiple problem domains, including the Traveling Salesman Problem (TSP), heuristic improvement for TSP, and symbolic regression. These problems are traditionally challenging for standard PSO due to the structured nature of their solutions. Experimental results demonstrate that LMPSO is particularly effective for solving problems where solutions are represented as structured sequences, such as mathematical expressions or programmatic constructs. By incorporating LLMs into the PSO framework, LMPSO establishes a new direction in swarm intelligence research. This method not only broadens the applicability of PSO to previously intractable problems but also showcases the potential of LLMs in addressing complex optimization challenges.

Related papers

• Exploiting Edited Large Language Models as General Scientific Optimizers [10.489095729605152]
  Large language models (LLMs) have been widely adopted in mathematical optimization in scientific scenarios.<n>We propose a conceptually simple and general bi-level optimization method, namely textbfGeneral textbfScientific textbfOptimizers (GSO)
  arXiv  Detail & Related papers  (2025-03-08T18:01:11Z)
• Are Language Models Up to Sequential Optimization Problems? From Evaluation to a Hegelian-Inspired Enhancement [0.0]
  Large Language Models (LLMs) have demonstrated impressive capabilities across numerous fields.<n>This paper explores the proficiency of LLMs in handling Sequential Optimization Problems (SOPs)<n>We introduce WorldGen, a dynamic framework for generating unseen SOPs with controllable complexities.<n>Inspired by the influential framework of Hegelian Dialectics, we propose ACE, demonstrating how the performance of LLMs in SOP contexts can be significantly improved.
  arXiv  Detail & Related papers  (2025-02-04T18:47:31Z)
• Large Language Models for Combinatorial Optimization of Design Structure Matrix [4.513609458468522]
  Combinatorial optimization (CO) is essential for improving efficiency and performance in engineering applications. When it comes to real-world engineering problems, algorithms based on pure mathematical reasoning are limited and incapable to capture the contextual nuances necessary for optimization. This study explores the potential of Large Language Models (LLMs) in solving engineering CO problems by leveraging their reasoning power and contextual knowledge.
  arXiv  Detail & Related papers  (2024-11-19T15:39:51Z)
• EVOLvE: Evaluating and Optimizing LLMs For Exploration [76.66831821738927]
  Large language models (LLMs) remain under-studied in scenarios requiring optimal decision-making under uncertainty. We measure LLMs' (in)ability to make optimal decisions in bandits, a state-less reinforcement learning setting relevant to many applications. Motivated by the existence of optimal exploration algorithms, we propose efficient ways to integrate this algorithmic knowledge into LLMs.
  arXiv  Detail & Related papers  (2024-10-08T17:54:03Z)
• LLaMA-Berry: Pairwise Optimization for O1-like Olympiad-Level Mathematical Reasoning [56.273799410256075]
  The framework combines Monte Carlo Tree Search (MCTS) with iterative Self-Refine to optimize the reasoning path. The framework has been tested on general and advanced benchmarks, showing superior performance in terms of search efficiency and problem-solving capability.
  arXiv  Detail & Related papers  (2024-10-03T18:12:29Z)
• Leveraging Large Language Models for Solving Rare MIP Challenges [35.38992171089948]
  Mixed Programming (MIP) has been extensively applied in areas requiring mathematical solvers to address complex instances within tight time constraints. The model-building cost for end-to-end models, such as large language models (LLMs), remains largely unaffected by problem scale due to their pattern recognition capabilities.
  arXiv  Detail & Related papers  (2024-09-03T07:25:01Z)
• On the Design and Analysis of LLM-Based Algorithms [74.7126776018275]
  Large language models (LLMs) are used as sub-routines in algorithms. LLMs have achieved remarkable empirical success. Our proposed framework holds promise for advancing LLM-based algorithms.
  arXiv  Detail & Related papers  (2024-07-20T07:39:07Z)
• When Large Language Model Meets Optimization [7.822833805991351]
  Large language models (LLMs) facilitate intelligent modeling and strategic decision-making in optimization. This review outlines the progress and potential of combining LLMs with optimization algorithms.
  arXiv  Detail & Related papers  (2024-05-16T13:54:37Z)
• Entropy-Regularized Token-Level Policy Optimization for Language Agent Reinforcement [67.1393112206885]
  Large Language Models (LLMs) have shown promise as intelligent agents in interactive decision-making tasks. We introduce Entropy-Regularized Token-level Policy Optimization (ETPO), an entropy-augmented RL method tailored for optimizing LLMs at the token level. We assess the effectiveness of ETPO within a simulated environment that models data science code generation as a series of multi-step interactive tasks.
  arXiv  Detail & Related papers  (2024-02-09T07:45:26Z)
• Optimizing Solution-Samplers for Combinatorial Problems: The Landscape of Policy-Gradient Methods [52.0617030129699]
  We introduce a novel theoretical framework for analyzing the effectiveness of DeepMatching Networks and Reinforcement Learning methods. Our main contribution holds for a broad class of problems including Max-and Min-Cut, Max-$k$-Bipartite-Bi, Maximum-Weight-Bipartite-Bi, and Traveling Salesman Problem. As a byproduct of our analysis we introduce a novel regularization process over vanilla descent and provide theoretical and experimental evidence that it helps address vanishing-gradient issues and escape bad stationary points.
  arXiv  Detail & Related papers  (2023-10-08T23:39:38Z)
• Thought Propagation: An Analogical Approach to Complex Reasoning with Large Language Models [62.96551299003463]
  We propose textbftextitThought Propagation (TP) to enhance the complex reasoning ability of Large Language Models. TP first prompts LLMs to propose and solve a set of analogous problems that are related to the input one. TP reuses the results of analogous problems to directly yield a new solution or derive a knowledge-intensive plan for execution to amend the initial solution obtained from scratch.
  arXiv  Detail & Related papers  (2023-10-06T01:40:09Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.