Heterogeneous Swarms: Jointly Optimizing Model Roles and Weights for Multi-LLM Systems

• URL: http://arxiv.org/abs/2502.04510v1
• Date: Thu, 06 Feb 2025 21:27:11 GMT
• Title: Heterogeneous Swarms: Jointly Optimizing Model Roles and Weights for Multi-LLM Systems
• Authors: Shangbin Feng, Zifeng Wang, Palash Goyal, Yike Wang, Weijia Shi, Huang Xia, Hamid Palangi, Luke Zettlemoyer, Yulia Tsvetkov, Chen-Yu Lee, Tomas Pfister,
• Abstract summary: We propose Heterogeneous Swarms, an algorithm to design multi-LLM systems by jointly optimizing model roles and weights. Experiments demonstrate that Heterogeneous Swarms outperforms 15 role- and/or weight-based baselines by 18.5% on average across 12 tasks.
• Score: 102.36545569092777
• License:
• Abstract: We propose Heterogeneous Swarms, an algorithm to design multi-LLM systems by jointly optimizing model roles and weights. We represent multi-LLM systems as directed acyclic graphs (DAGs) of LLMs with topological message passing for collaborative generation. Given a pool of LLM experts and a utility function, Heterogeneous Swarms employs two iterative steps: role-step and weight-step. For role-step, we interpret model roles as learning a DAG that specifies the flow of inputs and outputs between LLMs. Starting from a swarm of random continuous adjacency matrices, we decode them into discrete DAGs, call the LLMs in topological order, evaluate on the utility function (e.g. accuracy on a task), and optimize the adjacency matrices with particle swarm optimization based on the utility score. For weight-step, we assess the contribution of individual LLMs in the multi-LLM systems and optimize model weights with swarm intelligence. We propose JFK-score to quantify the individual contribution of each LLM in the best-found DAG of the role-step, then optimize model weights with particle swarm optimization based on the JFK-score. Experiments demonstrate that Heterogeneous Swarms outperforms 15 role- and/or weight-based baselines by 18.5% on average across 12 tasks. Further analysis reveals that Heterogeneous Swarms discovers multi-LLM systems with heterogeneous model roles and substantial collaborative gains, and benefits from the diversity of language models.

Related papers

• MALT: Improving Reasoning with Multi-Agent LLM Training [64.13803241218886]
  We present a first step toward "Multi-agent LLM training" (MALT) on reasoning problems. Our approach employs a sequential multi-agent setup with heterogeneous LLMs assigned specialized roles. We evaluate our approach across MATH, GSM8k, and CQA, where MALT on Llama 3.1 8B models achieves relative improvements of 14.14%, 7.12%, and 9.40% respectively.
  arXiv  Detail & Related papers  (2024-12-02T19:30:36Z)
• LLM-based Optimization of Compound AI Systems: A Survey [64.39860384538338]
  In a compound AI system, components such as an LLM call, a retriever, a code interpreter, or tools are interconnected. Recent advancements enable end-to-end optimization of these parameters using an LLM. This paper presents a survey of the principles and emerging trends in LLM-based optimization of compound AI systems.
  arXiv  Detail & Related papers  (2024-10-21T18:06:25Z)
• Model Swarms: Collaborative Search to Adapt LLM Experts via Swarm Intelligence [90.91152752062546]
  We propose Model Swarms, a collaborative search algorithm to adapt LLMs via swarm intelligence. We show that Model Swarms could flexibly adapt LLM experts to a single task, multi-task domains, reward models, as well as diverse human interests.
  arXiv  Detail & Related papers  (2024-10-15T00:59:17Z)
• Model-GLUE: Democratized LLM Scaling for A Large Model Zoo in the Wild [84.57103623507082]
  This paper introduces Model-GLUE, a holistic Large Language Models scaling guideline. We benchmark existing scaling techniques, especially selective merging, and variants of mixture. We then formulate an optimal strategy for the selection and aggregation of a heterogeneous model zoo. Our methodology involves the clustering of mergeable models and optimal merging strategy selection, and the integration of clusters.
  arXiv  Detail & Related papers  (2024-10-07T15:55:55Z)
• SelectLLM: Query-Aware Efficient Selection Algorithm for Large Language Models [8.558834738072363]
  Large language models (LLMs) have seen widespread adoption due to their remarkable performance across various applications. These individual LLMs show limitations in generalization and performance on complex tasks due to inherent training biases, model size constraints, and the quality or diversity of pre-training datasets. We introduce SelectLLM, which efficiently directs input queries to the most suitable subset of LLMs from a large pool.
  arXiv  Detail & Related papers  (2024-08-16T06:11:21Z)
• Ensemble Learning for Heterogeneous Large Language Models with Deep Parallel Collaboration [39.35476224845088]
  Large language models (LLMs) exhibit complementary strengths in various tasks, motivating the research of LLM ensembling. We propose a training-free ensemble framework DeePEn, fusing the informative probability distributions yielded by different LLMs at each decoding step.
  arXiv  Detail & Related papers  (2024-04-19T08:52:22Z)
• LLM-PQ: Serving LLM on Heterogeneous Clusters with Phase-Aware Partition and Adaptive Quantization [9.517540904818986]
  This paper proposes adaptive model quantization and phase-aware partition to improve LLM serving efficiency on heterogeneous GPU clusters. Experiments on production inference workloads in 11 different clusters demonstrate that LLM-PQ achieves up to 2.88x (2.26x on average) throughput improvement in inference.
  arXiv  Detail & Related papers  (2024-03-02T08:40:07Z)
• Efficient Model-Based Multi-Agent Mean-Field Reinforcement Learning [89.31889875864599]
  We propose an efficient model-based reinforcement learning algorithm for learning in multi-agent systems. Our main theoretical contributions are the first general regret bounds for model-based reinforcement learning for MFC. We provide a practical parametrization of the core optimization problem.
  arXiv  Detail & Related papers  (2021-07-08T18:01:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.