Multi-Objective Bayesian Optimization for Model Merging

• URL: http://arxiv.org/abs/2608.14264v1
• Date: Fri, 14 Aug 2026 12:41:37 GMT
• Title: Multi-Objective Bayesian Optimization for Model Merging
• Abstract summary: MOBO-Merge is a merge-operator framework that uses multi-objective Bayesian optimization to approximate the front under a limited evaluation budget.<n>We evaluate Qwen3-4B and Llama-3.1-8B in two-model instruction-math and three-model instruction-math-code settings.
• Score: 15.977415957314705
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Model merging combines trained models directly in weight space, offering a compute-efficient alternative to additional fine-tuning. Selecting merge parameters is nevertheless difficult because downstream evaluations are expensive, gradients are unavailable, and source capabilities can conflict. We formulate merge-parameter selection as a black-box multi-objective optimization problem and introduce MOBO-Merge, a merge-operator agnostic framework that uses multi-objective Bayesian optimization to approximate the Pareto front under a limited evaluation budget. We evaluate Qwen3-4B and Llama-3.1-8B in two-model instruction-math and three-model instruction-math-code settings using Linear, SLERP, TIES, and block-wise merge operators. On held-out benchmark partitions, MOBO-Merge obtains higher mean hypervolume than random search in 11 of 12 reported comparisons. The gain is small for one-dimensional Linear interpolation but substantially larger for several TIES, block-wise, and three-objective searches. No merge operator is uniformly best: TIES leads in three of four family-setting combinations, whereas Block-Linear 4x is strongest for the Llama three-model merge. These results show that multi-objective Bayesian optimization is valuable as a search layer for expressive merge parameterizations.

Related papers

• CoMerge: Conflict-Driven Preference Optimization for Multi-Task Model Merging [59.836482609414816]
  We propose a conflict-driven preference optimization framework for model merging (CoMerge)<n>CoMerge enables the model to mitigate parameter-space conflicts while preserving task-specific capabilities.<n>Experiments show that CoMerge achieves an average normalized performance of 0.9968 on MergeBench.
  arXiv  Detail & Related papers  (2026-09-02T08:21:28Z)
• RIMS: Preference Optimization via Smoothed Multi-pair Aggregation for Small-Scale LLM Retrieval-Augmented Generation [19.76481955873559]
  Small-scale language models (SLMs) are attractive for retrieval-augmented generation (RAG) in resource-constrained settings.<n>We propose RIMS, a three-stage preference optimization framework.<n>We show that our approach outperforms state-of-the-art baselines across multiple SLM backbones.
  arXiv  Detail & Related papers  (2026-07-17T18:24:19Z)
• SimMerge: Learning to Select Merge Operators from Similarity Signals [32.157558993834414]
  Model merging enables multiple large language models (LLMs) to be combined into a single model while preserving performance.<n>We present simmerge, empha predictive merge-selection method that selects the best merge using inexpensive, task-agnostic similarity signals between models.<n>Our results suggest that learning how to merge is a practical route to scalable model composition when checkpoint catalogs are large and evaluation budgets are tight.
  arXiv  Detail & Related papers  (2026-01-14T13:30:00Z)
• Learning to bin: differentiable and Bayesian optimization for multi-dimensional discriminants in high-energy physics [0.0]
  We propose a binning optimization for signal significance directly in multi-dimensional discriminants.<n>We use a Gaussian Mixture Model (GMM) to define flexible bin boundary shapes for multi-class scores, while in one dimension (binary classification) we move bin boundaries directly.<n>We show that, in particular for limited separability of the signal processes, our approach outperforms argmax classification even with optimized binning in the one-dimensional projections.
  arXiv  Detail & Related papers  (2026-01-12T17:40:45Z)
• BAMBO: Construct Ability and Efficiency LLM Pareto Set via Bayesian Adaptive Multi-objective Block-wise Optimization [4.196004665145396]
  BAMBO (Bayesian Adaptive Multi-objective Block-wise Optimization) is a novel framework that automatically constructs the Large Language Models (LLMs)<n>Formulated as a 1D clustering problem, this strategy leverages a dynamic programming approach to optimally balance intra-blockvolume and inter-block information distribution.
  arXiv  Detail & Related papers  (2025-12-10T15:32:56Z)
• Fine, I'll Merge It Myself: A Multi-Fidelity Framework for Automated Model Merging [30.38047100067552]
  Reasoning capabilities represent a critical frontier for large language models.<n>One way to efficiently supplement capabilities with is by model merging.<n>We propose an Automated Model Merging Framework that enables fine-grained exploration of merging strategies.
  arXiv  Detail & Related papers  (2025-02-06T12:47:25Z)
• If You Can't Use Them, Recycle Them: Optimizing Merging at Scale Mitigates Performance Tradeoffs [48.95875673503714]
  We study merging "generalist" models trained on many tasks.<n>Our algorithm tunes the weight of each checkpoint in a linear combination, resulting in an optimal model.<n>Good merges tend to include almost all checkpoints with non-zero weights, indicating that even seemingly bad initial checkpoints can contribute to good final merges.
  arXiv  Detail & Related papers  (2024-12-05T13:12:51Z)
• LLaMA-Berry: Pairwise Optimization for O1-like Olympiad-Level Mathematical Reasoning [56.273799410256075]
  The framework combines Monte Carlo Tree Search (MCTS) with iterative Self-Refine to optimize the reasoning path. The framework has been tested on general and advanced benchmarks, showing superior performance in terms of search efficiency and problem-solving capability.
  arXiv  Detail & Related papers  (2024-10-03T18:12:29Z)
• Decoding-Time Language Model Alignment with Multiple Objectives [116.42095026960598]
  Existing methods primarily focus on optimizing LMs for a single reward function, limiting their adaptability to varied objectives. Here, we propose $textbfmulti-objective decoding (MOD)$, a decoding-time algorithm that outputs the next token from a linear combination of predictions. We show why existing approaches can be sub-optimal even in natural settings and obtain optimality guarantees for our method.
  arXiv  Detail & Related papers  (2024-06-27T02:46:30Z)
• UCB-driven Utility Function Search for Multi-objective Reinforcement Learning [51.00436121587591]
  In Multi-objective Reinforcement Learning (MORL) agents are tasked with optimising decision-making behaviours.<n>We focus on the case of linear utility functions parametrised by weight vectors w.<n>We introduce a method based on Upper Confidence Bound to efficiently search for the most promising weight vectors during different stages of the learning process.
  arXiv  Detail & Related papers  (2024-05-01T09:34:42Z)
• BOtied: Multi-objective Bayesian optimization with tied multivariate ranks [33.414682601242006]
  In this paper, we show a natural connection between non-dominated solutions and the extreme quantile of the joint cumulative distribution function. Motivated by this link, we propose the Pareto-compliant CDF indicator and the associated acquisition function, BOtied. Our experiments on a variety of synthetic and real-world problems demonstrate that BOtied outperforms state-of-the-art MOBO acquisition functions.
  arXiv  Detail & Related papers  (2023-06-01T04:50:06Z)
• Hybrid Trilinear and Bilinear Programming for Aligning Partially Overlapping Point Sets [85.71360365315128]
  In many applications, we need algorithms which can align partially overlapping point sets are invariant to the corresponding corresponding RPM algorithm. We first show that the objective is a cubic bound function. We then utilize the convex envelopes of trilinear and bilinear monomial transformations to derive its lower bound. We next develop a branch-and-bound (BnB) algorithm which only branches over the transformation variables and runs efficiently.
  arXiv  Detail & Related papers  (2021-01-19T04:24:23Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.