Dynamic Collaboration of Multi-Language Models based on Minimal Complete Semantic Units

• URL: http://arxiv.org/abs/2508.18763v1
• Date: Tue, 26 Aug 2025 07:41:33 GMT
• Title: Dynamic Collaboration of Multi-Language Models based on Minimal Complete Semantic Units
• Authors: Chao Hao, Zezheng Wang, Yanhua Huang, Ruiwen Xu, Wenzhe Niu, Xin Liu, Zitong Yu,
• Abstract summary: This paper investigates the enhancement of reasoning capabilities in language models through token-level multi-model collaboration.<n>We introduce a distribution distance-based dynamic selection strategy (DDS) to optimize the multi-model collaboration process.
• Score: 29.79935180749153
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: This paper investigates the enhancement of reasoning capabilities in language models through token-level multi-model collaboration. Our approach selects the optimal tokens from the next token distributions provided by multiple models to perform autoregressive reasoning. Contrary to the assumption that more models yield better results, we introduce a distribution distance-based dynamic selection strategy (DDS) to optimize the multi-model collaboration process. To address the critical challenge of vocabulary misalignment in multi-model collaboration, we propose the concept of minimal complete semantic units (MCSU), which is simple yet enables multiple language models to achieve natural alignment within the linguistic space. Experimental results across various benchmarks demonstrate the superiority of our method. The code will be available at https://github.com/Fanye12/DDS.

Related papers

• Ensembling Language Models with Sequential Monte Carlo [48.149136054981334]
  We introduce a unified framework for composing $K$ language models into $f$-ensemble distributions.<n>We show that better posterior approximations can yield better ensemble performance.
  arXiv  Detail & Related papers  (2026-03-05T17:54:31Z)
• Model Merging to Maintain Language-Only Performance in Developmentally Plausible Multimodal Models [2.3193211674050516]
  This paper presents our approach to the multimodal track of the BabyLM challenge addressing this discrepancy.<n>We develop language-only and multimodal models in low-resource settings using developmentally plausible datasets.
  arXiv  Detail & Related papers  (2025-10-02T09:38:25Z)
• Unifying Multimodal Large Language Model Capabilities and Modalities via Model Merging [103.98582374569789]
  Model merging aims to combine multiple expert models into a single model, thereby reducing storage and serving costs.<n>Previous studies have primarily focused on merging visual classification models or Large Language Models (LLMs) for code and math tasks.<n>We introduce the model merging benchmark for MLLMs, which includes multiple tasks such as VQA, Geometry, Chart, OCR, and Grounding, providing both LoRA and full fine-tuning models.
  arXiv  Detail & Related papers  (2025-05-26T12:23:14Z)
• The Unreasonable Effectiveness of Model Merging for Cross-Lingual Transfer in LLMs [54.59207567677249]
  Large language models (LLMs) still struggle across tasks outside of high-resource languages.<n>In this work, we investigate cross-lingual transfer to lower-resource languages where task-specific post-training data is scarce.
  arXiv  Detail & Related papers  (2025-05-23T20:28:31Z)
• xVLM2Vec: Adapting LVLM-based embedding models to multilinguality using Self-Knowledge Distillation [2.9998889086656586]
  We propose an adaptation methodology for Large Vision-Language Models trained on English language data to improve their performance.<n>We introduce a benchmark to evaluate the effectiveness of multilingual and multimodal embedding models.
  arXiv  Detail & Related papers  (2025-03-12T12:04:05Z)
• P-MMEval: A Parallel Multilingual Multitask Benchmark for Consistent Evaluation of LLMs [84.24644520272835]
  We introduce P-MMEval, a large-scale benchmark covering effective fundamental and capability-specialized datasets.<n>P-MMEval delivers consistent language coverage across various datasets and provides parallel samples.<n>We conduct extensive experiments on representative multilingual model series to compare performances across models and tasks.
  arXiv  Detail & Related papers  (2024-11-14T01:29:36Z)
• Exploring Multiple Strategies to Improve Multilingual Coreference Resolution in CorefUD [0.8602553195689511]
  We present a novel end-to-end neural coreference resolution system utilizing the CorefUD 1.1 dataset.<n>The proposed model is based on the standard end-to-end neural coreference resolution system.<n>We propose several extensions to enhance performance across diverse linguistic contexts.
  arXiv  Detail & Related papers  (2024-08-29T20:27:05Z)
• IAA: Inner-Adaptor Architecture Empowers Frozen Large Language Model with Multimodal Capabilities [4.269326314400742]
  We introduce the Inner-Adaptor Architecture for multimodal large language models (MLLMs)<n>The architecture incorporates multiple multimodal adaptors at varying depths within the large language model to facilitate direct interaction with the inherently text-oriented transformer layers.<n>Unlike previous approaches of freezing language models that require large-scale aligned data, our proposed architecture is able to achieve superior performance on small-scale datasets.
  arXiv  Detail & Related papers  (2024-08-23T08:10:13Z)
• CharED: Character-wise Ensemble Decoding for Large Language Models [24.993790740335243]
  We present an inference-time ensembling algorithm aimed at "averaging" outputs from multiple large language models. Our proposed model is able to combine complimentary strengths of multiple LLMs, regardless of vocabulary, tokenization, or model size.
  arXiv  Detail & Related papers  (2024-06-25T22:35:07Z)
• A Large-Scale Evaluation of Speech Foundation Models [110.95827399522204]
  We establish the Speech processing Universal PERformance Benchmark (SUPERB) to study the effectiveness of the foundation model paradigm for speech. We propose a unified multi-tasking framework to address speech processing tasks in SUPERB using a frozen foundation model followed by task-specialized, lightweight prediction heads.
  arXiv  Detail & Related papers  (2024-04-15T00:03:16Z)
• Model Composition for Multimodal Large Language Models [71.5729418523411]
  We propose a new paradigm through the model composition of existing MLLMs to create a new model that retains the modal understanding capabilities of each original model. Our basic implementation, NaiveMC, demonstrates the effectiveness of this paradigm by reusing modality encoders and merging LLM parameters.
  arXiv  Detail & Related papers  (2024-02-20T06:38:10Z)
• AnyGPT: Unified Multimodal LLM with Discrete Sequence Modeling [115.89786751297348]
  We introduce AnyGPT, an any-to-any multimodal language model that utilizes discrete representations for the unified processing of various modalities. We build a multimodal text-centric dataset for multimodal alignment pre-training. We show that AnyGPT is capable of facilitating any-to-any multimodal conversation while achieving performance comparable to specialized models across all modalities.
  arXiv  Detail & Related papers  (2024-02-19T15:33:10Z)
• On the Analysis of Cross-Lingual Prompt Tuning for Decoder-based Multilingual Model [49.81429697921861]
  We study the interaction between parameter-efficient fine-tuning (PEFT) and cross-lingual tasks in multilingual autoregressive models. We show that prompt tuning is more effective in enhancing the performance of low-resource languages than fine-tuning.
  arXiv  Detail & Related papers  (2023-11-14T00:43:33Z)
• Exploiting Multilingualism in Low-resource Neural Machine Translation via Adversarial Learning [3.2258463207097017]
  Generative Adversarial Networks (GAN) offer a promising approach for Neural Machine Translation (NMT) In GAN, similar to bilingual models, multilingual NMT only considers one reference translation for each sentence during model training. This article proposes Denoising Adversarial Auto-encoder-based Sentence Interpolation (DAASI) approach to perform sentence computation.
  arXiv  Detail & Related papers  (2023-03-31T12:34:14Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.