MoCo: A One-Stop Shop for Model Collaboration Research

• URL: http://arxiv.org/abs/2601.21257v1
• Date: Thu, 29 Jan 2026 04:36:52 GMT
• Title: MoCo: A One-Stop Shop for Model Collaboration Research
• Authors: Shangbin Feng, Yuyang Bai, Ziyuan Yang, Yike Wang, Zhaoxuan Tan, Jiajie Yan, Zhenyu Lei, Wenxuan Ding, Weijia Shi, Haojin Wang, Zhenting Qi, Yuru Jiang, Heng Wang, Chengsong Huang, Yu Fei, Jihan Yao, Yilun Du, Luke Zettlemoyer, Yejin Choi, Yulia Tsvetkov,
• Abstract summary: We present MoCo: a one-stop Python library of executing, benchmarking, and comparing model collaboration algorithms at scale.<n>MoCo features 26 model collaboration methods, spanning diverse levels of cross-model information exchange.<n>Extensive experiments with MoCo demonstrate that most collaboration strategies outperform models without collaboration.<n>We envision MoCo as a valuable toolkit to facilitate and turbocharge the quest for an open, modular, decentralized, and collaborative AI future.
• Score: 132.52160996841505
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Advancing beyond single monolithic language models (LMs), recent research increasingly recognizes the importance of model collaboration, where multiple LMs collaborate, compose, and complement each other. Existing research on this topic has mostly been disparate and disconnected, from different research communities, and lacks rigorous comparison. To consolidate existing research and establish model collaboration as a school of thought, we present MoCo: a one-stop Python library of executing, benchmarking, and comparing model collaboration algorithms at scale. MoCo features 26 model collaboration methods, spanning diverse levels of cross-model information exchange such as routing, text, logit, and model parameters. MoCo integrates 25 evaluation datasets spanning reasoning, QA, code, safety, and more, while users could flexibly bring their own data. Extensive experiments with MoCo demonstrate that most collaboration strategies outperform models without collaboration in 61.0% of (model, data) settings on average, with the most effective methods outperforming by up to 25.8%. We further analyze the scaling of model collaboration strategies, the training/inference efficiency of diverse methods, highlight that the collaborative system solves problems where single LMs struggle, and discuss future work in model collaboration, all made possible by MoCo. We envision MoCo as a valuable toolkit to facilitate and turbocharge the quest for an open, modular, decentralized, and collaborative AI future.

Related papers

• The Single-Multi Evolution Loop for Self-Improving Model Collaboration Systems [55.28554025674495]
  We improve efficiency while preserving the strengths of collaboration by distilling collaborative patterns into a single model.<n>We propose the single-multi evolution loop: multiple LMs collaborate, each distills from the collaborative outputs, and these post-distillation improved LMs collaborate again.
  arXiv  Detail & Related papers  (2026-02-05T01:20:32Z)
• The Law of Multi-Model Collaboration: Scaling Limits of Model Ensembling for Large Language Models [54.51795784459866]
  We propose a theoretical framework of performance scaling for multi-model collaboration.<n>We show that multi-model systems follow a power-law scaling with respect to the total parameter count.<n> ensembles of heterogeneous model families achieve better performance scaling than those formed within a single model family.
  arXiv  Detail & Related papers  (2025-12-29T09:55:12Z)
• KCM: KAN-Based Collaboration Models Enhance Pretrained Large Models [62.658961779827145]
  We propose a KAN-based Collaborative Model (KCM) as an improved approach to large-small model collaboration.<n>KAN offers superior visualizability and interpretability while mitigating catastrophic forgetting.
  arXiv  Detail & Related papers  (2025-10-23T07:06:21Z)
• MCP: A Control-Theoretic Orchestration Framework for Synergistic Efficiency and Interpretability in Multimodal Large Language Models [0.0]
  This study proposes a three-layer collaboration framework based on model-controller-task adaptation (MCP)<n> Experiments show that the MCP framework improves the performance of cross-modal benchmarking tasks, such as GLUE, COCO, ScienceQA, etc., by 15-30% compared with the baseline model, improves the reasoning efficiency by 40%, and generates the interpretable intermediate results through the Presenter layer, obtaining 90% of the manual interpretability scores.
  arXiv  Detail & Related papers  (2025-09-20T09:44:11Z)
• Efficient LLM Collaboration via Planning [56.081879390960204]
  Small and large models take turns acting as planner and executor, exchanging plans in a multi-stage cascade to collaboratively solve tasks.<n>We demonstrate that COPE achieves performance comparable to large proprietary models, while drastically reducing the inference API cost.
  arXiv  Detail & Related papers  (2025-06-13T08:35:50Z)
• OptMerge: Unifying Multimodal LLM Capabilities and Modalities via Model Merging [124.91183814854126]
  Model merging seeks to combine multiple expert models into a single model.<n>We introduce a benchmark for model merging research that clearly divides the tasks for MLLM training and evaluation.<n>We find that model merging offers a promising way for building improved MLLMs without requiring training data.
  arXiv  Detail & Related papers  (2025-05-26T12:23:14Z)
• MergeBench: A Benchmark for Merging Domain-Specialized LLMs [25.333088749417414]
  MergeBench is an evaluation suite designed to assess model merging at scale.<n>It builds on state-of-the-art open-source language models, including Llama and Gemma families at 2B to 9B scales.<n>We assess eight representative merging methods across multi-task performance, forgetting and runtime efficiency.
  arXiv  Detail & Related papers  (2025-05-16T04:02:55Z)
• A Collaborative Ensemble Framework for CTR Prediction [73.59868761656317]
  We propose a novel framework, Collaborative Ensemble Training Network (CETNet), to leverage multiple distinct models. Unlike naive model scaling, our approach emphasizes diversity and collaboration through collaborative learning. We validate our framework on three public datasets and a large-scale industrial dataset from Meta.
  arXiv  Detail & Related papers  (2024-11-20T20:38:56Z)
• MAPL: Model Agnostic Peer-to-peer Learning [2.9221371172659616]
  We introduce Model Agnostic Peer-to-peer Learning (MAPL) to simultaneously learn heterogeneous personalized models and a collaboration graph. MAPL is comprised of two main modules: (i) local-level Personalized Model Learning (PML), leveraging a combination of intra- and inter-client contrastive losses; (ii) network-wide decentralized Collaborative Graph Learning (CGL) dynamically refining collaboration weights based on local task similarities.
  arXiv  Detail & Related papers  (2024-03-28T19:17:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.