A Survey on Collaborative Mechanisms Between Large and Small Language Models

• URL: http://arxiv.org/abs/2505.07460v1
• Date: Mon, 12 May 2025 11:48:42 GMT
• Title: A Survey on Collaborative Mechanisms Between Large and Small Language Models
• Authors: Yi Chen, JiaHao Zhao, HaoHao Han,
• Abstract summary: Large Language Models (LLMs) deliver powerful AI capabilities but face deployment challenges due to high resource costs and latency.<n>Small Language Models (SLMs) offer efficiency and deployability at the cost of reduced performance.
• Score: 5.1159419867547085
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Large Language Models (LLMs) deliver powerful AI capabilities but face deployment challenges due to high resource costs and latency, whereas Small Language Models (SLMs) offer efficiency and deployability at the cost of reduced performance. Collaboration between LLMs and SLMs emerges as a crucial paradigm to synergistically balance these trade-offs, enabling advanced AI applications, especially on resource-constrained edge devices. This survey provides a comprehensive overview of LLM-SLM collaboration, detailing various interaction mechanisms (pipeline, routing, auxiliary, distillation, fusion), key enabling technologies, and diverse application scenarios driven by on-device needs like low latency, privacy, personalization, and offline operation. While highlighting the significant potential for creating more efficient, adaptable, and accessible AI, we also discuss persistent challenges including system overhead, inter-model consistency, robust task allocation, evaluation complexity, and security/privacy concerns. Future directions point towards more intelligent adaptive frameworks, deeper model fusion, and expansion into multimodal and embodied AI, positioning LLM-SLM collaboration as a key driver for the next generation of practical and ubiquitous artificial intelligence.

Related papers

• AI Flow: Perspectives, Scenarios, and Approaches [51.38621621775711]
  We introduce AI Flow, a framework that integrates cutting-edge IT and CT advancements.<n>First, device-edge-cloud framework serves as the foundation, which integrates end devices, edge servers, and cloud clusters.<n>Second, we introduce the concept of familial models, which refers to a series of different-sized models with aligned hidden features.<n>Third, connectivity- and interaction-based intelligence emergence is a novel paradigm of AI Flow.
  arXiv  Detail & Related papers  (2025-06-14T12:43:07Z)
• LLM-Powered AI Agent Systems and Their Applications in Industry [3.103098467546532]
  Large Language Models (LLMs) have reshaped agent systems.<n>Unlike traditional rule-based agents with limited task scope, LLM-powered agents offer greater flexibility, cross-domain reasoning, and natural language interaction.
  arXiv  Detail & Related papers  (2025-05-22T01:52:15Z)
• Edge-Cloud Collaborative Computing on Distributed Intelligence and Model Optimization: A Survey [59.52058740470727]
  Edge-cloud collaborative computing (ECCC) has emerged as a pivotal paradigm for addressing the computational demands of modern intelligent applications.<n>Recent advancements in AI, particularly deep learning and large language models (LLMs), have dramatically enhanced the capabilities of these distributed systems.<n>This survey provides a structured tutorial on fundamental architectures, enabling technologies, and emerging applications.
  arXiv  Detail & Related papers  (2025-05-03T13:55:38Z)
• Federated Intelligence: When Large AI Models Meet Federated Fine-Tuning and Collaborative Reasoning at the Network Edge [10.848407787567519]
  Large artificial intelligence (AI) models exhibit remarkable capabilities in various application scenarios.<n> deploying them at the network edge poses significant challenges due to issues such as data privacy, computational resources, and latency.<n>In this paper, we explore federated fine-tuning and collaborative reasoning techniques to facilitate the implementation of large AI models in resource-constrained wireless networks.
  arXiv  Detail & Related papers  (2025-03-27T11:56:36Z)
• Towards Agentic Recommender Systems in the Era of Multimodal Large Language Models [75.4890331763196]
  Recent breakthroughs in Large Language Models (LLMs) have led to the emergence of agentic AI systems.<n>LLM-based Agentic RS (LLM-ARS) can offer more interactive, context-aware, and proactive recommendations.
  arXiv  Detail & Related papers  (2025-03-20T22:37:15Z)
• Small Vision-Language Models: A Survey on Compact Architectures and Techniques [0.28087862620958753]
  The emergence of small vision-language models (sVLMs) marks a critical advancement in multimodal AI.<n>This survey offers a taxonomy of architectures that highlight innovations in compact design and computational efficiency.
  arXiv  Detail & Related papers  (2025-03-09T16:14:46Z)
• An Overview of Large Language Models for Statisticians [109.38601458831545]
  Large Language Models (LLMs) have emerged as transformative tools in artificial intelligence (AI)<n>This paper explores potential areas where statisticians can make important contributions to the development of LLMs.<n>We focus on issues such as uncertainty quantification, interpretability, fairness, privacy, watermarking and model adaptation.
  arXiv  Detail & Related papers  (2025-02-25T03:40:36Z)
• Scaling Autonomous Agents via Automatic Reward Modeling And Planning [52.39395405893965]
  Large language models (LLMs) have demonstrated remarkable capabilities across a range of tasks.<n>However, they still struggle with problems requiring multi-step decision-making and environmental feedback.<n>We propose a framework that can automatically learn a reward model from the environment without human annotations.
  arXiv  Detail & Related papers  (2025-02-17T18:49:25Z)
• Large Language Models for Multi-Robot Systems: A Survey [9.31855372655603]
  Multi-Robot Systems (MRS) pose unique challenges, including coordination, scalability, and real-world adaptability.<n>This survey provides the first comprehensive exploration of Large Language Models (LLMs) integration into MRS.<n>We highlight key applications in diverse domains, such as household robotics, construction, formation control, target tracking, and robot games.
  arXiv  Detail & Related papers  (2025-02-06T06:52:14Z)
• Large Language Model as a Catalyst: A Paradigm Shift in Base Station Siting Optimization [62.16747639440893]
  Large language models (LLMs) and their associated technologies advance, particularly in the realms of prompt engineering and agent engineering.<n>Our proposed framework incorporates retrieval-augmented generation (RAG) to enhance the system's ability to acquire domain-specific knowledge and generate solutions.
  arXiv  Detail & Related papers  (2024-08-07T08:43:32Z)
• A General-Purpose Device for Interaction with LLMs [3.052172365469752]
  This paper investigates integrating large language models (LLMs) with advanced hardware. We focus on developing a general-purpose device designed for enhanced interaction with LLMs.
  arXiv  Detail & Related papers  (2024-08-02T23:43:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.