Apertus LLM Family Expansion via Distillation and Quantization

• URL: http://arxiv.org/abs/2605.29128v1
• Date: Wed, 27 May 2026 21:40:10 GMT
• Title: Apertus LLM Family Expansion via Distillation and Quantization
• Abstract summary: We produce Apertus-v1.1 - a distilled family of models with up to 4B parameters trained on 1.7T permissive license tokens.<n>We demonstrate cost-efficiency and strong accuracy performance of our approach for covering large ranges of hardware and systems requirements.
• Score: 68.33467651353419
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The wide adoption of LLMs has led to their use in great variety of applications and scenarios, such as chatbot assistants and data annotation, creating the need for the models to satisfy certain budget and hardware constraints. This has led to the trend of LLMs being released in batches consisting of similar models of various sizes for the family of models to adhere to as wide of a range of constraints as possible. In this paper, we validate distillation and quantization as a cost-effective way to expand model families to new sizes and hardware formats. Based on the open-recipe Apertus 8B LLM, we produce Apertus-v1.1 - a distilled family of models with up to 4B parameters trained on 1.7T permissive license tokens. We demonstrate cost-efficiency and strong accuracy performance of our approach for covering large ranges of hardware and systems requirements.

Related papers

• Nemotron Elastic: Towards Efficient Many-in-One Reasoning LLMs [80.72350166388601]
  Nemotron Elastic is a framework for building reasoning-oriented LLMs.<n>It embeds nested submodels within a single parent model.<n>Each of these submodels shares weights with the parent model and can be extracted zero-shot during deployment.
  arXiv  Detail & Related papers  (2025-11-20T18:59:21Z)
• Towards Budget-Friendly Model-Agnostic Explanation Generation for Large Language Models [14.110188927768736]
  We show that it is practical to generate faithful explanations for large-scale Large language models by sampling from some budget-friendly models.<n>Our analysis provides a new paradigm of model-agnostic explanation methods for LLMs, by including information from budget-friendly models.
  arXiv  Detail & Related papers  (2025-05-18T18:05:37Z)
• DistilQwen2.5: Industrial Practices of Training Distilled Open Lightweight Language Models [10.34623505096336]
  We present DistilQwen2.5, a family of distilled, lightweight language models (LLMs) derived from the public Qwen2.5 models.<n>These models exhibit enhanced instruction-following capabilities compared to the original models.<n>To facilitate practical use, we have released all the DistilQwen2.5 models to the open-source community.
  arXiv  Detail & Related papers  (2025-04-21T11:26:02Z)
• MLKV: Efficiently Scaling up Large Embedding Model Training with Disk-based Key-Value Storage [22.848456481878568]
  This paper presents MLKV, an efficient, reusable data storage framework designed to address the scalability challenges in embedding model training.<n>In experiments on open-source workloads, MLKV outperforms offloading strategies built on top of industrial-strength key-value stores by 1.6-12.6x.
  arXiv  Detail & Related papers  (2025-04-02T08:57:01Z)
• Mini-InternVL: A Flexible-Transfer Pocket Multimodal Model with 5% Parameters and 90% Performance [78.48606021719206]
  Mini-InternVL is a series of MLLMs with parameters ranging from 1B to 4B, which achieves 90% of the performance with only 5% of the parameters. We develop a unified adaptation framework for Mini-InternVL, which enables our models to transfer and outperform specialized models in downstream tasks.
  arXiv  Detail & Related papers  (2024-10-21T17:58:20Z)
• Fine-Tuning and Deploying Large Language Models Over Edges: Issues and Approaches [64.42735183056062]
  Large language models (LLMs) have evolved from specialized deep models to versatile foundation models.<n>LLMs require fine-tuning on local datasets and substantial memory for deployment over the network edges.<n>LLMs have been expanded beyond text generation to create images, audio, video, and multi-modal content.<n>Model fine-tuning and model-compression techniques have been developed to support the sustainable growth of LLMs.
  arXiv  Detail & Related papers  (2024-08-20T09:42:17Z)
• LLAVADI: What Matters For Multimodal Large Language Models Distillation [77.73964744238519]
  In this work, we do not propose a new efficient model structure or train small-scale MLLMs from scratch. Our studies involve training strategies, model choices, and distillation algorithms in the knowledge distillation process. By evaluating different benchmarks and proper strategy, even a 2.7B small-scale model can perform on par with larger models with 7B or 13B parameters.
  arXiv  Detail & Related papers  (2024-07-28T06:10:47Z)
• Delta-CoMe: Training-Free Delta-Compression with Mixed-Precision for Large Language Models [79.46938238953916]
  Fine-tuning large language models (LLMs) to diverse applications is crucial to meet complex demands. Recent studies suggest decomposing a fine-tuned LLM into a base model and corresponding delta weights, which are then compressed using low-rank or low-bit approaches to reduce costs. In this work, we observe that existing low-rank and low-bit compression methods can significantly harm the model performance for task-specific fine-tuned LLMs.
  arXiv  Detail & Related papers  (2024-06-13T07:57:27Z)
• Demystifying AI Platform Design for Distributed Inference of Next-Generation LLM models [8.02264001053969]
  Large language models (LLMs) have shown remarkable performance across a wide range of applications, often outperforming human experts.<n>With constant innovation in LLM serving optimizations and model architecture evolving at breakneck speed, the hardware requirements to meet Service Level Objectives (SLOs) remain an open research question.<n>We present an analytical tool, GenZ, to efficiently navigate the relationship between diverse LLM model architectures and AI platform design parameters.
  arXiv  Detail & Related papers  (2024-06-03T18:00:50Z)
• Any-Precision LLM: Low-Cost Deployment of Multiple, Different-Sized LLMs [3.450141240227484]
  We propose a lightweight method for any-precision quantization of Large Language Models (LLMs) Our solution significantly reduces the high costs of deploying multiple, different-sized LLMs. All the supported LLMs with varying bit-widths demonstrate state-of-the-art model quality and inference throughput.
  arXiv  Detail & Related papers  (2024-02-16T09:06:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.