Token-Operations-Oriented Inference Optimization Techniques for Large Models

• URL: http://arxiv.org/abs/2606.20295v1
• Date: Thu, 18 Jun 2026 14:33:09 GMT
• Title: Token-Operations-Oriented Inference Optimization Techniques for Large Models
• Abstract summary: Large model inference optimization serves as a key foundation for supporting the scalable, low-cost, and highly stable operation of large model services.<n>This paper proposes for the first time a four-layer technical architecture consisting of Multi-model Fusion, Model Optimization, Compute-Model Fusion, and Compute-Network-Model Fusion.
• Score: 97.15115857661591
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Large model inference optimization serves as a key foundation for supporting the scalable, low-cost, and highly stable operation of large model services. Centered on token-oriented inference optimization technology, this paper proposes for the first time a four-layer technical architecture consisting of Multi-model Fusion, Model Optimization, Compute-Model Fusion, and Compute-Network-Model Fusion. It systematically reviews the key technologies and current industry status across these four levels and analyzes the application value of related technologies in real-world business scenarios. This paper provides a practical technical path for reducing token production costs, improving token service efficiency, ensuring the stability of token supply, and driving the transition of large model services from being merely callable to being operable.

Related papers

• Constraint-Driven Model Optimization: An Industry Framework for Selecting Compression and Acceleration Techniques in Modern Machine Learning Systems [0.0]
  We argue that optimization should be formulated as a constraint-driven, multi-objective engineering decision.<n>This work provides one of the first structured attempts to formalize model optimization as a constraint-aware, multi-objective engineering process.
  arXiv  Detail & Related papers  (2026-07-15T11:47:24Z)
• BERT4beam: Large AI Model Enabled Generalized Beamforming Optimization [77.17508487745026]
  This paper investigates the large-scale AI model designed for beamforming optimization to adapt and generalize to diverse tasks defined by system utilities and scales.<n>We propose a novel framework based on bidirectional encoder representations from transformers (BERT), termed BERT4 encoder.<n>Based on the framework, we propose two BERT-based approaches for single-task and multi-task beamforming optimization, respectively.
  arXiv  Detail & Related papers  (2025-09-14T02:49:29Z)
• TCPO: Thought-Centric Preference Optimization for Effective Embodied Decision-making [75.29820290660065]
  This paper proposes Thought-Centric Preference Optimization ( TCPO) for effective embodied decision-making.<n>It emphasizes the alignment of the model's intermediate reasoning process, mitigating the problem of model degradation.<n>Experiments in the ALFWorld environment demonstrate an average success rate of 26.67%, achieving a 6% improvement over RL4VLM.
  arXiv  Detail & Related papers  (2025-09-10T11:16:21Z)
• CRACI: A Cloud-Native Reference Architecture for the Industrial Compute Continuum [0.0]
  This paper introduces CRACI, a Cloud-native Reference Architecture for the Industrial Compute Continuum.<n>It promotes a decoupled and event-driven model to enable flexible, non-hierarchical data flows across the continuum.
  arXiv  Detail & Related papers  (2025-09-09T08:24:08Z)
• A Survey on Inference Optimization Techniques for Mixture of Experts Models [50.40325411764262]
  Large-scale Mixture of Experts (MoE) models offer enhanced model capacity and computational efficiency through conditional computation.<n> deploying and running inference on these models presents significant challenges in computational resources, latency, and energy efficiency.<n>This survey analyzes optimization techniques for MoE models across the entire system stack.
  arXiv  Detail & Related papers  (2024-12-18T14:11:15Z)
• ORLM: A Customizable Framework in Training Large Models for Automated Optimization Modeling [15.67321902882617]
  We propose a viable path for training open-source LLMs capable of optimization modeling and developing solver codes.<n>This work also introduces IndustryOR, the first industrial benchmark for evaluating LLMs in solving practical OR problems.
  arXiv  Detail & Related papers  (2024-05-28T01:55:35Z)
• Majority Kernels: An Approach to Leverage Big Model Dynamics for Efficient Small Model Training [32.154166415680066]
  Methods like distillation, compression or quantization help leverage the highly performant large models to induce smaller performant ones. This paper explores the hypothesis that a single training run can simultaneously train a larger model for performance and derive a smaller model for deployment.
  arXiv  Detail & Related papers  (2024-02-07T17:07:41Z)
• Parameter-Efficient Fine-Tuning for Pre-Trained Vision Models: A Survey and Benchmark [95.0484665958967]
  Pre-trained vision models (PVMs) have demonstrated remarkable adaptability across a wide range of downstream vision tasks.<n>As these models scale to billions or even trillions of parameters, conventional full fine-tuning has become increasingly impractical due to its high computational and storage demands.<n> parameter-efficient fine-tuning (PEFT) has emerged as a promising alternative, aiming to achieve performance comparable to full fine-tuning while making minimal adjustments to the model parameters.
  arXiv  Detail & Related papers  (2024-02-03T19:12:20Z)
• When Parameter-efficient Tuning Meets General-purpose Vision-language Models [65.19127815275307]
  PETAL revolutionizes the training process by requiring only 0.5% of the total parameters, achieved through a unique mode approximation technique. Our experiments reveal that PETAL not only outperforms current state-of-the-art methods in most scenarios but also surpasses full fine-tuning models in effectiveness.
  arXiv  Detail & Related papers  (2023-12-16T17:13:08Z)
• Empirical Analysis of the Strengths and Weaknesses of PEFT Techniques for LLMs [1.867982979635437]
  We provide a benchmark of various PEFT techniques and evaluate model performance across different data scales. Contrary to popular belief, we empirically prove that PEFT techniques converge slower than full tuning in low data scenarios. We further optimize these PEFT techniques by selectively choosing which parts of the model to train, and find that these techniques can be applied with significantly fewer parameters.
  arXiv  Detail & Related papers  (2023-04-28T17:39:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.