YouZhi: Towards High-Concurrency Financial LLMs via Adaptive GQA-to-MLA Transition

• URL: http://arxiv.org/abs/2606.05868v1
• Date: Thu, 04 Jun 2026 08:44:37 GMT
• Title: YouZhi: Towards High-Concurrency Financial LLMs via Adaptive GQA-to-MLA Transition
• Abstract summary: Large language models (LLMs) drive significant financial innovations, yet their deployment is bottlenecked by KV cache memory overhead.<n>We propose YouZhi-LLM, a highly efficient financial LLM empowered by a comprehensive structural transition and training pipeline built on the Huawei Ascend ecosystem.
• Score: 78.52944812151664
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Large language models (LLMs) drive significant financial innovations, yet their high-concurrency deployment is severely bottlenecked by KV cache memory overhead, which inflates infrastructure costs and throttles scalability. To address this, we propose YouZhi-LLM, a highly efficient financial LLM empowered by a comprehensive structural transition and training pipeline natively built on the Huawei Ascend ecosystem. At its algorithmic core, YouZhi-LLM features a layer-adaptive GQA-to-MLA transition framework that dynamically assigns per-layer FreqFold sizes, maximizing KV-cache compression while minimizing perplexity degradation. To recover representation capacity and inject domain expertise, the Ascend-based training pipeline seamlessly integrates generalized knowledge distillation with financial-specific supervised fine-tuning. Evaluations demonstrate the superiority of this systematic approach, with the adaptive transition reducing perplexity degradation by up to 35% over uniform baselines. Crucially, when evaluated on Ascend NPUs via vLLM-Ascend, the massive KV-cache reduction translates directly into deployment efficiency. Compared to their respective base models, YouZhi-7B yields a 12.3% improvement in average financial benchmark score alongside a 2.69$\times$ increase in maximum concurrency; similarly, YouZhi-14B achieves a 7.0% accuracy gain and a 2.43$\times$ concurrency boost, establishing a new paradigm for cost-effective, high-throughput financial inference.

Related papers

• MMOE: Modernizing Diffusion Transformers with Efficient Expert Design [76.86793131833551]
  Modern large language models scale successfully by pairing capacity growth with efficiency, keeping per-token and deployment costs under control as capacity grows.<n>Recent efforts mostly enlarge total parameter counts and sparsity ratios without importing the efficiency mechanisms that made LLM scaling practical.<n>We introduce ModernMOE, a modernization of SiT-style diffusion transformers that adapts routed experts, shared and lightweight experts, gate-residual routing, and attention-residual information reuse to AIGC generation.
  arXiv  Detail & Related papers  (2026-07-27T17:05:04Z)
• A1: A Fully Transparent Open-Source, Adaptive and Efficient Truncated Vision-Language-Action Model [112.9420001646428]
  Vision-Language-Action (VLA) models have emerged as a powerful paradigm for open-world robot manipulation, but their practical deployment is often constrained by cost.<n>We present A1, a fully open-source and transparent VLA framework designed for low-cost, high- throughput inference.<n>A1 achieves state-of-the-art success rates while significantly reducing inference cost.
  arXiv  Detail & Related papers  (2026-04-07T10:18:40Z)
• Cost-Efficient Multimodal LLM Inference via Cross-Tier GPU Heterogeneity [0.0]
  Multimodal large language model (MLLM) inference splits into two phases with opposing hardware demands.<n>We show that under standard transformer KV caching, the modality boundary minimizes cross-device transfer.<n>We build HeteroServe, a phase-aware runtime with modality-level partitioning and cross-tier scheduling.
  arXiv  Detail & Related papers  (2026-03-13T06:42:35Z)
• Fast KVzip: Efficient and Accurate LLM Inference with Gated KV Eviction [50.99402504483692]
  We propose a novel gating-based KV cache eviction method for frozen-weight language models.<n>Our approach integrates seamlessly into both the prefill and decoding stages.<n>Experiments show that our method maintains near-lossless performance while evicting up to 70% of the KV cache.
  arXiv  Detail & Related papers  (2026-01-25T03:07:54Z)
• Financial Text Classification Based On rLoRA Finetuning On Qwen3-8B model [0.0]
  State-of-the-art model Qwen3-8B exhibits strong instruction-following and multilingual capabilities.<n>It is specifically optimized for efficient fine tuning and high performance on reasoning-based benchmarks.<n>The synergy of instruction-based fine-tuning and memory-efficient optimization methods suggests Qwen3-8B can potentially serve as a scalable, economical option for real-time financial NLP applications.
  arXiv  Detail & Related papers  (2025-11-29T21:04:13Z)
• EcoSpa: Efficient Transformer Training with Coupled Sparsity [79.5008521101473]
  Transformers have become the backbone of modern AI, yet their high computational demands pose critical system challenges.<n>We introduce EcoSpa, an efficient structured sparse training method that jointly evaluates and sparsifies coupled weight matrix pairs.
  arXiv  Detail & Related papers  (2025-11-09T11:23:43Z)
• Type-Compliant Adaptation Cascades: Adapting Programmatic LM Workflows to Data [12.136710894967088]
  We introduce Type-Compliant Adaptation Cascades, a framework that recasts workflow adaptation as learning typed probabilistic programs.<n> Empirically, TACs significantly outperform state-of-the-art prompt-optimization baselines.
  arXiv  Detail & Related papers  (2025-08-25T17:36:21Z)
• ECHO-LLaMA: Efficient Caching for High-Performance LLaMA Training [18.081172277354145]
  ECHO-LLaMA transforms LLaMA models into shared KV caching across certain layers.<n>ECHO-LLaMA achieves up to 77% higher token-per-second throughput during training.<n>ECHO-LLaMA offers a scalable and cost-effective solution for pretraining and finetuning large language models.
  arXiv  Detail & Related papers  (2025-05-22T22:54:21Z)
• Towards Economical Inference: Enabling DeepSeek's Multi-Head Latent Attention in Any Transformer-based LLMs [92.7279890407059]
  Multi-head Latent Attention (MLA) is an innovative architecture designed to ensure efficient and economical inference.<n>This paper proposes the first data-efficient fine-tuning method for transitioning from Multi-Head Attention to MLA.
  arXiv  Detail & Related papers  (2025-02-20T18:50:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.