DLLG: Dynamic Logit-Level Gating of LLM Experts

• URL: http://arxiv.org/abs/2606.04378v1
• Date: Wed, 03 Jun 2026 02:51:56 GMT
• Title: DLLG: Dynamic Logit-Level Gating of LLM Experts
• Abstract summary: We propose a dynamic logit-level ensembling framework that learns token-level expert fusion from sparse response-level supervision.<n>A lightweight gating module predicts step-wise fusion weights, linking trajectory-level correctness to generation without token-level labels or expert retraining.
• Score: 63.26663173267099
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Leveraging multiple specialized LLMs can combine complementary strengths, but existing approaches trade adaptability for stability: routing commits prematurely, heuristic ensembling depends on fragile proxies, and parameter merging introduces interference. We propose DLLG (Dynamic Logit-Level Gating), a dynamic logit-level ensembling framework that learns token-level expert fusion from sparse response-level supervision. A lightweight gating module predicts step-wise fusion weights, linking trajectory-level correctness to generation without token-level labels or expert retraining. Across diverse reasoning and code benchmarks, DLLG consistently outperforms strong routing, heuristic ensembling, and parameter-merging baselines across model scales, highlighting learned logit-level fusion as a robust and scalable paradigm for integrating specialized experts.

Related papers

• Empowering Compact LLMs with Fusion of Layer-wise Exits for Recommendation [59.216721553953]
  Large language model-based recommender systems (LLM-RSs) have demonstrated remarkable capabilities, but are computationally unsustainable for many real-world applications.<n>We propose a discriminative framework that enhances compact LLMs while retaining scalable full-corpus ranking.<n>Experiments on three real-world datasets with Qwen 3 1.7B and Llama 3.2 3B show that FLEXRec achieves state-of-the-art accuracy while remaining highly efficient.
  arXiv  Detail & Related papers  (2026-08-18T03:14:28Z)
• When Model Merging Breaks Routing: Training-Free Calibration for MoE [67.27839627141972]
  Hessian-Aware Router (HARC) is a training-free framework that leverages second-order curvature information to realign the merged router.<n> Experiments on mathematical reasoning and code generation tasks show that HARC effectively mitigates routing breakdown across diverse MoE merging baselines.
  arXiv  Detail & Related papers  (2026-06-02T09:33:33Z)
• Harmonizing Real-Time Constraints and Long-Horizon Reasoning: An Asynchronous Agentic Framework for Dynamic Scheduling [11.212756680882444]
  We introduce RACE-Sched, an agent-based framework that decouples policy execution from logical reasoning via a dual-stream architecture.<n>This approach harmonizes real-time constraints with long-horizon reasoning to achieve superior solution quality and robust adaptation to dynamic events.
  arXiv  Detail & Related papers  (2026-05-28T02:26:18Z)
• Proximity-Based Multi-Turn Optimization: Practical Credit Assignment for LLM Agent Training [26.571744733431448]
  Multi-turn LLM agents are pivotal to production systems, spanning customer service automation, e-commerce assistance, and interactive task management.<n>We propose Proximity-based Multi-turn Optimization (ProxMO), a practical and robust framework engineered specifically for the constraints of real-world deployment.
  arXiv  Detail & Related papers  (2026-02-22T15:18:03Z)
• CRAFT-LoRA: Content-Style Personalization via Rank-Constrained Adaptation and Training-Free Fusion [27.087994191559904]
  Low-Rank Adaptation (LoRA) offers an efficient personalization approach, with potential for precise control through combining LoRA weights on different concepts.<n>Existing combination techniques face persistent challenges: entanglement between content and style representations, insufficient guidance for controlling elements' influence, and unstable weight fusion that often require additional training.<n>We address these limitations through CRAFT-LoRA, with complementary components: (1) rank-constrained backbone fine-tuning that injects low-rank projection residuals to encourage learning decoupled content and style subspaces; (2) a prompt-guided approach featuring an expert encoder with specialized branches that enables semantic extension and precise control
  arXiv  Detail & Related papers  (2026-02-21T19:05:11Z)
• Refer-Agent: A Collaborative Multi-Agent System with Reasoning and Reflection for Referring Video Object Segmentation [50.22481337087162]
  Referring Video Object (RVOS) aims to segment objects in videos based on textual queries.<n>Refer-Agent is a collaborative multi-agent system with alternating reasoning-reflection mechanisms.
  arXiv  Detail & Related papers  (2026-02-03T14:48:12Z)
• Semantic Routing: Exploring Multi-Layer LLM Feature Weighting for Diffusion Transformers [31.67315012315044]
  We introduce a unified normalized convex fusion framework equipped with lightweight gates to systematically organize multi-layer LLM hidden states.<n>Experiments establish Depth-wise Semantic Routing as the superior conditioning strategy.<n>We find that purely time-wise fusion can paradoxically degrade visual generation fidelity.
  arXiv  Detail & Related papers  (2026-02-03T13:30:13Z)
• RISER: Orchestrating Latent Reasoning Skills for Adaptive Activation Steering [62.63376387138257]
  We propose a plug-and-play intervention framework that adaptively steers large language models (LLMs) reasoning in activation space.<n>RISER constructs a library of reusable reasoning vectors and employs a lightweight Router to dynamically compose them for each input.<n>The Router is optimized via reinforcement learning under task-level rewards, activating latent cognitive primitives in an emergent and compositional manner.
  arXiv  Detail & Related papers  (2026-01-14T08:04:33Z)
• Performant LLM Agentic Framework for Conversational AI [1.6114012813668932]
  We introduce the Performant Agentic Framework (PAF), a novel system that assists Large Language Models (LLMs) in selecting appropriate nodes and executing actions in order when traversing complex graphs.<n>PAF combines LLM-based reasoning with a mathematically grounded vector scoring mechanism, achieving both higher accuracy and reduced latency.<n>Experiments demonstrate that PAF significantly outperforms baseline methods, paving the way for scalable, real-time Conversational AI systems in complex business environments.
  arXiv  Detail & Related papers  (2025-03-09T02:58:34Z)
• Unleash LLMs Potential for Recommendation by Coordinating Twin-Tower Dynamic Semantic Token Generator [60.07198935747619]
  We propose Twin-Tower Dynamic Semantic Recommender (T TDS), the first generative RS which adopts dynamic semantic index paradigm. To be more specific, we for the first time contrive a dynamic knowledge fusion framework which integrates a twin-tower semantic token generator into the LLM-based recommender. The proposed T TDS recommender achieves an average improvement of 19.41% in Hit-Rate and 20.84% in NDCG metric, compared with the leading baseline methods.
  arXiv  Detail & Related papers  (2024-09-14T01:45:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.