DND: Boosting Large Language Models with Dynamic Nested Depth

• URL: http://arxiv.org/abs/2510.11001v1
• Date: Mon, 13 Oct 2025 04:22:57 GMT
• Title: DND: Boosting Large Language Models with Dynamic Nested Depth
• Abstract summary: We introduce Dynamic Nested Depth (DND), a novel method that improves performance for off-the-shelf LLMs.<n>DND identifies more critical tokens with a router and feeds them back for an extra round of processing.<n>On diverse benchmarks, this approach boosts the performances of the dense Qwen3-1.7B by 1.88% and the MoE Qwen3-30B-A3B by 0.87%.
• Score: 45.76934615780697
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We introduce Dynamic Nested Depth (DND), a novel method that improves performance for off-the-shelf LLMs by selecting critical tokens to reprocess in a nested depth manner. Specifically, at the end of the given transformer layer, DND identifies more critical tokens with a router and feeds them back for an extra round of processing, effectively ``reviewing" difficult tokens while avoiding redundant computation for easier ones. The dynamic selection mechanism is tailored for precise control via two novel strategies: a router controlling loss to enhance token selection distinguishability, and a threshold control scheme to ensure selection stability. We demonstrate the effectiveness of DND by directly integrating it into pre-trained dense and MoE models during a post-training phase. On diverse benchmarks, this approach boosts the performances of the dense Qwen3-1.7B by 1.88% and the MoE Qwen3-30B-A3B by 0.87%, all with a minimal parameter and computing increase.

Related papers

• Adaptive Depth Sparse Framework: Similarity-Driven Resource Allocation for Pre-Trained LLMs [18.457250148137494]
  We present an Adaptive Depth Sparse Framework (AdaDSF) that converts off-the-shelf pre-trained large language models into depth-sparse models without retraining.<n>On GPT-NeoX and Qwen2.5 over language modeling and commonsense reasoning, AdaDSF substantially reduces inference FLOPs while preserving performance close to dense counterparts.
  arXiv  Detail & Related papers  (2026-07-23T13:13:04Z)
• Point-Selection Fine-Tuning Framework for Robust Point Cloud Classification [13.13510696965873]
  PSFT is a point-selection fine-tuning framework that improves robustness while remaining parameter-efficient.<n>We show that PSFT consistently reduces corruption error on Model-C and ModelNet40-C across all tested 3D pre-trained backbones.
  arXiv  Detail & Related papers  (2026-07-22T03:27:18Z)
• Multi-scale Coarse-to-fine Modeling for Test-time Human Motion Control [51.92884966472683]
  MSCoT is a multi-scale, coarse-to-fine model for test-time human motion synthesis and control.<n> MSCoT discretizes motion into a multi-scale hierarchical representation and predicts the entire token sequence at each temporal scale.
  arXiv  Detail & Related papers  (2026-05-14T15:09:49Z)
• Universally Empowering Zeroth-Order Optimization via Adaptive Layer-wise Sampling [43.822941944402544]
  Zeroth-Order optimization presents a promising memory-efficient paradigm for fine-tuning Large Language Models.<n>However, its practical adoption is severely constrained by slow wall-clock convergence and high estimation variance.<n>We propose AdaLeZO, an Adaptive Layer-wise ZO optimization framework.
  arXiv  Detail & Related papers  (2026-04-20T13:37:31Z)
• Depth Adaptive Efficient Visual Autoregressive Modeling [18.16894544431278]
  Visual Autoregressive ( VAR) modeling inefficiently applies a fixed computational depth to each position when generating high-resolution images.<n>We propose a paradigm shift from pruning entire tokens to adaptively allocating per-token computational depth.
  arXiv  Detail & Related papers  (2026-04-19T06:59:34Z)
• Hybrid DQN-TD3 Reinforcement Learning for Autonomous Navigation in Dynamic Environments [1.241204035960416]
  This paper presents a hierarchical path-planning and control framework that combines a high-level Deep Q-Network (DQN) for discrete sub-goal selection with a low-level Twin Delayed Deep Deterministic Policy Gradient (TD3) controller for continuous actuation.<n>We design a practical reward shaping scheme (direction, distance, obstacle avoidance, action smoothness, collision penalty, time penalty, and progress) together with a LiDAR-based safety gate that prevents unsafe motions.
  arXiv  Detail & Related papers  (2025-10-30T16:12:01Z)
• R-Stitch: Dynamic Trajectory Stitching for Efficient Reasoning [80.104336426172]
  Chain-of-thought (CoT) enhances problem-solving ability of large language models.<n>CoT incurs substantial inference cost due to long autoregressive trajectories.<n>We introduce R-Stitch, a training-free hybrid decoding framework.
  arXiv  Detail & Related papers  (2025-07-23T08:14:36Z)
• Leveraging Stochastic Depth Training for Adaptive Inference [1.996143466020199]
  We propose a simpler yet effective alternative for adaptive inference with a zero-overhead, single-model, and time-predictable inference.<n>Compared to original ResNets, our method shows improvements of up to 2X in power efficiency at accuracy drops as low as 0.71%.
  arXiv  Detail & Related papers  (2025-05-23T08:36:56Z)
• KNN-SSD: Enabling Dynamic Self-Speculative Decoding via Nearest Neighbor Layer Set Optimization [20.230236656479207]
  Speculative Decoding (SD) has emerged as a widely used paradigm to accelerate the inference of large language models (LLMs)<n>We introduce KNN-SSD, an algorithm that leverages K-Nearest Neighbor (KNN) search to match different skipped layers with various domain inputs.
  arXiv  Detail & Related papers  (2025-05-22T03:04:47Z)
• Dynamic layer selection in decoder-only transformers [21.18795712840146]
  We empirically examine two common dynamic inference methods for natural language generation. We find that a pre-trained decoder-only model is significantly more robust to layer removal via layer skipping. We also show that dynamic computation allocation on a per-sequence basis holds promise for significant efficiency gains.
  arXiv  Detail & Related papers  (2024-10-26T00:44:11Z)
• Progressive Mixed-Precision Decoding for Efficient LLM Inference [49.05448842542558]
  We introduce Progressive Mixed-Precision Decoding (PMPD) to address the memory-boundedness of decoding.<n>PMPD achieves 1.4$-$12.2$times$ speedup in matrix-vector multiplications over fp16 models.<n>Our approach delivers a throughput gain of 3.8$-$8.0$times$ over fp16 models and up to 1.54$times$ over uniform quantization approaches.
  arXiv  Detail & Related papers  (2024-10-17T11:46:33Z)
• DDPG-Driven Deep-Unfolding with Adaptive Depth for Channel Estimation with Sparse Bayesian Learning [23.158142411929322]
  We first develop a framework of deep deterministic policy gradient (DDPG)-driven deep-unfolding with adaptive depth for different inputs. Specifically, the framework is employed to deal with the channel estimation problem in massive multiple-input multiple-output systems.
  arXiv  Detail & Related papers  (2022-01-20T22:35:42Z)
• Targeted Attack against Deep Neural Networks via Flipping Limited Weight Bits [55.740716446995805]
  We study a novel attack paradigm, which modifies model parameters in the deployment stage for malicious purposes. Our goal is to misclassify a specific sample into a target class without any sample modification. By utilizing the latest technique in integer programming, we equivalently reformulate this BIP problem as a continuous optimization problem.
  arXiv  Detail & Related papers  (2021-02-21T03:13:27Z)
• Joint Deep Reinforcement Learning and Unfolding: Beam Selection and Precoding for mmWave Multiuser MIMO with Lens Arrays [54.43962058166702]
  millimeter wave (mmWave) multiuser multiple-input multiple-output (MU-MIMO) systems with discrete lens arrays have received great attention. In this work, we investigate the joint design of a beam precoding matrix for mmWave MU-MIMO systems with DLA.
  arXiv  Detail & Related papers  (2021-01-05T03:55:04Z)
• DyCo3D: Robust Instance Segmentation of 3D Point Clouds through Dynamic Convolution [136.7261709896713]
  We propose a data-driven approach that generates the appropriate convolution kernels to apply in response to the nature of the instances. The proposed method achieves promising results on both ScanetNetV2 and S3DIS. It also improves inference speed by more than 25% over the current state-of-the-art.
  arXiv  Detail & Related papers  (2020-11-26T14:56:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.