Depth Exploration for LLM Decoding

• URL: http://arxiv.org/abs/2606.29223v1
• Date: Sun, 28 Jun 2026 06:22:09 GMT
• Title: Depth Exploration for LLM Decoding
• Abstract summary: LLM decoding evaluates every generated token through the full layer stack, even though many tokens become predictable at intermediate depths.<n>We propose Depth Exploration Decoding (DEX) to replace single-depth selection with parallel exploration over multiple candidate depths.<n>DEX outperforms representative depth-selection baselines and achieves competitive end-to-end throughput against speculative and distributed decoding methods.
• Score: 0.920033202445524
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Autoregressive LLM decoding evaluates every generated token through the full layer stack, even though many tokens become predictable at intermediate depths. Existing lossless depth-adaptive methods exploit this redundancy by choosing a single non-final exit depth and verifying its prediction with the final-depth model. However, our measurements show that this selection-based strategy leaves substantial headroom: choosing an exit too late wastes computation, while choosing one too early triggers fallback and discards dependent drafts. We propose Depth Exploration Decoding (DEX), a lossless decoding algorithm that replaces single-depth selection with parallel exploration over multiple candidate depths. At each commit position, DEX validates candidates against the final-depth reference, commits exactly the final-depth token, and collapses the exploration lattice to retain only reusable branch states. This expand--commit--collapse procedure preserves equivalence to standard autoregressive decoding while reducing the cost of committing each token. Across early-exit-trained and standard LLMs, DEX outperforms representative depth-selection baselines and achieves competitive end-to-end throughput against speculative and distributed decoding methods. Moreover, DEX improves as the explored depths become finer, showing that parallel depth exploration provides a scalable way to exploit the underused depth axis of LLM decoding.

Related papers

• ResiSpec: Enhancing Multi-Candidate Speculative Sampling via Residual Distribution Shaping [79.1905186547363]
  ResiSpec is a framework that strategically reforms the proposal distribution during verification.<n>It prevents candidate obsolescence and achieves up to 1.92$times$ speedup over state-of-the-art multi-candidate methods.
  arXiv  Detail & Related papers  (2026-08-25T11:25:55Z)
• Ripple-Pivot Search: Active Parallel Decoding for Diffusion Large Language Models [66.32132912853372]
  Diffusion Large Language Models (dLLMs) have emerged as a competitive alternative to autoregressive language models.<n>We propose Ripple-Pivot Search (RPS), a training-free decoding method that seeks mid-entropy positions as promising candidate pivots.<n>RPS achieves 4-10$times$ wall-clock speedup over the standard decoder while preserving generation quality, and improves accuracy over the previous lookahead baseline by up to 5.49%.
  arXiv  Detail & Related papers  (2026-08-12T07:32:29Z)
• Deeper is Not Always Better: Mitigating the Alignment Tax via Confident Layer Decoding [92.1521161575198]
  We introduce Confident Decoding, a training-free decoding strategy that dynamically selects the most reliable near-final layer.<n> Experiments across dense and Mixture-of-Experts LLMs demonstrate consistent gains on challenging reasoning benchmarks.
  arXiv  Detail & Related papers  (2026-06-20T07:03:26Z)
• Zero-shot Depth Completion via Test-time Alignment with Affine-invariant Depth Prior [15.802986215292309]
  We propose a zero-shot depth completion method composed of an affine-invariant depth diffusion model and test-time alignment.<n>Our approach aligns the affine-invariant depth prior with metric-scale sparse measurements, enforcing them as hard constraints via an optimization loop at test-time.
  arXiv  Detail & Related papers  (2025-02-10T10:38:33Z)
• SemHint-MD: Learning from Noisy Semantic Labels for Self-Supervised Monocular Depth Estimation [19.229255297016635]
  Self-supervised depth estimation can be trapped in a local minimum due to the gradient-locality issue of the photometric loss. We present a framework to enhance depth by leveraging semantic segmentation to guide the network to jump out of the local minimum.
  arXiv  Detail & Related papers  (2023-03-31T17:20:27Z)
• Towards Domain-agnostic Depth Completion [28.25756709062647]
  Existing depth completion methods are often targeted at a specific sparse depth type and generalize poorly across task domains. We present a method to complete sparse/semi-dense, noisy, and potentially low-resolution depth maps obtained by various range sensors. Our method shows superior cross-domain generalization ability against state-of-the-art depth completion methods.
  arXiv  Detail & Related papers  (2022-07-29T04:10:22Z)
• Densely Constrained Depth Estimator for Monocular 3D Object Detection [48.12271792836015]
  Estimating accurate 3D locations of objects from monocular images is a challenging problem because of lacking depth. We propose a method that utilizes dense projection constraints from edges of any direction. The proposed method achieves state-of-the-art performance on the KITTI and WOD benchmarks.
  arXiv  Detail & Related papers  (2022-07-20T17:24:22Z)
• Robust Depth Completion with Uncertainty-Driven Loss Functions [60.9237639890582]
  We introduce uncertainty-driven loss functions to improve the robustness of depth completion and handle the uncertainty in depth completion. Our method has been tested on KITTI Depth Completion Benchmark and achieved the state-of-the-art robustness performance in terms of MAE, IMAE, and IRMSE metrics.
  arXiv  Detail & Related papers  (2021-12-15T05:22:34Z)
• Direct Depth Learning Network for Stereo Matching [79.3665881702387]
  A novel Direct Depth Learning Network (DDL-Net) is designed for stereo matching. DDL-Net consists of two stages: the Coarse Depth Estimation stage and the Adaptive-Grained Depth Refinement stage. We show that DDL-Net achieves an average improvement of 25% on the SceneFlow dataset and $12%$ on the DrivingStereo dataset.
  arXiv  Detail & Related papers  (2020-12-10T10:33:57Z)
• Deep feature fusion for self-supervised monocular depth prediction [7.779007880126907]
  We propose a deep feature fusion method for learning self-supervised depth from scratch. Our fusion network selects features from both upper and lower levels at every level in the encoder network. We also propose a refinement module learning higher scale residual depth from a combination of higher level deep features and lower level residual depth.
  arXiv  Detail & Related papers  (2020-05-16T09:42:36Z)
• Don't Forget The Past: Recurrent Depth Estimation from Monocular Video [92.84498980104424]
  We put three different types of depth estimation into a common framework. Our method produces a time series of depth maps. It can be applied to monocular videos only or be combined with different types of sparse depth patterns.
  arXiv  Detail & Related papers  (2020-01-08T16:50:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.