Skip a Layer or Loop It? Learning Program-of-Layers in LLMs

• URL: http://arxiv.org/abs/2606.06574v1
• Date: Thu, 04 Jun 2026 17:59:58 GMT
• Title: Skip a Layer or Loop It? Learning Program-of-Layers in LLMs
• Abstract summary: Large language models (LLMs) perform inference by following a fixed depth and order, non-recurrent execution of all layers.<n>We reveal the wide existence of training-free, flexible, dynamic program-of-layers (PoLar)<n>We propose a lightweight PoLar prediction network, which learns to generate execution programs that dynamically skip or repeat pretrained layers for each input.
• Score: 22.794776940675828
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Large language models (LLMs) perform inference by following a fixed depth and order, non-recurrent execution of all layers. We reveal the wide existence of training-free, flexible, dynamic program-of-layers (PoLar), where pretrained layers can be packed as modules and then skipped or looped to form a customized program for each input. For most inputs, substantially shorter program executions can achieve the same or better accuracy, while incorrect predictions of the original LLM can be corrected by alternative programs with fewer layers. These observations indicate that inference admits multiple valid latent computations beyond the standard forward pass. To efficiently achieve PoLar in practice, we propose a lightweight PoLar prediction network, which learns to generate execution programs that dynamically skip or repeat pretrained layers for each input. Experiments on mathematical reasoning benchmarks demonstrate that PoLar consistently improves accuracy over standard inference and prior dynamic-depth methods, often while executing fewer layers, and that these gains persist under out-of-distribution evaluation. Our results suggest that fixed-depth execution captures only a narrow subset of an LLM's latent reasoning capacity.

Related papers

• LLM-ERM: Sample-Efficient Program Learning via LLM-Guided Search [23.97383442759484]
  LLM-ERM is a propose-and-verify framework that replaces exhaustive enumeration with an LLM-guided search over candidate programs.<n>We show that coordinate-wise online mini-batch SGD requires many samples to learn certain short programs.<n>These results indicate that language-guided program synthesis recovers much of the statistical efficiency of finite-class ERM.
  arXiv  Detail & Related papers  (2025-10-16T06:10:11Z)
• Skip a Layer or Loop it? Test-Time Depth Adaptation of Pretrained LLMs [21.541258368039955]
  We find that layers of a pretrained large language model (LLM) can be manipulated as separate modules to build a better and even shallower model customized for each test sample.<n>In particular, each layer from the pretrained model can be skipped/pruned or repeated multiple times as recurrent neural networks (RNN), and stacked with others in arbitrary orders, yielding a chain-of-layers (CoLa) per sample.
  arXiv  Detail & Related papers  (2025-07-10T17:59:53Z)
• Can Prompt Difficulty be Online Predicted for Accelerating RL Finetuning of Reasoning Models? [65.18157595903124]
  This work investigates iterative approximate evaluation for arbitrary prompts.<n>It introduces Model Predictive Prompt Selection (MoPPS), a Bayesian risk-predictive framework.<n>MoPPS reliably predicts prompt difficulty and accelerates training with significantly reduced rollouts.
  arXiv  Detail & Related papers  (2025-07-07T03:20:52Z)
• Towards Effectively Leveraging Execution Traces for Program Repair with Code LLMs [13.708569727719434]
  Large Language Models (LLMs) show promising performance on various programming tasks.<n>We aim to remedy this potential blind spot by augmenting standard APR prompts with program execution traces.
  arXiv  Detail & Related papers  (2025-05-07T14:12:41Z)
• LESA: Learnable LLM Layer Scaling-Up [57.0510934286449]
  Training Large Language Models (LLMs) from scratch requires immense computational resources, making it prohibitively expensive.<n>Model scaling-up offers a promising solution by leveraging the parameters of smaller models to create larger ones.<n>We propose textbfLESA, a novel learnable method for depth scaling-up.
  arXiv  Detail & Related papers  (2025-02-19T14:58:48Z)
• AlphaPruning: Using Heavy-Tailed Self Regularization Theory for Improved Layer-wise Pruning of Large Language Models [94.82766517752418]
  We propose AlphaPruning, which uses shape metrics to allocate layerwise sparsity ratios in a more theoretically principled manner. Our results show that AlphaPruning prunes LLaMA-7B to 80% sparsity while maintaining reasonable perplexity, marking a first in the literature on LLMs.
  arXiv  Detail & Related papers  (2024-10-14T03:35:11Z)
• Not All Layers of LLMs Are Necessary During Inference [68.88671495401483]
  We show that for some tasks, Large Language Models can achieve results comparable to the final output at some intermediate layers. We propose a simple yet effective algorithm named AdaInfer to adaptively terminate the inference process for an input instance.
  arXiv  Detail & Related papers  (2024-03-04T16:23:58Z)
• Dynamic Sparse No Training: Training-Free Fine-tuning for Sparse LLMs [67.38165028487242]
  We introduce Dynamic Sparse No Training (DSnoT), a training-free fine-tuning approach to fine-tune large language models (LLMs) Inspired by the Dynamic Sparse Training, DSnoT minimizes the reconstruction error between the dense and sparse LLMs. Our paper offers fresh insights into how to fine-tune sparse LLMs in an efficient training-free manner and open new venues to scale the great potential of sparsity to LLMs.
  arXiv  Detail & Related papers  (2023-10-13T07:38:52Z)
• Joint Prompt Optimization of Stacked LLMs using Variational Inference [66.04409787899583]
  Large language models (LLMs) can be seen as atomic units of computation mapping sequences to a distribution over sequences. By stacking two such layers and feeding the output of one layer to the next, we obtain a Deep Language Network (DLN) We show that DLN-2 can reach higher performance than a single layer, showing promise that we might reach comparable performance to GPT-4.
  arXiv  Detail & Related papers  (2023-06-21T18:45:56Z)
• Investigating the Effectiveness of Task-Agnostic Prefix Prompt for Instruction Following [44.701091969256055]
  We present our finding that prepending a Task-Agnostic Prefix Prompt (TAPP) to the input improves the instruction-following ability of various Large Language Models (LLMs) during inference. We observe that both base LLMs (i.e. not fine-tuned to follow instructions) and instruction-tuned models benefit from TAPP, resulting in 34.58% and 12.26% improvement on average.
  arXiv  Detail & Related papers  (2023-02-28T16:06:35Z)
• Fault-Aware Neural Code Rankers [64.41888054066861]
  We propose fault-aware neural code rankers that can predict the correctness of a sampled program without executing it. Our fault-aware rankers can significantly increase the pass@1 accuracy of various code generation models.
  arXiv  Detail & Related papers  (2022-06-04T22:01:05Z)
• ReLU activated Multi-Layer Neural Networks trained with Mixed Integer Linear Programs [0.0]
  multilayer feedforward neural networks activated by ReLU functions can be trained iteratively with Mixed Linear Programs (MILPs) Algorithm starts at the output layer and propagates information back to the first hidden layer to adjust the weights using MILPs or Linear Programs. The method was tested and compared withflow/Keras (Adam) using two simple networks on the MNIST dataset containing handwritten digits.
  arXiv  Detail & Related papers  (2020-08-19T11:42:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.