Chiseling Out Efficiency: Structured Skeleton Supervision for Efficient Code Generation

• URL: http://arxiv.org/abs/2606.06821v1
• Date: Fri, 05 Jun 2026 01:49:20 GMT
• Title: Chiseling Out Efficiency: Structured Skeleton Supervision for Efficient Code Generation
• Abstract summary: Large Language Models (LLMs) are capable of generating syntactically correct and functionally complete programs, greatly streamlining software development.<n>Existing approaches to bridging this efficiency gap typically involve either iteratively optimizing code after generation or fine-tuning models on corpora of efficient code.<n>EffiSkel is an efficiency skeleton-guided framework that explicitly extracts and learns efficiency skeletons-abstract, reusable structural patterns efficient code.
• Score: 61.72280804975896
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Large Language Models (LLMs) are capable of generating syntactically correct and functionally complete programs, greatly streamlining software development. However, recent studies reveal that these programs typically execute substantially slower than human-optimized counterparts. Existing approaches to bridging this efficiency gap typically involve either iteratively optimizing code after generation or fine-tuning models on corpora of efficient code. Yet, these methods expose the model to efficiency signals only by mimicking complete, optimized solutions, without explicitly encoding the structural code patterns essential for achieving high runtime performance. Addressing this gap presents two core challenges: (1) extracting and representing latent, efficiency-oriented structural patterns embedded within complex syntax and control flows, and (2) effectively learning these patterns without destabilizing the semantic training of LLMs. To tackle these challenges, we propose EffiSkel, an efficiency skeleton-guided framework that explicitly extracts and learns efficiency skeletons-abstract, reusable structural patterns underpinning efficient code-by leveraging three complementary strategies. These skeletons are integrated into a multi-task learning regime that jointly optimizes code generation and skeleton prediction. Experiments across multiple programming languages and benchmarks demonstrate that EffiSkel significantly enhances both functional correctness and efficiency, resulting on Mercury with DeepSeek-Coder (7B) a +11.11% (vs. EffiCoder) and +3.71% (vs. CodeDPO) higher Efficiency Ratio (ER), and a +0.36 (vs. EffiCoder) and +0.22 (vs. CodeDPO) increase in Average Speedup (AS). These results highlight the effectiveness of explicitly modeling efficiency skeletons in improving the runtime performance of code generated by LLMs.

Related papers

• SkelDPO: A Skeleton-Guided Direct Preference Optimization Framework for Efficient Code Generation [4.9858352320162]
  SkelDPO is a skeleton-guided preference optimization framework.<n>It identifies efficient and inefficient implementations from a code dataset.<n>During training, a joint code and skeleton preference loss is introduced, enabling the model to learn semantic correctness.
  arXiv  Detail & Related papers  (2026-06-05T02:01:38Z)
• EffiSkill: Agent Skill Based Automated Code Efficiency Optimization [18.68243664364758]
  EffiSkill is a framework for code-efficiency optimization that builds a portable optimization toolbox for LLM-based agents.<n>EffiSkill adopts a two-stage design: Stage I mines Operator and Meta Skills from large-scale slow/fast program pairs to build a skill library.<n>Stage II applies this library to unseen programs through execution-free diagnosis, skill retrieval, plan composition, and candidate generation.
  arXiv  Detail & Related papers  (2026-03-29T20:09:57Z)
• An Initial Exploration of Contrastive Prompt Tuning to Generate Energy-Efficient Code [46.879983975894135]
  This study aims to investigate whether and how LLMs can be optimized to promote the generation of energy-efficient code.<n>CPT combines Contrastive Learning techniques, which help the model to distinguish between efficient and inefficient code.<n>This study evaluates CPT on Python, Java and C++ coding problems across three different models to provide a comprehensive evaluation.
  arXiv  Detail & Related papers  (2026-03-03T12:36:15Z)
• ShortCoder: Knowledge-Augmented Syntax Optimization for Token-Efficient Code Generation [27.9837392531619]
  We propose a knowledge-infused framework named ShortCoder to optimize code generation efficiency.<n>ShortCoder consistently outperforms state-of-the-art methods on HumanEval.
  arXiv  Detail & Related papers  (2026-01-14T18:57:31Z)
• WebLeaper: Empowering Efficiency and Efficacy in WebAgent via Enabling Info-Rich Seeking [60.35109192765302]
  Information seeking is a core capability that enables autonomous reasoning and decision-making.<n>We propose WebLeaper, a framework for constructing high-coverage IS tasks and generating efficient solution trajectories.<n>Our method consistently achieves improvements in both effectiveness and efficiency over strong baselines.
  arXiv  Detail & Related papers  (2025-10-28T17:51:42Z)
• Towards Better Correctness and Efficiency in Code Generation [47.06216040246783]
  We propose an efficiency-oriented reinforcement learning framework guided by a novel performance reward.<n>Online exploration is most effective when starting from a high-correctness baseline.<n>Experiments show the effectiveness of the method, which improves code correctness by 10.18% and runtime efficiency by 7.75% on a 7B model.
  arXiv  Detail & Related papers  (2025-08-24T16:47:19Z)
• Afterburner: Reinforcement Learning Facilitates Self-Improving Code Efficiency Optimization [46.33639431414019]
  Large Language Models generate functionally correct solutions but often fall short in code efficiency.<n>We introduce a novel test-time iterative optimization framework to address this.
  arXiv  Detail & Related papers  (2025-05-29T12:14:29Z)
• EfficientFCN: Holistically-guided Decoding for Semantic Segmentation [49.27021844132522]
  State-of-the-art semantic segmentation algorithms are mostly based on dilated Fully Convolutional Networks (dilatedFCN) We propose the EfficientFCN, whose backbone is a common ImageNet pre-trained network without any dilated convolution. Such a framework achieves comparable or even better performance than state-of-the-art methods with only 1/3 of the computational cost.
  arXiv  Detail & Related papers  (2020-08-24T14:48:23Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.