SemRep: Generative Code Representation Learning with Code Transformations

• URL: http://arxiv.org/abs/2603.13640v1
• Date: Fri, 13 Mar 2026 22:47:11 GMT
• Title: SemRep: Generative Code Representation Learning with Code Transformations
• Abstract summary: SemRep is a framework that improves code transformation through generative code representation learning.<n>Our key insight is to employ the semantics-preserving transformations as the intermediate representation.<n>With the improved exploration of diverse code transformations, SemRep is particularly amenable to evolutionary search.
• Score: 5.790824314596833
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Code transformation is a foundational capability in the software development process, where its effectiveness relies on constructing a high-quality code representation to characterize the input code semantics and guide the transformation. Existing approaches treat code transformation as an end-to-end learning task, leaving the construction of the representation needed for semantic reasoning implicit in model weights or relying on rigid compiler-level abstractions. We present SemRep, a framework that improves code transformation through generative code representation learning. Our key insight is to employ the semantics-preserving transformations as the intermediate representation, which serves as both a generative mid-training task and the guidance for subsequent instruction-specific code transformations. Across general code editing and optimization tasks (e.g., GPU kernel optimization), SemRep outperforms the extensively finetuned baselines with strictly the same training budget by 6.9% in correctness, 1.1x in performance, 13.9% in generalization, and 6.7% in robustness. With the improved exploration of diverse code transformations, SemRep is particularly amenable to evolutionary search. Combined with an evolutionary coding agent, SemRep finds optimizations that 685B larger-weight baselines fail to discover while achieving the same performance with 25% less inference compute.

Related papers

• Fool Me If You Can: On the Robustness of Binary Code Similarity Detection Models against Semantics-preserving Transformations [7.222996408214315]
  We evaluate the robustness of deep learning models for the task of binary code similarity detection.<n>We construct a dataset of 9,565 binary variants from 620 baseline samples.
  arXiv  Detail & Related papers  (2026-02-13T07:23:15Z)
• BatCoder: Self-Supervised Bidirectional Code-Documentation Learning via Back-Translation [37.23116269029009]
  BatCoder is a self-supervised reinforcement learning framework designed to jointly optimize code generation and documentation production.<n>A documentation is first generated from code, and then the generated documentation is used to reconstruct the original code.<n> evaluated on HumanEval and MBPP with a 7B model, BatCoder achieved 83.5% and 81.0% pass@1, outperforming strong open-source baselines.
  arXiv  Detail & Related papers  (2026-01-30T11:32:15Z)
• Readability-Robust Code Summarization via Meta Curriculum Learning [53.44612630063336]
  In the real world, code is often poorly structured or obfuscated, significantly degrading model performance.<n>We propose RoFTCodeSum, a novel fine-tuning method that enhances the robustness of code summarization against poorly readable code.
  arXiv  Detail & Related papers  (2026-01-09T02:38:24Z)
• ChangeGuard: Validating Code Changes via Pairwise Learning-Guided Execution [16.130469984234956]
  ChangeGuard is an approach that uses learning-guided execution to compare the runtime behavior of a modified function.<n>Our results show that the approach identifies semantics-changing code changes with a precision of 77.1% and a recall of 69.5%.
  arXiv  Detail & Related papers  (2024-10-21T15:13:32Z)
• A Unified Framework for Automated Code Transformation and Pragma Insertion [5.669524030828314]
  High-level synthesis, source-to-source compilers, and various Design Space Exploration techniques for pragma insertion have significantly improved the Quality of Results of generated designs.<n>These tools offer benefits such as reduced development time and enhanced performance.<n>However, achieving high-quality results often requires additional manual code transformations and tiling selections, which are typically performed separately or as pre-processing steps.
  arXiv  Detail & Related papers  (2024-05-05T21:41:43Z)
• LLM-Assisted Code Cleaning For Training Accurate Code Generators [53.087019724256606]
  We investigate data quality for code and find that making the code more structured and readable leads to improved code generation performance of the system. We build a novel data-cleaning pipeline that uses these principles to transform existing programs. We evaluate our approach on two challenging algorithmic code generation benchmarks and find that fine-tuning CodeLLaMa-7B improves the performance by up to 30% compared to fine-tuning on the original dataset.
  arXiv  Detail & Related papers  (2023-11-25T02:45:50Z)
• A Closer Look into Transformer-Based Code Intelligence Through Code Transformation: Challenges and Opportunities [54.039855851891815]
  Transformer-based models have demonstrated state-of-the-art performance in many intelligent coding tasks.<n>We empirically study the effect of semantic-preserving code transformation on the performance of Transformer.
  arXiv  Detail & Related papers  (2022-07-09T15:02:39Z)
• Enhancing Semantic Code Search with Multimodal Contrastive Learning and Soft Data Augmentation [50.14232079160476]
  We propose a new approach with multimodal contrastive learning and soft data augmentation for code search. We conduct extensive experiments to evaluate the effectiveness of our approach on a large-scale dataset with six programming languages.
  arXiv  Detail & Related papers  (2022-04-07T08:49:27Z)
• ReACC: A Retrieval-Augmented Code Completion Framework [53.49707123661763]
  We propose a retrieval-augmented code completion framework, leveraging both lexical copying and referring to code with similar semantics by retrieval. We evaluate our approach in the code completion task in Python and Java programming languages, achieving a state-of-the-art performance on CodeXGLUE benchmark.
  arXiv  Detail & Related papers  (2022-03-15T08:25:08Z)
• A Transformer-based Approach for Source Code Summarization [86.08359401867577]
  We learn code representation for summarization by modeling the pairwise relationship between code tokens. We show that despite the approach is simple, it outperforms the state-of-the-art techniques by a significant margin.
  arXiv  Detail & Related papers  (2020-05-01T23:29:36Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.