Instruction Alignment for Binary Code Representation Learning

• URL: http://arxiv.org/abs/2608.11766v1
• Date: Wed, 12 Aug 2026 08:07:26 GMT
• Title: Instruction Alignment for Binary Code Representation Learning
• Abstract summary: We propose to leverage instruction alignment knowledge to improve binary code representation learning.<n>Our preliminary study reveals that models finetuned for function-level binary code similarity exhibit substantially better instruction alignment than their pre-trained model.
• Score: 6.28246120991306
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Binary code representation learning is a fundamental problem in software security and reverse engineering. Existing methods mainly learn function-level embeddings that capture coarse-grained semantic relationships between binary functions, but they largely ignore fine-grained instruction-level correspondences. This limitation misses valuable supervision signals available from compiler debug information, which can support the learning of more accurate and interpretable binary code representations. We propose to leverage instruction alignment knowledge to further improve binary code representation learning. Our preliminary study reveals that models finetuned for function-level binary code similarity exhibit substantially better instruction alignment than their pre-trained model, suggesting a strong correlation between instruction alignment and function-level embedding quality. Motivated by this observation, we design a training approach that explicitly incorporates instruction alignment as an auxiliary training objective. Our experiments show that instruction alignment training improves retrieval accuracy and provides more discriminative signal for the model's similarity judgments.

Related papers

• Readability-Robust Code Summarization via Meta Curriculum Learning [53.44612630063336]
  In the real world, code is often poorly structured or obfuscated, significantly degrading model performance.<n>We propose RoFTCodeSum, a novel fine-tuning method that enhances the robustness of code summarization against poorly readable code.
  arXiv  Detail & Related papers  (2026-01-09T02:38:24Z)
• CodeBoost: Boosting Code LLMs by Squeezing Knowledge from Code Snippets with RL [28.43882967593511]
  Code large language models (LLMs) have become indispensable tools for building efficient and automated coding pipelines.<n>Existing models are typically post-trained using reinforcement learning (RL) from general-purpose LLMs using "human instruction-final answer" pairs.<n>We propose CodeBoost, a framework that enhances code LLMs purely from code snippets, without relying on human-annotated instructions.
  arXiv  Detail & Related papers  (2025-08-07T10:31:24Z)
• An Empirical Study on the Effectiveness of Large Language Models for Binary Code Understanding [50.17907898478795]
  This work proposes a benchmark to evaluate the effectiveness of Large Language Models (LLMs) in real-world reverse engineering scenarios.<n>Our evaluations reveal that existing LLMs can understand binary code to a certain extent, thereby improving the efficiency of binary code analysis.
  arXiv  Detail & Related papers  (2025-04-30T17:02:06Z)
• On the Role of Pre-trained Embeddings in Binary Code Analysis [7.161446721947512]
  Pre-trained embeddings of assembly code have become a gold standard for solving binary code analysis tasks.<n>In contrast to natural language processing, label information is not scarce for many tasks in binary code analysis.<n>We systematically evaluate recent embeddings for assembly code on five downstream tasks using a corpus of 1.2 million functions.
  arXiv  Detail & Related papers  (2025-02-12T10:50:46Z)
• LLM Critics Help Catch Bugs in Mathematics: Towards a Better Mathematical Verifier with Natural Language Feedback [71.95402654982095]
  We propose Math-Minos, a natural language feedback-enhanced verifier. Our experiments reveal that a small set of natural language feedback can significantly boost the performance of the verifier.
  arXiv  Detail & Related papers  (2024-06-20T06:42:27Z)
• CLAP: Learning Transferable Binary Code Representations with Natural Language Supervision [22.42846252594693]
  We present CLAP (Contrastive Language-Assembly Pre-training), which employs natural language supervision to learn better representations of binary code. At the core, our approach boosts superior transfer learning capabilities by effectively aligning binary code with their semantics explanations. We have generated 195 million pairs of binary code and explanations and trained a prototype of CLAP.
  arXiv  Detail & Related papers  (2024-02-26T13:49:52Z)
• Code Representation Learning At Scale [75.04686476303436]
  We fuel code representation learning with a vast amount of code data via a two-stage pretraining scheme. We first train the encoders via a mix that leverages both randomness in masking language modeling and the structure aspect of programming language. We then enhance the representations via contrastive learning with hard negative and hard positive constructed in an unsupervised manner.
  arXiv  Detail & Related papers  (2024-02-02T22:19:15Z)
• Soft-Labeled Contrastive Pre-training for Function-level Code Representation [127.71430696347174]
  We present textbfSCodeR, a textbfSoft-labeled contrastive pre-training framework with two positive sample construction methods. Considering the relevance between codes in a large-scale code corpus, the soft-labeled contrastive pre-training can obtain fine-grained soft-labels. SCodeR achieves new state-of-the-art performance on four code-related tasks over seven datasets.
  arXiv  Detail & Related papers  (2022-10-18T05:17:37Z)
• Pre-Training Representations of Binary Code Using Contrastive Learning [20.467504340485856]
  ContraBin is a contrastive learning technique that integrates source code and comment information along with binaries.<n>We analyze the impact of human-written and synthetic comments on binary code comprehension tasks.
  arXiv  Detail & Related papers  (2022-10-11T02:39:06Z)
• SimCLF: A Simple Contrastive Learning Framework for Function-level Binary Embeddings [2.1222884030559315]
  We propose SimCLF: A Simple Contrastive Learning Framework for Function-level Binary Embeddings. We take an unsupervised learning approach and formulate binary code similarity detection as instance discrimination. SimCLF directly operates on disassembled binary functions and could be implemented with any encoder.
  arXiv  Detail & Related papers  (2022-09-06T12:09:45Z)
• Auto-Encoding Twin-Bottleneck Hashing [141.5378966676885]
  This paper proposes an efficient and adaptive code-driven graph. It is updated by decoding in the context of an auto-encoder. Experiments on benchmarked datasets clearly show the superiority of our framework over the state-of-the-art hashing methods.
  arXiv  Detail & Related papers  (2020-02-27T05:58:12Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.