Understanding Binary Code Similarity for Real-World Vulnerability Detection: A Large-Scale Empirical Study

• URL: http://arxiv.org/abs/2606.28870v1
• Date: Sat, 27 Jun 2026 11:26:28 GMT
• Title: Understanding Binary Code Similarity for Real-World Vulnerability Detection: A Large-Scale Empirical Study
• Abstract summary: We conduct a large-scale study of vulnerability detection across 60,000 firmware images from 200 vendors using BCSD.<n>We examine the influence of four key factors -- vulnerable function versions, vulnerability search space, function sizes, and compilation toolchains on BCSD performance.<n>A TPL-aware, two-stage search process significantly enhances accuracy, improving MRR by 18.5% by limiting the search space.
• Score: 15.569141472526502
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Firmware lies at the heart of IoT devices. Its development depends heavily on third-party libraries (TPLs), which greatly accelerate the process but simultaneously introduce associated vulnerabilities. Binary Code Similarity Detection (BCSD) is an effective technique for identifying vulnerabilities in firmware by comparing pairs of code segments. However, existing studies either evaluate their performance only on small-scale datasets or lack diversity in terms of vulnerabilities, TPLs, and firmware. Consequently, a comprehensive understanding of BCSD for real-world vulnerability detection remains absent. To bridge this gap, we conduct a large-scale study of vulnerability detection across 60,000 firmware images from 200 vendors using BCSD. Rather than introducing a novel model, we examine the influence of four key factors -- vulnerable function versions, vulnerability search space, function sizes, and compilation toolchains on BCSD performance. Our results reveal that these factors substantially affect performance, often by wide margins. To address this, we propose a build-aware query strategy that derives queries from representative real-world binaries, effectively closing the gap and raising the mean reciprocal rank (MRR) from 0.818 to 0.981. Furthermore, we demonstrate that a TPL-aware, two-stage search process significantly enhances accuracy, improving MRR by 18.5\% by limiting the search space.

Related papers

• Statistical Analysis of Executability and Program Equivalence in Decompilation for IoT Vulnerability Detection [1.9637266103693347]
  We propose a nine-dimensional quality evaluation metric consisting of three categories: structural, behavioral, and semantic similarity.<n>Vulnerabilities often arise in details that are easily lost in this process, such as error-handling flows and boundary checks.<n>This study provides a statistical evaluation foundation for quantifying implementation defects in IoT devices.
  arXiv  Detail & Related papers  (2026-08-07T08:36:35Z)
• RealSec-bench: A Benchmark for Evaluating Secure Code Generation in Real-World Repositories [58.32028251925354]
  Large Language Models (LLMs) have demonstrated remarkable capabilities in code generation, but their proficiency in producing secure code remains a critical, under-explored area.<n>We introduce RealSec-bench, a new benchmark for secure code generation meticulously constructed from real-world, high-risk Java repositories.
  arXiv  Detail & Related papers  (2026-01-30T08:29:01Z)
• A Large Scale Study of AI-based Binary Function Similarity Detection Techniques for Security Researchers and Practitioners [20.470574733047442]
  Binary Function Similarity Detection (BFSD) is a foundational technique in software security.<n>Recent advances in AI-based BFSD tools have led to significant performance improvements.<n>We present the first large-scale empirical study of AI-based BFSD tools to address these gaps.
  arXiv  Detail & Related papers  (2025-11-03T03:10:25Z)
• DiffuGuard: How Intrinsic Safety is Lost and Found in Diffusion Large Language Models [50.21378052667732]
  We conduct an in-depth analysis of dLLM vulnerabilities to jailbreak attacks across two distinct dimensions: intra-step and inter-step dynamics.<n>We propose DiffuGuard, a training-free defense framework that addresses vulnerabilities through a dual-stage approach.
  arXiv  Detail & Related papers  (2025-09-29T05:17:10Z)
• Ensembling Large Language Models for Code Vulnerability Detection: An Empirical Evaluation [69.8237598448941]
  This study investigates the potential of ensemble learning to enhance the performance of Large Language Models (LLMs) in source code vulnerability detection.<n>We propose Dynamic Gated Stacking (DGS), a Stacking variant tailored for vulnerability detection.
  arXiv  Detail & Related papers  (2025-09-16T03:48:22Z)
• VulnRepairEval: An Exploit-Based Evaluation Framework for Assessing Large Language Model Vulnerability Repair Capabilities [41.85494398578654]
  VulnRepairEval is an evaluation framework anchored in functional Proof-of-Concept exploits.<n>Our framework delivers a comprehensive, containerized evaluation pipeline that enables reproducible differential assessment.
  arXiv  Detail & Related papers  (2025-09-03T14:06:10Z)
• VulBinLLM: LLM-powered Vulnerability Detection for Stripped Binaries [4.1417640577742425]
  Vul-BinLLM is a framework for binary vulnerability detection using Large Language Models.<n>Vul-BinLLM mirrors traditional binary analysis with fine-grained optimizations in decompilation and vulnerability reasoning with an extended context.<n>Our evaluations show that Vul-BinLLM is highly effective in detecting vulnerabilities on the compiled Juliet dataset.
  arXiv  Detail & Related papers  (2025-05-28T06:17:56Z)
• Benchmarking LLMs and LLM-based Agents in Practical Vulnerability Detection for Code Repositories [8.583591493627276]
  We introduce JitVul, a vulnerability detection benchmark linking each function to its vulnerability-introducing and fixing commits.<n>We show that ReAct Agents, leveraging thought-action-observation and interprocedural context, perform better than LLMs in distinguishing vulnerable from benign code.
  arXiv  Detail & Related papers  (2025-03-05T15:22:24Z)
• Binary Code Similarity Detection via Graph Contrastive Learning on Intermediate Representations [52.34030226129628]
  Binary Code Similarity Detection (BCSD) plays a crucial role in numerous fields, including vulnerability detection, malware analysis, and code reuse identification. In this paper, we propose IRBinDiff, which mitigates compilation differences by leveraging LLVM-IR with higher-level semantic abstraction. Our extensive experiments, conducted under varied compilation settings, demonstrate that IRBinDiff outperforms other leading BCSD methods in both One-to-one comparison and One-to-many search scenarios.
  arXiv  Detail & Related papers  (2024-10-24T09:09:20Z)
• The Impact of SBOM Generators on Vulnerability Assessment in Python: A Comparison and a Novel Approach [56.4040698609393]
  Software Bill of Materials (SBOM) has been promoted as a tool to increase transparency and verifiability in software composition. Current SBOM generation tools often suffer from inaccuracies in identifying components and dependencies. We propose PIP-sbom, a novel pip-inspired solution that addresses their shortcomings.
  arXiv  Detail & Related papers  (2024-09-10T10:12:37Z)
• Asteria-Pro: Enhancing Deep-Learning Based Binary Code Similarity Detection by Incorporating Domain Knowledge [8.93208472340743]
  We propose a novel deep learning enhancement architecture by incorporating domain knowledge-based pre-filtration and re-ranking modules. Asteria-Pro manages to detect 1,482 vulnerable functions with a high precision 91.65%.
  arXiv  Detail & Related papers  (2023-01-02T03:16:26Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.