Statistical Analysis of Executability and Program Equivalence in Decompilation for IoT Vulnerability Detection

• URL: http://arxiv.org/abs/2608.06960v1
• Date: Fri, 07 Aug 2026 08:36:35 GMT
• Title: Statistical Analysis of Executability and Program Equivalence in Decompilation for IoT Vulnerability Detection
• Abstract summary: We propose a nine-dimensional quality evaluation metric consisting of three categories: structural, behavioral, and semantic similarity.<n>Vulnerabilities often arise in details that are easily lost in this process, such as error-handling flows and boundary checks.<n>This study provides a statistical evaluation foundation for quantifying implementation defects in IoT devices.
• Score: 1.9637266103693347
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Internet of Things (IoT) devices handle sensitive privacy-related information such as user audio, video, and authentication data, making it essential to detect vulnerabilities in their firmware. Decompilation, a key detection technique, has recently attracted attention because Large Language Models (LLMs) enable high readability and high recompilation success rates. However, because LLM outputs depend on probabilistic token prediction, they tend to prioritize syntactic correctness and may generate plausible-looking code that is semantically different from the original binary. Vulnerabilities often arise in details that are easily lost in this process, such as error-handling flows and boundary checks. Existing evaluation metrics focus mainly on passing test cases and cannot sufficiently identify code whose internal structure has been altered despite appearing behaviorally valid, so a metric that quantifies the internal structure of decompiled code from multiple perspectives is needed. We propose a nine-dimensional quality evaluation metric consisting of three categories: structural, behavioral, and semantic similarity. Targeting 318 programs from OpenWrt, an open-source router platform underlying many commercial routers, we generated 19,625 decompilation results using five methods (one rule-based and four LLM-based) and analyzed them statistically. The recompilation-success group achieved significantly higher overall scores than the failure group (Cohen's d=0.92); behavioral similarity showed d=0.96 and structural similarity d=0.69, demonstrating that these metrics are important predictors of decompilation quality. This study provides a statistical evaluation foundation for quantifying implementation defects in IoT devices and a framework that generalizes to quality evaluation of black-box generative models.

Related papers

• Conformal Changepoint Localization and Root Cause Analysis with Corrupted Observations [55.76952683833966]
  This paper proposes weighted CONCH (W-CONCH) and weighted CROC (W-CROC) to reduce confidence set size when data may be corrupted.<n>Experiments on image-based and real-world changepoint and root-cause benchmarks show that uncertainty-based weighting substantially reduces confidence set size.
  arXiv  Detail & Related papers  (2026-07-29T05:16:59Z)
• DREA: Decoupled Reasoning and Exploration Agents for Repository-Level Vulnerability Detection [16.007199928732614]
  We present DREA, a hypothesis-driven framework for repository-level vulnerability detection.<n>Goal-directed context acquisition is the primary source of detection improvement in this design.<n>To support evaluation, we construct RepoPairBench, a repository-grounded benchmark of validated Python vulnerability-fix pairs from real-world projects.
  arXiv  Detail & Related papers  (2026-07-15T04:49:05Z)
• Understanding Binary Code Similarity for Real-World Vulnerability Detection: A Large-Scale Empirical Study [15.569141472526502]
  We conduct a large-scale study of vulnerability detection across 60,000 firmware images from 200 vendors using BCSD.<n>We examine the influence of four key factors -- vulnerable function versions, vulnerability search space, function sizes, and compilation toolchains on BCSD performance.<n>A TPL-aware, two-stage search process significantly enhances accuracy, improving MRR by 18.5% by limiting the search space.
  arXiv  Detail & Related papers  (2026-06-27T11:26:28Z)
• Measuring Model Robustness via Fisher Information: Spectral Bounds, Theoretical Guarantees, and Practical Algorithms [18.10438822471092]
  robustness of deep neural networks is crucial for safety-critical deployments.<n>We propose a principled, attack-agnostic robustness metric based on the spectral norm of the Fisher Information Matrix.<n>Our framework serves as an interpretable diagnostic tool that complements attack-based evaluations.
  arXiv  Detail & Related papers  (2026-06-03T11:50:38Z)
• Dynamic analysis enhances issue resolution [53.50448142467294]
  DAIRA (Dynamic Analysis-enhanced Issue Resolution Agent) is an automated repair framework that embeds dynamic analysis into the agent's reasoning cycle.<n> Driven by a Test Tracing-Driven methodology, DAIRA utilizes lightweight monitors to extract critical runtime data.<n>When powered by Gemini 3 Flash Preview, DAIRA establishes a new state-of-the-art (SOTA) performance, achieving a 79.4% resolution rate on the SWE-bench Verified dataset.
  arXiv  Detail & Related papers  (2026-03-23T14:48:54Z)
• Automated Self-Testing as a Quality Gate: Evidence-Driven Release Management for LLM Applications [51.56484100374058]
  We present an automated self-testing framework that introduces quality gates with evidence-based release decisions.<n>We evaluate the framework through a longitudinal case study of an internally deployed multi-agent conversational AI system.
  arXiv  Detail & Related papers  (2026-03-13T20:44:15Z)
• Evaluating Line-level Localization Ability of Learning-based Code Vulnerability Detection Models [9.543689542888599]
  We propose an explainability-based evaluation procedure for vulnerability detectors.<n>Our approach, defined as Detection Alignment (DA), quantifies the agreement between the input source code lines.<n>We show how the predictions of such models are consistently biased by non-vulnerable lines.
  arXiv  Detail & Related papers  (2025-10-13T09:34:40Z)
• Ensembling Large Language Models for Code Vulnerability Detection: An Empirical Evaluation [69.8237598448941]
  This study investigates the potential of ensemble learning to enhance the performance of Large Language Models (LLMs) in source code vulnerability detection.<n>We propose Dynamic Gated Stacking (DGS), a Stacking variant tailored for vulnerability detection.
  arXiv  Detail & Related papers  (2025-09-16T03:48:22Z)
• SecVulEval: Benchmarking LLMs for Real-World C/C++ Vulnerability Detection [8.440793630384546]
  Large Language Models (LLMs) have shown promise in software engineering tasks.<n> evaluating their effectiveness in vulnerability detection is challenging due to the lack of high-quality datasets.<n>This benchmark includes 25,440 function samples covering 5,867 unique CVEs in C/C++ projects from 1999 to 2024.
  arXiv  Detail & Related papers  (2025-05-26T11:06:03Z)
• Reasoning with LLMs for Zero-Shot Vulnerability Detection [0.9208007322096533]
  We present textbfVulnSage, a comprehensive evaluation framework and a curated dataset from diverse, large-scale open-source system software projects.<n>The framework supports multi-granular analysis across function, file, and inter-function levels.<n>It employs four diverse zero-shot prompt strategies: Baseline, Chain-of-context, Think, and Think & verify.
  arXiv  Detail & Related papers  (2025-03-22T23:59:17Z)
• Lie Detector: Unified Backdoor Detection via Cross-Examination Framework [68.45399098884364]
  We propose a unified backdoor detection framework in the semi-honest setting.<n>Our method achieves superior detection performance, improving accuracy by 5.4%, 1.6%, and 11.9% over SoTA baselines.<n> Notably, it is the first to effectively detect backdoors in multimodal large language models.
  arXiv  Detail & Related papers  (2025-03-21T06:12:06Z)
• SeCodePLT: A Unified Platform for Evaluating the Security of Code GenAI [58.29510889419971]
  Existing benchmarks for evaluating the security risks and capabilities of code-generating large language models (LLMs) face several key limitations.<n>We introduce a general and scalable benchmark construction framework that begins with manually validated, high-quality seed examples and expands them via targeted mutations.<n>Applying this framework to Python, C/C++, and Java, we build SeCodePLT, a dataset of more than 5.9k samples spanning 44 CWE-based risk categories and three security capabilities.
  arXiv  Detail & Related papers  (2024-10-14T21:17:22Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.