The Deception Delta: Adversarial Evaluation of LLM-Based Smart Contract Bytecode Forensics

• URL: http://arxiv.org/abs/2609.14098v1
• Date: Sat, 12 Sep 2026 18:45:24 GMT
• Title: The Deception Delta: Adversarial Evaluation of LLM-Based Smart Contract Bytecode Forensics
• Abstract summary: Large language models are increasingly used in blockchain forensic investigations.<n> adversarial deception reduces drain detection by 20.0 percentage points.<n>Models correctly describe the hidden drain mechanism but accept the contract's framing and dismiss it as benign.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Large language models are increasingly used in blockchain forensic investigations to interpret unverified smart contract bytecode. Their robustness has not been systematically tested against contracts adversarially designed to mislead analysis. We evaluate 22 frontier models on 13 purpose-built contracts (9 deception vectors, 4 controls) across six prompt strategies, yielding 8,528 analyzable non-refusal runs against contracts with EVM-verified ground truth. A calibrated LLM-as-judge pipeline, supported by two judge-independent metrics and 50 human gold-standard labels, shows that adversarial deception reduces drain detection by 20.0 percentage points (95% CI: [17.2, 22.8]) relative to functionally matched controls. Structural camouflage via multi-hop call chains, XOR-masked selectors, and storage-loaded drain parameters resists detection across nearly all models. Beyond non-detection, we identify rationalization: models correctly describe the hidden drain mechanism but accept the contract's deceptive framing and dismiss it as benign, yielding positive but incorrect evidence of safety. Simple guard instructions provide no aggregate benefit and destabilize individual models in both directions. Structural deception is largely insensitive across the six tested prompt strategies, more consistent with a capability limitation than with a simple prompting problem. Only five models from two providers exceed 50% detection. Under our single-shot, raw-bytecode-only protocol, current LLMs are not reliable standalone forensic tools. Our central claim does not extend to multi-turn, tool-augmented, source-aware, or decompiler-in-the-loop workflows; a source-code boundary check is reported as an explicit subset analysis rather than as part of the main evaluation.

Related papers

• Recognition Without Enforcement: Configuration-Dependent Failures in LLM Agent Instruction Arbitration and External Control [0.0]
  LLM agents arbitrate among instructions from system prompts, users, memory, and tools, but this arbitration cannot be assumed to enforce trust boundaries.<n>We identify a recognition-enforcement gap: source-format features are linearly decodable from model activations, and models can explicitly identify forged authority when prompted.<n>We treat model self-arbitration as a capability rather than a security boundary and implement an external reference monitor combining authenticated source routing with capability-gated tool execution.
  arXiv  Detail & Related papers  (2026-08-28T16:34:05Z)
• Fantastic Adaptive Taxonomies and How to Use Them [100.20614173157708]
  AdaMAST converts execution traces into a compact, evidence-grounded failure taxonomy.<n>No code is hand-authored, no trace human-annotated.<n>In agent-system search, taxonomy-coded diagnoses of failed candidates outperform free-form.
  arXiv  Detail & Related papers  (2026-07-17T17:41:16Z)
• Real-Time Source-Free Object Detection [40.10658212485104]
  Real-world detectors for autonomous driving, surveillance, and robotics must handle domain-shifts under strict latency and memory constraints.<n>We show that building on YOLOv10, an NMS-free dual-head detector, we achieve state-of-the-art adaptation accuracy while being faster and more compact.
  arXiv  Detail & Related papers  (2026-06-30T15:38:56Z)
• SNARE: Adaptive Scenario Synthesis for Eliciting Overeager Behavior in Coding Agents [40.270213696031625]
  A coding agent executes a benign task as a sequence of shell, file, and network actions, any of which can quietly exceed the authorized scope while the task still completes.<n>We call this overeager behavior: the prompt is not adversarial and the run succeeds, yet an out-of-scope step can leak credentials or delete files.<n>Existing benchmarks miss it: task-completion suites credit any finished run, jailbreak suites probe adversarial prompts, and the one prior overeager benchmark applies a single fixed prompt to every agent-model pair.<n>We present SNARE, a pipeline that composes benign scenarios from reusable scope
  arXiv  Detail & Related papers  (2026-05-27T08:14:07Z)
• Distributional Energy-Based Models for Uncertainty-Aware Structured LLM Reasoning [40.342912574072024]
  Large Language Models produce structured outputs such as travel plans or code solutions.<n>Individual reasoning steps may appear correct while the output as a whole violates budgets, fails test cases, or contradicts earlier deductions.<n>We propose a learned quality scorer with deterministic analytical constraint penalties for verifying structured LLM outputs.
  arXiv  Detail & Related papers  (2026-05-15T17:08:27Z)
• GAMBIT: A Three-Mode Benchmark for Adversarial Robustness in Multi-Agent LLM Collectives [48.545980031973556]
  GAMBIT is a benchmark with three evaluation modes and two independent scores for evaluating imposter detectors.<n>The benchmark comes with a dataset of 27,804 labeled instances spanning 240 co-evolved imposter strategies.
  arXiv  Detail & Related papers  (2026-05-09T16:07:23Z)
• TwinGate: Stateful Defense against Decompositional Jailbreaks in Untraceable Traffic via Asymmetric Contrastive Learning [60.68349524623048]
  Decompositional jailbreaks pose a critical threat to large language models.<n>We present TwinGate, a stateful dual-encoder defense framework.<n>We construct a comprehensive dataset of over 3.62 million instructions spanning 8,600 distinct malicious intents.
  arXiv  Detail & Related papers  (2026-04-30T13:44:01Z)
• SELFDOUBT: Uncertainty Quantification for Reasoning LLMs via the Hedge-to-Verify Ratio [0.038379177968040606]
  We propose SELFDOUBT, a single-pass uncertainty framework for reasoning language models.<n>Our key signal, the Hedge-to-Verify Ratio (HVR), detects whether a reasoning trace contains uncertainty markers and, if so, whether they are offset by explicit selfchecking behavior.<n> SELFDOUBT operates on a single observed reasoning trajectory, making it suitable for latency- and cost-constrained deployment over any proprietary API.
  arXiv  Detail & Related papers  (2026-04-07T19:19:29Z)
• TraceGuard: Structured Multi-Dimensional Monitoring as a Collusion-Resistant Control Protocol [1.0195618602298684]
  TraceGuard is a structured multi-dimensional monitoring protocol that evaluates agent actions across five dimensions.<n>It is implemented as a new monitor type for the open-source ControlArena framework.
  arXiv  Detail & Related papers  (2026-04-05T05:05:59Z)
• Understanding NPM Malicious Package Detection: A Benchmark-Driven Empirical Analysis [10.599261033874884]
  NPM ecosystem has become a primary target for software supply chain attacks.<n>We conduct a benchmark-driven empirical analysis of NPM malware detection.<n>We build a dataset of 6,420 malicious and 7,288 benign packages annotated with 11 behavior categories and 8 evasion techniques.
  arXiv  Detail & Related papers  (2026-03-29T07:04:31Z)
• SemGuard: Real-Time Semantic Evaluator for Correcting LLM-Generated Code [46.20378145112059]
  Post-hoc repair pipelines detect such faults only after execution.<n>We present SemGuard, a semantic-evaluator-driven framework that performs real-time, line-level semantic supervision.
  arXiv  Detail & Related papers  (2025-09-29T09:21:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.