RiskChainBench: A Benchmark for Obfuscated Platform Message Restoration and Evidence-Grounded Web Investigation

• URL: http://arxiv.org/abs/2609.16900v1
• Date: Tue, 15 Sep 2026 09:32:45 GMT
• Title: RiskChainBench: A Benchmark for Obfuscated Platform Message Restoration and Evidence-Grounded Web Investigation
• Abstract summary: We introduce RiskChainBench, pairing 3,600 synthetic token-text restoration inputs from 600 source sessions with 600 corresponding human-labeled local web environments.<n>A model first restores the message, operational intent, and destination; the same underlying model then acts as a VLM-driven web agent.<n>We score restoration and correct-routing web investigation separately and compose them offline by applying the frozen primary-entry prediction as a gate to the same Task 2 result.
• Score: 32.78931752588511
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Platform abuse campaigns conceal redirection instructions with emojis, homophones, character decomposition, and redundant symbols, then route users through disguised links to services associated with pornography, fraud, gambling, or illicit transactions. Existing benchmarks evaluate obfuscated text and risky webpages separately, obscuring how target recovery affects downstream evidence acquisition. We introduce RiskChainBench, pairing 3,600 synthetic token-text restoration inputs from 600 source sessions with 600 corresponding human-labeled local web environments. A model first restores the message, operational intent, and destination; the same underlying model then acts as a VLM-driven web agent that investigates the correctly associated website and produces a frozen, evidence-cited risk report without message-side semantics or domain-reputation cues. We score restoration and correct-routing web investigation separately and compose them offline by applying the frozen primary-entry prediction as a gate to the same Task 2 result. Human labels determine task correctness, while a fixed multimodal evidence judge assesses faithfulness, sufficiency, completeness, and consistency. Across ten models, Entry Top-1 ranges from 35.2% to 95.2% and web decision accuracy from 26.3% to 62.8%; the leading systems differ across entry recovery, full reconstruction, website decisions, and fine-grained typing. Execution failures account for 31.9% of web runs, whereas post-decision type errors account for only 0.9%, identifying stable exploration and risk judgment as the principal bottlenecks. We release the benchmark, protocol, and resettable local sandbox.

Related papers

• AtomCite: Verification and Correction of Supplied Page-Level Citations in Multi-Page Documents [21.842813905857863]
  AtomCite parses an answer into claims, checks each claim against the image of the cited page, and applies a deterministic repair policy.<n>It reaches around 93% binary verification accuracy on the injected benchmark, significantly outperforming every OCR-only condition.<n>Its repair policy lifts citation precision on the injected mix from a constructed 34% to 87-90% while retaining over 90% of correct claims.
  arXiv  Detail & Related papers  (2026-09-05T01:31:50Z)
• SIR: Self-improving Red-teaming for Compute Use Agents [71.71987044117371]
  Computer use agents (CUAs) are vision-language models that perceive a screen and act on a real operating system through mouse, keyboard, and terminal.<n>Because they can be exposed to untrusted content while operating, they are vulnerable to indirect prompt injection (IPI)<n>We present SIR, a black box IPI attack that composes stealthy injections from a small library of reusable principles stated in plain language.
  arXiv  Detail & Related papers  (2026-08-31T03:39:43Z)
• Fantastic Adaptive Taxonomies and How to Use Them [100.20614173157708]
  AdaMAST converts execution traces into a compact, evidence-grounded failure taxonomy.<n>No code is hand-authored, no trace human-annotated.<n>In agent-system search, taxonomy-coded diagnoses of failed candidates outperform free-form.
  arXiv  Detail & Related papers  (2026-07-17T17:41:16Z)
• Efficient Visual Pointing for Embodied AI:Agent-Driven Data Synthesis, Cross-Block Attention, and Iterative Correction [55.11480729304395]
  We describe our PointArena 2026 solution, which achieves 77.2% overall accuracy and ranks second on the benchmark.<n>The ap proach targets three failure modes. First, agent-driven syn thesis builds large semantic and anchor-relative candidate pools.<n>Second, a determinis tic steerable-data pipeline creates a verified 10,000-sample main set, plus reserve samples, using masks, templates, and path verification.
  arXiv  Detail & Related papers  (2026-06-29T06:39:03Z)
• A Hybrid, Multi-Layered Pipeline for Phishing and Threat Classification: Independently Validated URL and NLP Engines with a Calibrated Multi-Channel Fusion Stage [0.0]
  We present a hybrid pipeline that scores each modality with its own engine and fuses the results.<n>Three engines are built, deployed, and independently benchmarked.<n>For deployable detection, the limiting factor is how well a model generalizes.
  arXiv  Detail & Related papers  (2026-06-19T18:59:36Z)
• FraudSMSWalker: Benchmarking Agentic Large Language Models for SMS-to-Webpage Fraud Detection [0.0]
  FraudSMSWalker is a benchmark for URL-masked SMS-to-webpage fraud judgment.<n>It contains 699 bilingual chains, including 332 fraudulent and 367 benign cases, across ten service scenarios.<n>Results show that current agents can detect suspicious cues, but struggle to preserve benign recall and often produce positive predictions.
  arXiv  Detail & Related papers  (2026-06-15T12:53:55Z)
• Hallucination as Exploit: Evidence-Carrying Multimodal Agents [10.441697487723568]
  Multimodal agents increasingly choose tool calls from screenshots, documents, and webpages, where a false perceptual claim can turn from an answer-quality error into an authorization failure.<n>We propose evidence-carrying multimodal agents (ECA), which treat free-form model text as inadmissible evidence.<n>ECA converts opaque model belief into auditable residuals at the verifier, schema, and implementation levels.
  arXiv  Detail & Related papers  (2026-05-18T23:40:43Z)
• AgentTrust: Runtime Safety Evaluation and Interception for AI Agent Tool Use [2.9991161518367875]
  AgentTrust intercepts agent tool calls before execution and returns a structured verdict: allow, warn, block, or review.<n>We release a 300-scenario benchmark across six risk categories and an additional 630 independently constructed real-world adversarial scenarios.<n>On the 630-second benchmark, evaluated under a patched ruleset, AgentTrust achieves 96.7% verdict accuracy, including about 93% on shell-obfuscated payloads.
  arXiv  Detail & Related papers  (2026-05-06T11:38:16Z)
• RoguePrompt: Dual-Layer Ciphering for Self-Reconstruction to Circumvent LLM Moderation [0.0]
  This paper presents an automated jailbreak attack that converts a disallowed user query into a self reconstructing prompt.<n>We instantiate RoguePrompt against GPT 4o and evaluate it on 2 448 prompts that a production moderation system previously marked as strongly rejected.<n>Under an evaluation protocol that separates three security relevant outcomes bypass, reconstruction, and execution the attack attains 84.7 percent bypass, 80.2 percent reconstruction, and 71.5 percent full execution.
  arXiv  Detail & Related papers  (2025-11-24T05:42:54Z)
• BrowserArena: Evaluating LLM Agents on Real-World Web Navigation Tasks [51.803138848305814]
  We introduce BrowserArena, a live open-web agent evaluation platform that collects user-submitted tasks.<n>We identify three consistent failure modes: captcha resolution, pop-up banner removal, and direct navigation to URLs.<n>Our findings surface both the diversity and brittleness of current web agents.
  arXiv  Detail & Related papers  (2025-10-02T15:22:21Z)
• VulAgent: Hypothesis-Validation based Multi-Agent Vulnerability Detection [55.957275374847484]
  VulAgent is a multi-agent vulnerability detection framework based on hypothesis validation.<n>It implements a semantics-sensitive, multi-view detection pipeline, each aligned to a specific analysis perspective.<n>On average, VulAgent improves overall accuracy by 6.6%, increases the correct identification rate of vulnerable--fixed code pairs by up to 450%, and reduces the false positive rate by about 36%.
  arXiv  Detail & Related papers  (2025-09-15T02:25:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.