Cross-Session Decomposition Attacks: Scaling Risk and Intent-Aligned Retrieval Defense

• URL: http://arxiv.org/abs/2608.27945v1
• Date: Fri, 28 Aug 2026 05:39:29 GMT
• Title: Cross-Session Decomposition Attacks: Scaling Risk and Intent-Aligned Retrieval Defense
• Abstract summary: Scaling laws are usually read as a capability story: lower language-modeling loss yields more useful models.<n>We study a safety consequence of this mechanism in emphcross-session decomposition attacks, where benign-looking subqueries are asked across independent interactions and later recomposed toward a forbidden objective.<n>We formalize this setting as emphcompositional safety risk and prove a conditional risk-transfer bound.
• Score: 49.05879299987137
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Scaling laws are usually read as a capability story: lower language-modeling loss yields more useful models. We study a safety consequence of this mechanism in \emph{cross-session decomposition attacks}, where benign-looking subqueries are asked across independent interactions and later recomposed toward a forbidden objective. We formalize this setting as \emph{compositional safety risk} and prove a conditional risk-transfer bound: when the reference environment already contains dispersed evidence for a risky reconstruction, the gap between deployed composed risk and reference composed risk is controlled by the model's excess loss on allowed subqueries. Synthetic withholding experiments show that wider transformers assign lower loss to held-out instructions that never appear verbatim in training but are recoverable from injected supporting facts. A 600-intent pretrained-LLM evaluation shows that larger Qwen3 and Gemma3 family members can yield greater harmful-capability uplift under a fixed decomposition-composition pipeline. As a defense, IntentAlign-MiniLM, our 22M-parameter intent-aligned retriever, outperforms much larger embedding models on held-out intent retrieval and yields the best learned-retriever harmful recall across tested guardrails. Code is available in \href{https://github.com/liaodisen/Cross-Session-Decomposition-Attacks}{our GitHub repository}.

Related papers

• Bilevel Coordinated Reflection: A Game-Theoretic Approach to Multi-Agent LLM Systems [40.47928908205563]
  Multi-agent LLM systems commonly use an orchestrator to decompose a task for a team of workers and then improve through textual reflection.<n>Despite strong empirical results, these systems lack a unified account of coordination, memory improvement, and the role of external verification.<n>We provide confidence gating for evaluation and re-anchoring guarantees for piecewise-stationary environments.<n>Experiments instantiate these objects with environment-grounded metrics and test the predicted coordination and drift laws.
  arXiv  Detail & Related papers  (2026-09-02T15:50:10Z)
• Reassembling Distributed Risk: Trajectory-Conditioned Action Generation for Multi-Turn Agent Safety [12.022506016268112]
  Under multi-turn decomposition attacks, a harmful objective can be distributed across individually plausible requests and tool calls.<n>We propose emphReassembling Distributed Risk (ReDiR), a generation-time defense that conditions action generation on trajectory-level security evidence.<n>We evaluate ReDiR on two agent-safety benchmarks across three model families and eight held-out tool domains.
  arXiv  Detail & Related papers  (2026-08-26T12:28:35Z)
• Quantifying Risk Under Evolving Uncertainty: Belief-Dependent Robustness for Safe Sequential Decision Making [0.0]
  We propose RATTL (Risk-Adrial Total-Reward Learning), which ties caution to uncertainty.<n>We show the resulting planning problem is well posed under transience and compactness conditions.<n>RATTL targets safety for agents, including runtime-based systems, acting under uncertainty.
  arXiv  Detail & Related papers  (2026-08-18T09:36:19Z)
• Capability-Routed Guard: Defending Large Reasoning Models Against Reasoning-Centric Jailbreaks [6.260656015665084]
  We introduce Capability-Routed Guard (CRG), a model-agnostic inference-time guardrail for closed-source large reasoning models.<n>CRG reframes prompt defense as a capability-routing problem, separating executable intent from untrusted reasoning context.<n>It effectively mitigates diverse reasoning-centric jailbreaks while preserving benign utility and avoiding common over-refusal issues.
  arXiv  Detail & Related papers  (2026-08-08T03:40:06Z)
• From Sparse Features to Trustworthy Proxies: Certifying SAE-Based Interpretability [53.681678236115836]
  We study when can an SAE-based explanation be treated as a faithful view of an underlying frozen LM.<n>Our framework derives an upper bound on the base model's expected risk using four measurable quantities.<n>We show that the bound becomes non-vacuous on GPT-2 Small, Gemma-2B, and Llama-3-8B at practical sample sizes.
  arXiv  Detail & Related papers  (2026-06-16T18:28:23Z)
• Hidden in Plain Sight: Benchmarking Agent Safety Against Decomposition Attacks with DECOMPBENCH [21.171990974350773]
  DeCompBench is a benchmark designed to evaluate agentic safety under decomposition attacks.<n>Our experiments show that state-of-the-art agents exhibit high refusal rates on monolithic harmful tasks, but significantly lower refusal rates on their variants.
  arXiv  Detail & Related papers  (2026-06-12T00:30:29Z)
• Beyond Semantic Relevance: Counterfactual Risk Minimization for Robust Retrieval-Augmented Generation [19.888565537472363]
  CoRM-RAG (Counterfactual Risk Minimization for RAG) is a framework that aligns retrieval with decision safety rather than mere similarity.<n>We introduce a Cognitive Perturbation Protocol to simulate user biases during training, which is then distilled into a lightweight Evidence Critic.<n>This scoring module learns to identify documents that possess sufficient evidential strength to steer the model toward correctness despite adversarial query perturbations.
  arXiv  Detail & Related papers  (2026-05-02T07:22:24Z)
• TwinGate: Stateful Defense against Decompositional Jailbreaks in Untraceable Traffic via Asymmetric Contrastive Learning [60.68349524623048]
  Decompositional jailbreaks pose a critical threat to large language models.<n>We present TwinGate, a stateful dual-encoder defense framework.<n>We construct a comprehensive dataset of over 3.62 million instructions spanning 8,600 distinct malicious intents.
  arXiv  Detail & Related papers  (2026-04-30T13:44:01Z)
• When Models Outthink Their Safety: Mitigating Self-Jailbreak in Large Reasoning Models with Chain-of-Guardrails [74.63933201261595]
  Large Reasoning Models (LRMs) demonstrate remarkable capabilities on complex reasoning tasks.<n>LRMs remain vulnerable to severe safety risks, including harmful content generation and jailbreak attacks.<n>We propose the Chain-of-Guardrail (CoG), a training framework that recomposes or backtracks unsafe reasoning steps.
  arXiv  Detail & Related papers  (2025-10-24T09:32:25Z)
• Building a Foundational Guardrail for General Agentic Systems via Synthetic Data [76.18834864749606]
  LLM agents can plan multi-step tasks, intervening at the planning stage-before any action is executed-is often the safest way to prevent harm.<n>Existing guardrails mostly operate post-execution, which is difficult to scale and leaves little room for controllable supervision at the plan level.<n>We introduce AuraGen, a controllable engine that synthesizes benign trajectories, injects category-labeled risks with difficulty, and filters outputs via an automated reward model.
  arXiv  Detail & Related papers  (2025-10-10T18:42:32Z)
• Revisiting Backdoor Attacks on LLMs: A Stealthy and Practical Poisoning Framework via Harmless Inputs [54.90315421117162]
  We propose a novel poisoning method via completely harmless data.<n>Inspired by the causal reasoning in auto-regressive LLMs, we aim to establish robust associations between triggers and an affirmative response prefix.<n>We observe an interesting resistance phenomenon where the LLM initially appears to agree but subsequently refuses to answer.
  arXiv  Detail & Related papers  (2025-05-23T08:13:59Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.