SentGuard: Sentence-Level Streaming Guardrails for Large Language Models

• URL: http://arxiv.org/abs/2606.02041v1
• Date: Mon, 01 Jun 2026 10:30:08 GMT
• Title: SentGuard: Sentence-Level Streaming Guardrails for Large Language Models
• Abstract summary: We propose SentGuard, a sentence-level streaming guardrail that operates in parallel with generation.<n>A benchmark with structured per-sentence annotations captures the evolution of safety risks across both reasoning and response segments.<n>Experiments show that SentGuard outperforms existing baselines, detecting 90.5% of unsafe cases within two sentences while maintaining a low streaming false-positive rate of 7.41%.
• Score: 40.00517152303363
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Large language models increasingly stream long, reasoning-intensive responses in real time, making when to moderate as critical as whether to moderate. Existing guardrails fall into two unsatisfactory extremes: response-level methods delay intervention until the full output is generated, whereas token-level methods act on incomplete semantics, often producing unstable decisions and excessive guard invocations. To address this challenge, we propose SentGuard, a sentence-level streaming guardrail that operates in parallel with generation. A lightweight waiting buffer groups streamed tokens into sentence chunks and releases only verified chunks to the user, introducing a small offset that enables SentGuard to assess the current prefix while the target LLM decodes subsequent content. To support this, we construct StreamSafe, a benchmark with structured per-sentence annotations across 8 harm categories, capturing the evolution of safety risks across both reasoning and response segments. We further train SentGuard with a coarse-to-fine objective to detect unsafe intent as soon as it emerges at sentence boundaries. Experiments on 5 safety benchmarks show that SentGuard outperforms existing baselines, detecting 90.5% of unsafe cases within two sentences while maintaining a low streaming false-positive rate of 7.41%.

Related papers

• SAFEGuard: Detect Optimization-Based Jailbreak Attacks Through Harmful Semantic Analysis and Fluency Measurement [27.972348333797385]
  We propose a unified detection framework for large language models (LLMs)<n>Our method is grounded in a paramount observation: high fluency prompts maintain their malicious intention close to harmful prompts.<n>Our evaluation demonstrates that SAFEGuard consistently outperforms state-of-the-art baselines.
  arXiv  Detail & Related papers  (2026-09-05T03:25:31Z)
• LongGuard: Mechanistic Analysis and Training-Free Mitigation of Long-Context Failure in Safety Guardrails [21.90973380292976]
  LongGuard is a framework that evaluates, mechanistically analyzes, and mitigates long-context guardrail failure.<n>We formulate the task as Safety Needle-in-a-Haystack (SafetyNIAH) over a 0.25k-32k length grid.<n>Across 15 mainstream guardrails, unsafe recall drops monotonically by more than 50% on average, and a paired Benign-Fill vs. Needle-Repeat design attributes the failure to proportional dilution of the unsafe needle rather than to absolute length.
  arXiv  Detail & Related papers  (2026-08-27T18:07:25Z)
• Prompt Overflow: What the Guardrail Inspects Is Not What the Model Infers [22.77036856603702]
  Guardrail models are widely deployed to screen user inputs before they reach large language models (LLMs)<n>We introduce a novel Prompt Overflow Attack, which exploits a mismatch between the limited inspection windows of guardrail models and the substantially larger context inference windows of downstream LLMs.<n>We demonstrate that prompts reliably detected in short-context settings can evade guardrail models once adversarially manipulated into over-length inputs.
  arXiv  Detail & Related papers  (2026-05-22T03:27:18Z)
• GLiGuard: Schema-Conditioned Classification for LLM Safeguard [5.918280835312565]
  We introduce textbfGLiGuard, a bidirectional encoder adapted from GLiNER2 for LLM content moderation.<n>The key idea is to encode task definitions and label semantics directly into the input sequence as structured token schemas.<n>GLiGuard achieves F1 scores competitive with 7B--27B decoder-based guards despite being 23--90$times$ smaller.
  arXiv  Detail & Related papers  (2026-05-08T16:44:07Z)
• TwinGate: Stateful Defense against Decompositional Jailbreaks in Untraceable Traffic via Asymmetric Contrastive Learning [60.68349524623048]
  Decompositional jailbreaks pose a critical threat to large language models.<n>We present TwinGate, a stateful dual-encoder defense framework.<n>We construct a comprehensive dataset of over 3.62 million instructions spanning 8,600 distinct malicious intents.
  arXiv  Detail & Related papers  (2026-04-30T13:44:01Z)
• DeepGuard: Secure Code Generation via Multi-Layer Semantic Aggregation [22.14448091276763]
  A common strategy for security hardening is to fine-tune models using supervision derived from the final transformer layer.<n>This design may suffer from a final-layer bottleneck: vulnerability-discriminative cues can be distributed across layers and become less detectable near the output representations optimized for next-token prediction.<n>We introduce DeepGuard, a framework that leverages distributed security-relevant cues by aggregating representations from multiple upper layers via an attention-based module.
  arXiv  Detail & Related papers  (2026-04-10T08:19:48Z)
• TrajGuard: Streaming Hidden-state Trajectory Detection for Decoding-time Jailbreak Defense [22.678364306206493]
  Existing jailbreak defense paradigms rely on static detection of prompts, outputs, or internal states.<n>We show that hidden states in critical layers during decoding carry stronger and more stable risk signals than input jailbreak prompts.<n>We propose TrajGuard, a training-free, decoding-time defense framework.
  arXiv  Detail & Related papers  (2026-04-09T02:22:44Z)
• The Trojan Knowledge: Bypassing Commercial LLM Guardrails via Harmless Prompt Weaving and Adaptive Tree Search [58.8834056209347]
  Large language models (LLMs) remain vulnerable to jailbreak attacks that bypass safety guardrails to elicit harmful outputs.<n>We introduce the Correlated Knowledge Attack Agent (CKA-Agent), a dynamic framework that reframes jailbreaking as an adaptive, tree-structured exploration of the target model's knowledge base.
  arXiv  Detail & Related papers  (2025-12-01T07:05:23Z)
• Bag of Tricks for Subverting Reasoning-based Safety Guardrails [62.139297207938036]
  We present a bag of jailbreak methods that subvert the reasoning-based guardrails.<n>Our attacks span white-, gray-, and black-box settings and range from effortless template manipulations to fully automated optimization.
  arXiv  Detail & Related papers  (2025-10-13T16:16:44Z)
• DiffuGuard: How Intrinsic Safety is Lost and Found in Diffusion Large Language Models [50.21378052667732]
  We conduct an in-depth analysis of dLLM vulnerabilities to jailbreak attacks across two distinct dimensions: intra-step and inter-step dynamics.<n>We propose DiffuGuard, a training-free defense framework that addresses vulnerabilities through a dual-stage approach.
  arXiv  Detail & Related papers  (2025-09-29T05:17:10Z)
• AdaptiveGuard: Towards Adaptive Runtime Safety for LLM-Powered Software [11.606665113249298]
  Guardrails are critical for the safe deployment of Large Language Models (LLMs)-powered software.<n>We propose AdaptiveGuard, an adaptive guardrail that detects novel jailbreak attacks as out-of-distribution (OOD) inputs.<n>We show that AdaptiveGuard achieves 96% OOD detection accuracy, adapts to new attacks in just two update steps, and retains over 85% F1-score on in-distribution data post-adaptation.
  arXiv  Detail & Related papers  (2025-09-21T01:22:42Z)
• Revisiting Backdoor Attacks on LLMs: A Stealthy and Practical Poisoning Framework via Harmless Inputs [54.90315421117162]
  We propose a novel poisoning method via completely harmless data.<n>Inspired by the causal reasoning in auto-regressive LLMs, we aim to establish robust associations between triggers and an affirmative response prefix.<n>We observe an interesting resistance phenomenon where the LLM initially appears to agree but subsequently refuses to answer.
  arXiv  Detail & Related papers  (2025-05-23T08:13:59Z)
• Steering Dialogue Dynamics for Robustness against Multi-turn Jailbreaking Attacks [59.300698230887114]
  Large language models (LLMs) are shown to be vulnerable to jailbreaking attacks where adversarial prompts are designed to elicit harmful responses.<n>We propose a safety steering framework grounded in safe control theory, ensuring invariant safety in multi-turn dialogues.
  arXiv  Detail & Related papers  (2025-02-28T21:10:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.