Dead text or binding clause? Measuring and restoring constraint influence in black-box LLM dialogues
- URL: http://arxiv.org/abs/2608.12599v1
- Date: Wed, 12 Aug 2026 21:12:32 GMT
- Title: Dead text or binding clause? Measuring and restoring constraint influence in black-box LLM dialogues
- Abstract summary: Multi-turn dialogues let users revoke constraints as easily as impose them, but revocation does not reliably take effect.<n>No existing instrument measures this influence per clause, predicts it before delivery, or repairs it under matched budgets.<n>sysname closes the three gaps through the model API alone.
- Score: 0.0
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Multi-turn dialogues let users revoke constraints as easily as impose them, but revocation does not reliably take effect: models keep enacting withdrawn requirements (occasionally beneath comments asserting their removal), a failure we call \emph{behavioral relapse}, or revocation inertia. No existing instrument measures this influence per clause, predicts it before delivery, or repairs it under matched budgets. \sysname{} closes the three gaps through the model API alone: a contract ledger pairs every constraint with an executable checker, records revocations as tombstones, and compiles the net constraint state ahead of time into a single specification; a sequential ablation probe measures per-clause adherence and incremental behavioral effect; a repair ladder operates under token- and attempt-matched budgets. On \dataname{} (\NTasks{} HumanEval tasks, \NClauses{} verified checkers), relapse at an 8B operating point climbs from \ScaleDelayedMTwo{} to \ScaleDelayedMEight{} as constraint load grows, while stronger models sit at floor. Under matched checkers, model, and budget, ahead-of-time compilation significantly reduces relapse against a no-ledger verifier-retry baseline (\RestoreDiff{}, 95\% CI \RestoreDiffCI{}, $p$ \RestoreDiffP{}); adaptive ladder interventions stacked on top add no detectable gain (95\% confidence excludes gains $\geq$ \LadderExcludedGain{}). The probe predicts relapse before delivery (AUROC \AurocPrimary{}); a one-sentence tombstone note recovers about a third of the compilation effect and survives a placebo control. At \CostDeliveryFactor{} delivery overhead and \CostTotalHedged{} of API compute for every result, revocation failure becomes a measurable, predictable, and repairable property of dialogue state rather than an invisible one.
Related papers
- TRACE: Trajectory-robust Admission with Evidence Ordering for Efficient GUI Agents [54.88208747581851]
Training-free visual token pruning can reduce this cost, but cache reuse introduces a fundamental constraint.<n>We propose textbfmethod, a training-free framework for emphtextbfTrajectory-textbfrobust textbfAdmission and textbfCoverage-aware textbfEvidence ordering.
arXiv Detail & Related papers (2026-09-09T15:12:48Z) - Resume Means Resume: A Machine-Checked Conformance Contract for Checkpoint, Interrupt, and Resume Semantics in Workflow Persistence Layers [0.0]
A framework that persists execution state so a run can be interrupted, survive a crash, and continue must decide what a resume means for effects that already happened.<n>Five widely deployed agent workflow frameworks answer differently, none exposes a machine-checkable contract.<n>LangGraph 1.2.9 records a second resume value and never consults it, persists-invalid state silently, and re-executes durably recorded work after a real SIGKILL.
arXiv Detail & Related papers (2026-08-04T15:45:31Z) - Knowing When to Quit: Diagnosing and Training LLMs to Abort Futile Reasoning [109.33717747526583]
Large language models generate computationally expensive yet semantically void reasoning on beyond-capability tasks.<n>We introduce textbfCaRL (textbfCapability-textbfaligned textbfReinforcement textbfL) to align behavior with capability boundaries.<n>Experiments demonstrate a substantial reduction in futile reasoning while preserving performance across task difficulties.
arXiv Detail & Related papers (2026-07-31T09:30:33Z) - ShortOPD: Recovering Pruned LLMs with Short-to-Long On-Policy Distillation [72.54604107217669]
textscpass@$k$ nearly vanishes after compression, yet textscpass@$k$ recovers substantially under repeated sampling.<n>Recovery should train on the compressed model's own on-policy states with dense token-level supervision.<n>We propose textbfshortopd, a short-to-long OPD schedule that detects teacher-confirmed repetitive suffixes.
arXiv Detail & Related papers (2026-07-14T17:50:50Z) - Partial Contracts Suffice: Sound, LLM-Inferred Regression Verification [7.004725450780227]
We present the first contract-based regression verification tool.<n>Contract soundness is ensured by proving all function versions match the behavior.<n>We ask whether a partial, caller-sufficient contract, rather than a full behavioral specification, is enough.
arXiv Detail & Related papers (2026-07-11T12:40:22Z) - Ghost in the Context: Measuring Policy-Carriage Failures in Decision-Time Assembly [0.0]
LLM agents do not act on raw interaction history; they act on a bounded decision state assembled by truncation, summarization, reordering, and rewriting.<n>We study this failure mode over local Llama 3.1 8B, Qwen 2.5 7B, and Mistral 7B using judged exact constraint respect and direct audits of assembled-state visibility.
arXiv Detail & Related papers (2026-05-02T18:07:42Z) - Governing What You Cannot Observe: Adaptive Runtime Governance for Autonomous AI Agents [0.0]
Autonomous AI agents can remain fully authorized and still become unsafe as behavior drifts, adversaries adapt, and decision patterns shift without any code change.<n>We propose the textbfInformational Viability Principle: governing an agent reduces to estimating a bound on unobserved risk.<n>textbfRiskGate instantiates the framework with dedicated statistical estimators (KL divergence, segment-vs-rest $z$-tests, sequential pattern matching), a fail-secure monotonic pipeline, and a closed-loop Autopilot.
arXiv Detail & Related papers (2026-04-27T16:46:15Z) - Compiling Activation Steering into Weights via Null-Space Constraints for Stealthy Backdoors [48.881343993730844]
Safety-aligned large language models (LLMs) are increasingly deployed in real-world pipelines.<n> adversaries can distribute backdoored checkpoints that behave normally under standard evaluation but jailbreak when a hidden trigger is present.<n>Recent post-hoc weight-editing methods offer an efficient approach to injecting such backdoors.
arXiv Detail & Related papers (2026-04-14T06:48:33Z) - SABER: Small Actions, Big Errors -- Safeguarding Mutating Steps in LLM Agents [52.20768003832476]
We analyze execution traces on $$-Bench (Airline/Retail) and SWE-Bench Verified.<n>We formalize emphdecisive deviations, earliest action, level divergences that flip success to failure.<n>We introduce cm, a model-agnostic, gradient-free, test-time safeguard.
arXiv Detail & Related papers (2025-11-26T01:28:22Z) - Fast Controlled Generation from Language Models with Adaptive Weighted Rejection Sampling [90.86991492288487]
evaluating constraint on every token can be prohibitively expensive.<n> LCD can distort the global distribution over strings, sampling tokens based only on local information.<n>We show that our approach is superior to state-of-the-art baselines.
arXiv Detail & Related papers (2025-04-07T18:30:18Z) - Towards Variable-Length Textual Adversarial Attacks [68.27995111870712]
It is non-trivial to conduct textual adversarial attacks on natural language processing tasks due to the discreteness of data.
In this paper, we propose variable-length textual adversarial attacks(VL-Attack)
Our method can achieve $33.18$ BLEU score on IWSLT14 German-English translation, achieving an improvement of $1.47$ over the baseline model.
arXiv Detail & Related papers (2021-04-16T14:37:27Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.