Filtering Harmful Actions Isn't Enough: Phantom Transfer in Agentic SDF

• URL: http://arxiv.org/abs/2607.10750v1
• Date: Sun, 12 Jul 2026 13:00:41 GMT
• Title: Filtering Harmful Actions Isn't Enough: Phantom Transfer in Agentic SDF
• Abstract summary: We investigate the effects of training on synthetic agentic trajectories containing adversarial interactions.<n>Finetuning on these trajectories consistently increases misaligned behavior.<n>Our results suggest that action level filtering is insufficient to ensure the safety of synthetic agentic training data.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Synthetic data is widely used to train large language models because it is inexpensive to generate and easy to control. As models are increasingly deployed as agents, synthetic trajectories are likely to become an important source of training data for agentic behavior. We investigate the effects of training on synthetic agentic trajectories containing adversarial interactions, including actions such as terminating another agents process, lowering its scheduling priority, or accessing resources without authorization. We finetune Llama 3.3 70B Instruct on these trajectories, generated to approximate reinforcement learning rollouts, and evaluate the resulting models on Anthropics Agentic Misalignment suite and Apollos in context scheming scenarios. Finetuning on these trajectories consistently increases misaligned behavior. Leaking rises by roughly a factor of five over the baseline, 4.6% to 24.9%. This increase survives the removal of every adversarial action from the trajectories. Finetuning on structurally comparable trajectories generated benign from the start produce a substantially smaller effect, 15.5%. These results indicate that the misaligned disposition is introduced during the generation process and encoded diffusely throughout the trajectory, rather than being localized to the harmful actions themselves. The effect also depends on the generating model. Benign trajectories produced by Gemini 2.5 Flash induce slightly higher leaking rates than trajectories generated from identical tasks by Claude 3.7 Sonnet. In contrast, broad safety benchmarks degrade similarly across all finetuned models and therefore fail to distinguish these effects. Our results suggest that action level filtering is insufficient to ensure the safety of synthetic agentic training data and that dispositions introduced by the generating model can survive semantic inspection.

Related papers

• TailBooster: A Dual-Layer Generative Framework for Extreme Value Augmentation with Operational Validity Enforcement [0.0]
  Extreme events in air transport cause cascading network disruptions with substantial operational, economic, and safety costs.<n>Synthetic data augmentation offers a principled solution, but conventional generative models under-represent distributional tails.<n>We propose TailBooster, a dual-layer generative framework combining generative modelling with anomaly detection layers.
  arXiv  Detail & Related papers  (2026-08-12T11:37:09Z)
• A Systematic Evaluation of Trajectory Data Curation for LoRA Fine-Tuning of Code Agents [0.0]
  Supervised fine-tuning of open-weight LLMs on expert agent trajectories has emerged as a prominent approach to building capable code agents.<n>We present a systematic empirical study of trajectory data filtering for LoRA fine-tuning of Qwen2.5-Coder-7B-Instruct on the SWE-trajectory dataset.<n>We propose a two-axis quality scoring framework -- Efficiency and Style -- and evaluate it through 16 controlled experiments spanning strategy, scale, and ablation analyses.
  arXiv  Detail & Related papers  (2026-07-19T11:52:32Z)
• What Makes Interaction Trajectories Effective for Training Terminal Agents? [55.62817294510983]
  Terminal-Lego is a scalable pipeline that transforms real-world issues into environment-verified agentic tasks.<n>Students fine-tuned on trajectories from DeepSeek-V3.2, a lower-scoring agent, exhibit significantly stronger generalization.<n>With only 15.3k Terminal-Lego trajectories, Qwen3-32B achieves a 24.3% score on Terminal-Bench 2.0, rivaling previous SOTA performance established with over 30x the data volume.
  arXiv  Detail & Related papers  (2026-06-02T10:37:47Z)
• Signals: Trajectory Sampling and Triage for Agentic Interactions [1.6631602844999722]
  We propose a lightweight, signal-based framework for triaging agentic interaction trajectories.<n>Our approach computes cheap, broadly applicable signals from live interactions and attaches them as structured attributes for trajectory triage.<n>We show that signal-based sampling achieves an 82% informativeness rate compared to 74% for filtering and 54% for random sampling.
  arXiv  Detail & Related papers  (2026-04-01T01:08:07Z)
• TopoCurate:Modeling Interaction Topology for Tool-Use Agent Training [53.93696896939915]
  Training tool-use agents typically rely on Supervised Fine-Tuning (SFT) on successful trajectories and Reinforcement Learning (RL) on pass-rate-selected tasks.<n>We propose TopoCurate, an interaction-aware framework that projects multi-trial rollouts from the same task into a unified semantic quotient topology.<n>TopoCurate achieves consistent gains of 4.2% (SFT) and 6.9% (RL) over state-of-the-art baselines.
  arXiv  Detail & Related papers  (2026-03-02T10:38:54Z)
• In-the-Wild Model Organisms: Mitigating Undesirable Emergent Behaviors in Production LLM Post-Training via Data Attribution [1.518298096221251]
  We propose activation-based data attribution.<n>We identify datapoints that cause specific behaviors and validate these attributions causally by retraining with modified data.<n>Applying this to OLMo 2's production DPO training, we surfaced distractor-triggered compliance.
  arXiv  Detail & Related papers  (2026-02-11T17:45:31Z)
• Can Active Sampling Reduce Causal Confusion in Offline Reinforcement Learning? [58.942118128503104]
  Causal confusion is a phenomenon where an agent learns a policy that reflects imperfect spurious correlations in the data. This phenomenon is particularly pronounced in domains such as robotics. In this paper, we study causal confusion in offline reinforcement learning.
  arXiv  Detail & Related papers  (2023-12-28T17:54:56Z)
• t-RAIN: Robust generalization under weather-aliasing label shift attacks [0.0]
  We analyze the impact of label shift on the task of multi-weather classification for autonomous vehicles. We propose t-RAIN a similarity mapping technique for synthetic data augmentation using large scale generative models. We present state-of-the-art pedestrian detection results on real and synthetic weather domains with best performing 82.69 AP (snow) and 62.31 AP (fog) respectively.
  arXiv  Detail & Related papers  (2023-05-15T02:05:56Z)
• Improving Adversarial Transferability via Intermediate-level Perturbation Decay [79.07074710460012]
  We develop a novel intermediate-level method that crafts adversarial examples within a single stage of optimization. Experimental results show that it outperforms state-of-the-arts by large margins in attacking various victim models.
  arXiv  Detail & Related papers  (2023-04-26T09:49:55Z)
• Robust Trajectory Prediction against Adversarial Attacks [84.10405251683713]
  Trajectory prediction using deep neural networks (DNNs) is an essential component of autonomous driving systems. These methods are vulnerable to adversarial attacks, leading to serious consequences such as collisions. In this work, we identify two key ingredients to defend trajectory prediction models against adversarial attacks.
  arXiv  Detail & Related papers  (2022-07-29T22:35:05Z)
• Tracking the risk of a deployed model and detecting harmful distribution shifts [105.27463615756733]
  In practice, it may make sense to ignore benign shifts, under which the performance of a deployed model does not degrade substantially. We argue that a sensible method for firing off a warning has to both (a) detect harmful shifts while ignoring benign ones, and (b) allow continuous monitoring of model performance without increasing the false alarm rate.
  arXiv  Detail & Related papers  (2021-10-12T17:21:41Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.