Why Trust Your Agent? Empirical Security Gains from TRiSM-Guided Agentic Workflows in Healthcare

• URL: http://arxiv.org/abs/2606.28666v1
• Date: Sat, 27 Jun 2026 01:00:39 GMT
• Title: Why Trust Your Agent? Empirical Security Gains from TRiSM-Guided Agentic Workflows in Healthcare
• Abstract summary: This paper applies the AI Trust, Risk, and Security Management (TRiSM) framework to a medical report-generation application.<n>It demonstrates how an insecure agent workflow can be transformed into security-conscious agentic workflow.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Agent-based AI has enabled the automation of tasks by exposing application tools and resources to large language models (LLMs). However, to improve scope and accuracy, agents are often given access rights that exceed those of ordinary users, introducing significant security risks. AI is routinely integrated into applications with a disregard to security, risking data exposure and breaching regulations. This paper applies the AI Trust, Risk, and Security Management (TRiSM) framework to a medical report-generation application to demonstrate how an insecure agent workflow can be transformed into security-conscious agentic workflow. Both workflows were evaluated across five LLMs (Claude Haiku 4.5, GPT-4.1-nano, GPT-4.1-mini, GPT-5.4-mini, and Gemini 2.5 Flash) on two report types, totalling 800 generations and 500 attack scenarios including RAG poisoning, data-field injection, and client-side network injection. The TRiSM-guided agentic workflow reduced mean attack success rates from 31% to 10% for RAG poisoning and from 42% to 25% for data-field injection, while eliminating the network injection vector entirely through server-side prompt construction. Furthermore, report accuracy increased by 14 percentage points (72.5% to 86.5%) with the agentic workflow, demonstrating a secure design which provides more reliable outputs. This paper contributes to knowledge by demonstrating least-privilege, defence in depth agentic workflows improving security and accuracy, while also highlighting model choice is a necessary architectural consideration.

Related papers

• EvoSafeHarness: Evolving Model- and Domain-Specific Harnesses for Securing Agents [74.54929751174289]
  EvoSafeHarness is a safety-specific optimization framework that synthesizes a deployable harness for a frozen model in a target domain.<n>It reduces average attack success rate from 45.6% to 10.0% at a 3.3-point utility cost.<n>It also achieves the best score on Agent-SafetyBench for every victim.
  arXiv  Detail & Related papers  (2026-09-05T05:56:41Z)
• Tool Specifications Matter: Uncovering and Mitigating Safety Risks in AI Agents [21.622196229181103]
  We identify schema-formatted tool specifications as a primary source of agent safety degradation.<n>We propose SafeKeep, an inference-time safeguard that decouples safety judgment from tool execution.<n>It increases the average refusal rate for harmful requests from 23.8% to 70.6% and reduces the average attack success rate under observation-level prompt injection from 25.6% to 2.5%.
  arXiv  Detail & Related papers  (2026-07-31T10:25:04Z)
• Trust but Verify? Uncovering the Security Debt of Autonomous Coding Agents [3.30025498453752]
  This paper presents a large-scale empirical study using the AIDev dataset to characterize security code smells in agent-generated pull requests.<n>We find that 38.9% of agent-generated PRs contain at least one security smell, with supply chain integrity issues accounting for 82.3% of all detected security smells.<n>Human collaborators are responsible for introducing 67.6% of genuine leaked secrets within these agent-assisted systems.
  arXiv  Detail & Related papers  (2026-07-14T06:59:41Z)
• OccuBench: Evaluating AI Agents on Real-World Professional Tasks via Language Environment Simulation [57.505743202759646]
  OccuBench is a benchmark covering 100 real-world professional task scenarios across 10 industry categories and 65 specialized domains.<n>Our multi-agent synthesis pipeline automatically produces evaluation instances with guaranteed solvability, calibrated difficulty, and document-grounded diversity.
  arXiv  Detail & Related papers  (2026-04-13T00:27:32Z)
• The Verifier Tax: Horizon Dependent Safety Success Tradeoffs in Tool Using LLM Agents [1.3299507495084417]
  We study how runtime enforcement against unsafe actions affects end-to-end task performance in multi-step tool using large language model (LLM) agents.<n>We identify model dependent interaction horizons (15 to 30 turns) and decompose outcomes into overall success rate (SR), safe success rate (SSR), and unsafe success rate (USR)
  arXiv  Detail & Related papers  (2026-03-18T16:16:34Z)
• AgentRaft: Automated Detection of Data Over-Exposure in LLM Agents [40.88941407116349]
  We present AgentRaft, the first automated framework for detecting Data Over-Exposure (DOE) risks in Large Language Model (LLM) agents.<n>We evaluate AgentRaft on a testing environment of 6,675 real-world agent tools.
  arXiv  Detail & Related papers  (2026-03-08T09:40:54Z)
• INFA-Guard: Mitigating Malicious Propagation via Infection-Aware Safeguarding in LLM-Based Multi-Agent Systems [70.37731999972785]
  In this paper, we propose Infection-Aware Guard, INFA-Guard, a novel defense framework that explicitly identifies and addresses infected agents as a distinct threat category.<n>During remediation, INFA-Guard replaces attackers and rehabilitates infected ones, avoiding malicious propagation while preserving topological integrity.
  arXiv  Detail & Related papers  (2026-01-21T05:27:08Z)
• It's a TRAP! Task-Redirecting Agent Persuasion Benchmark for Web Agents [52.81924177620322]
  Web-based agents powered by large language models are increasingly used for tasks such as email management or professional networking.<n>Their reliance on dynamic web content makes them vulnerable to prompt injection attacks: adversarial instructions hidden in interface elements that persuade the agent to divert from its original task.<n>We introduce the Task-Redirecting Agent Persuasion Benchmark (TRAP), an evaluation for studying how persuasion techniques misguide autonomous web agents on realistic tasks.
  arXiv  Detail & Related papers  (2025-12-29T01:09:10Z)
• Penetration Testing of Agentic AI: A Comparative Security Analysis Across Models and Frameworks [0.0]
  Agentic AI introduces security vulnerabilities that traditional LLM safeguards fail to address.<n>We conduct the first systematic testing and comparative evaluation of agentic AI systems.<n>We identify six distinct defensive behavior patterns including a novel "hallucinated compliance" strategy.
  arXiv  Detail & Related papers  (2025-12-16T19:22:50Z)
• WebGuard: Building a Generalizable Guardrail for Web Agents [59.31116061613742]
  WebGuard is the first dataset designed to support the assessment of web agent action risks.<n>It contains 4,939 human-annotated actions from 193 websites across 22 diverse domains.
  arXiv  Detail & Related papers  (2025-07-18T18:06:27Z)
• When Developer Aid Becomes Security Debt: A Systematic Analysis of Insecure Behaviors in LLM Coding Agents [1.7587442088965226]
  LLM-based coding agents are rapidly being deployed in software development, yet their safety implications remain poorly understood.<n>We conducted the first systematic safety evaluation of autonomous coding agents, analyzing over 12,000 actions across five state-of-the-art models.<n>We developed a high-precision detection system that identified four major vulnerability categories, with information exposure the most prevalent.
  arXiv  Detail & Related papers  (2025-07-12T16:11:07Z)
• OpenAgentSafety: A Comprehensive Framework for Evaluating Real-World AI Agent Safety [58.201189860217724]
  We introduce OpenAgentSafety, a comprehensive framework for evaluating agent behavior across eight critical risk categories.<n>Unlike prior work, our framework evaluates agents that interact with real tools, including web browsers, code execution environments, file systems, bash shells, and messaging platforms.<n>It combines rule-based analysis with LLM-as-judge assessments to detect both overt and subtle unsafe behaviors.
  arXiv  Detail & Related papers  (2025-07-08T16:18:54Z)
• SafeAgentBench: A Benchmark for Safe Task Planning of Embodied LLM Agents [58.65256663334316]
  We present SafeAgentBench -- the first benchmark for safety-aware task planning of embodied LLM agents in interactive simulation environments.<n>SafeAgentBench includes: (1) an executable, diverse, and high-quality dataset of 750 tasks, rigorously curated to cover 10 potential hazards and 3 task types; (2) SafeAgentEnv, a universal embodied environment with a low-level controller, supporting multi-agent execution with 17 high-level actions for 9 state-of-the-art baselines; and (3) reliable evaluation methods from both execution and semantic perspectives.
  arXiv  Detail & Related papers  (2024-12-17T18:55:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.