Comment and Control: Hijacking Agentic Workflows via Context-Grounded Evolution

• URL: http://arxiv.org/abs/2605.11229v1
• Date: Mon, 11 May 2026 20:45:31 GMT
• Title: Comment and Control: Hijacking Agentic Workflows via Context-Grounded Evolution
• Abstract summary: GitHub Actions and n8n integrate Large Language Model (LLM) agents for tasks such as code review and data synchronization.<n>An adversary may control and craft certain inputs, such as GitHub issue comments, to manipulate the LLM agent for unwanted actions.<n>In this paper, we design the first detection and exploitation framework, called JAW, to hijack agentic runtime platforms.
• Score: 13.979573641143212
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Automation platforms such as GitHub Actions and n8n are increasingly adopting so-called agentic workflows, which integrate Large Language Model (LLM) agents for tasks such as code review and data synchronization. While bringing convenience for developers, this integration exposes a new risk: An adversary may control and craft certain inputs, such as GitHub issue comments, to manipulate the LLM agent for unwanted actions, such as credential exfiltration and arbitrary command execution. To our knowledge, no prior academic work has studied such a risk in agentic workflows. In this paper, we design the first detection and exploitation framework, called JAW, to hijack agentic workflows hosted on automation platforms via a novel approach called Context-Grounded Evolution. Our key idea is to evolve agentic workflow inputs under the contexts derived from hybrid program analysis for hijacking purposes. Specifically, JAW generates agentic workflow contexts through three analyses: (i) static path-feasibility analysis to identify feasible agent-invocation paths and the input constraints required to trigger them, (ii) dynamic prompt-provenance analysis to determine how that input is transformed and embedded into the LLM context, and (iii) capability analysis to identify the actions and restrictions available to the agent at runtime. Our evaluation of JAW on GitHub workflows and n8n templates showed that 4714 GitHub workflows and eight n8n templates can be successfully hijacked, for example, to leak user credentials. Our findings span 15 widely-used GitHub Actions, including official GitHub Actions for Claude Code, Gemini CLI, Qwen CLI, and Cursor CLI, and two official n8n nodes. We responsibly disclosed all findings to the affected vendors and received many acknowledgements, fixes, and bug bounties, notably from GitHub, Google, and Anthropic.

Related papers

• ToFu: A White-Box, Token-Efficient Agent Harness for Researchers [73.72929185946842]
  ToFu is an agentic harness for researchers that reads your files, edits files, runs commands, and integrates with your development tools.<n>As a research assistant, it supports practical research with superior token efficiency, lower cost, and multilingual capability.<n>As a research object, ToFu provides a white-box agentic harness that allows researchers to inspect, modify, and evaluate its orchestration logic, tool-use behavior, and harness design.
  arXiv  Detail & Related papers  (2026-07-13T11:26:06Z)
• GitInject: Real-World Prompt Injection Attacks in AI-Powered CI/CD Pipelines [22.29523291822444]
  We present GitInject, an open-source framework for evaluating prompt injection vulnerabilities in real, live GitHub.<n>We study workflow configurations across four AI providers and document eleven named attacks spanning config-file injection, credential exfiltration, judgment manipulation, and availability.<n>We find that all tested providers are susceptible to at least one attack class in their default configuration, and that the most critical vulnerabilities are structural.
  arXiv  Detail & Related papers  (2026-06-07T19:17:18Z)
• On the GitHub Actions Language: Usage, Evolution, and Workflow Reliability [44.257027748554606]
  This paper empirically explores how the adoption and evolution of GitHub Actions language constructs impacts workflow reliability and maintainability.<n>We quantitatively analyse 260K from 49K GitHub repositories to understand how they are used in practice and how their usage has evolved from July 2019 to August 2025.
  arXiv  Detail & Related papers  (2026-05-26T10:43:14Z)
• Code as Agent Harness [107.31925305395957]
  In emerging agentic systems, code is no longer only a target output.<n>Code increasingly serves as an operational substrate for agent reasoning, acting, environment modeling, and execution-based verification.<n>This survey provides a unified roadmap toward executable, verifiable, and stateful AI agent systems.
  arXiv  Detail & Related papers  (2026-05-18T17:59:03Z)
• Demystifying and Detecting Agentic Workflow Injection Vulnerabilities in GitHub Actions [13.116427545802571]
  GitHub Actions is increasingly used to deploy LLM-based agents for repository-centric tasks.<n>In this paper, we introduce Agentic Injection (AWI), a workflow-level injection flaw.<n>We characterize 1,033 real-world AI-assisted actions and extract AWI-specific specifications.<n>Based on these specifications, we design TaintAWI, a taint-analysis tool that tracks flows from untrusted event context to agent prompt inputs.
  arXiv  Detail & Related papers  (2026-05-08T02:13:04Z)
• From Threads to Trajectories: A Multi-LLM Pipeline for Community Knowledge Extraction from GitHub Issue Discussions [5.590965631053725]
  We present SWE-MIMIC-Bench, an issue trajectory dataset generated from raw GitHub discussions.<n>Unlike simple summarization, this pipeline utilizes a group of closed-source LLMs to perform granular tasks.<n>We evaluated our system on 800 real-world GitHub issues drawn from the SWE-Bench-Pro, SWE-Bench-Multilingual and SWE-Bench-Verified dataset.
  arXiv  Detail & Related papers  (2026-04-28T17:21:46Z)
• AJ-Bench: Benchmarking Agent-as-a-Judge for Environment-Aware Evaluation [71.49152943451328]
  We introduce a benchmark AJ-Bench to evaluate Agent-as-a-Judge across three domains-search, data systems, and graphical user interfaces-comprising 155 tasks and 516 annotated trajectories.<n>Experiments demonstrate consistent performance gains over LLM-as-a-Judge baselines, while also revealing substantial open challenges in agent-based verification.
  arXiv  Detail & Related papers  (2026-04-20T13:23:38Z)
• LLM-Augmented Release Intelligence: Automated Change Summarization and Impact Analysis in Cloud-Native CI/CD Pipelines [0.0]
  Cloud-native software delivery platforms orchestrate releases through complex, multi-stage pipelines composed of dozens of independently versioned tasks.<n>Manual preparation of such release communication is slow, inconsistent, and particularly error-prone in repositories.<n>We present a framework for AI-augmented release intelligence that combines three capabilities: (1) automated commit collection with semantic filtering to surface substantive changes while suppressing routine maintenance, (2) structured large language model summarization that produces categorized, stakeholder-oriented promotion reports, and (3) static task-pipeline dependency analysis that maps modified tasks to every pipeline they participate in, quantifying the blast radius of each change
  arXiv  Detail & Related papers  (2026-03-15T21:30:52Z)
• Unpacking Security Scanners for GitHub Actions Workflows [2.046588369793562]
  GitHub Actions is a widely used platform that allows developers to automate the build and deployment of their projects.<n>As the platform's popularity continues to grow, it has become a target of choice for recent software supply chain attacks.<n>Several security scanners have emerged to help developers harden their GitHub Actions.
  arXiv  Detail & Related papers  (2026-01-20T20:25:11Z)
• SwingArena: Competitive Programming Arena for Long-context GitHub Issue Solving [90.32201622392137]
  We present SwingArena, a competitive evaluation framework for Large Language Models (LLMs)<n>Unlike traditional static benchmarks, SwingArena models the collaborative process of software by pairing LLMs as iterations, who generate patches, and reviewers, who create test cases and verify the patches through continuous integration (CI) pipelines.
  arXiv  Detail & Related papers  (2025-05-29T18:28:02Z)
• Automatic Categorization of GitHub Actions with Transformers and Few-shot Learning [12.254055731378045]
  GitHub Actions (GHA) have been conceived to provide developers with a practical tool to create and maintain a pipeline. To expose actions to search engines, GitHub allows developers to assign them to one or more categories manually. We propose Gavel, a practical solution to increasing the visibility of actions in GitHub.
  arXiv  Detail & Related papers  (2024-07-24T02:27:36Z)
• Alibaba LingmaAgent: Improving Automated Issue Resolution via Comprehensive Repository Exploration [64.19431011897515]
  This paper presents Alibaba LingmaAgent, a novel Automated Software Engineering method designed to comprehensively understand and utilize whole software repositories for issue resolution.<n>Our approach introduces a top-down method to condense critical repository information into a knowledge graph, reducing complexity, and employs a Monte Carlo tree search based strategy.<n>In production deployment and evaluation at Alibaba Cloud, LingmaAgent automatically resolved 16.9% of in-house issues faced by development engineers, and solved 43.3% of problems after manual intervention.
  arXiv  Detail & Related papers  (2024-06-03T15:20:06Z)
• ML-Bench: Evaluating Large Language Models and Agents for Machine Learning Tasks on Repository-Level Code [76.84199699772903]
  ML-Bench is a benchmark rooted in real-world programming applications that leverage existing code repositories to perform tasks. To evaluate both Large Language Models (LLMs) and AI agents, two setups are employed: ML-LLM-Bench for assessing LLMs' text-to-code conversion within a predefined deployment environment, and ML-Agent-Bench for testing autonomous agents in an end-to-end task execution within a Linux sandbox environment.
  arXiv  Detail & Related papers  (2023-11-16T12:03:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.