Interactive Tools Substantially Assist LM Agents in Finding Security Vulnerabilities

• URL: http://arxiv.org/abs/2409.16165v2
• Date: Tue, 04 Feb 2025 09:49:41 GMT
• Title: Interactive Tools Substantially Assist LM Agents in Finding Security Vulnerabilities
• Authors: Talor Abramovich, Meet Udeshi, Minghao Shao, Kilian Lieret, Haoran Xi, Kimberly Milner, Sofija Jancheska, John Yang, Carlos E. Jimenez, Farshad Khorrami, Prashanth Krishnamurthy, Brendan Dolan-Gavitt, Muhammad Shafique, Karthik Narasimhan, Ramesh Karri, Ofir Press,
• Abstract summary: We present EnIGMA, an LM agent for autonomously solving Capture The Flag (CTF) challenges.<n>We introduce new tools and interfaces to improve the agent's ability to find and exploit security vulnerabilities.<n> Empirical analysis on 390 CTF challenges demonstrate that these new tools and interfaces substantially improve our agent's performance.
• Score: 46.34031902647788
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Although language model (LM) agents have demonstrated increased performance in multiple domains, including coding and web-browsing, their success in cybersecurity has been limited. We present EnIGMA, an LM agent for autonomously solving Capture The Flag (CTF) challenges. We introduce new tools and interfaces to improve the agent's ability to find and exploit security vulnerabilities, focusing on interactive terminal programs. These novel Interactive Agent Tools enable LM agents, for the first time, to run interactive utilities, such as a debugger and a server connection tool, which are essential for solving these challenges. Empirical analysis on 390 CTF challenges across four benchmarks demonstrate that these new tools and interfaces substantially improve our agent's performance, achieving state-of-the-art results on NYU CTF, Intercode-CTF, and CyBench. Finally, we analyze data leakage, developing new methods to quantify it and identifying a new phenomenon we term soliloquizing, where the model self-generates hallucinated observations without interacting with the environment. Our code and development dataset are available at https://github.com/SWE-agent/SWE-agent/tree/v0.7 and https://github.com/NYU-LLM-CTF/NYU_CTF_Bench/tree/main/development respectively.

Related papers

• Adaptive Tool Use in Large Language Models with Meta-Cognition Trigger [49.81945268343162]
  We propose MeCo, an adaptive decision-making strategy for external tool use. MeCo captures high-level cognitive signals in the representation space, guiding when to invoke tools. Our experiments show that MeCo accurately detects LLMs' internal cognitive signals and significantly improves tool-use decision-making.
  arXiv  Detail & Related papers  (2025-02-18T15:45:01Z)
• SMART: Self-Aware Agent for Tool Overuse Mitigation [58.748554080273585]
  Current Large Language Model (LLM) agents demonstrate strong reasoning and tool use capabilities, but often lack self-awareness. This imbalance leads to Tool Overuse, where models unnecessarily rely on external tools for tasks with parametric knowledge. We introduce SMART (Strategic Model-Aware Reasoning with Tools), a paradigm that enhances an agent's self-awareness to optimize task handling and reduce tool overuse.
  arXiv  Detail & Related papers  (2025-02-17T04:50:37Z)
• D-CIPHER: Dynamic Collaborative Intelligent Agents with Planning and Heterogeneous Execution for Enhanced Reasoning in Offensive Security [22.86304661035188]
  Large Language Models (LLMs) have been used in cybersecurity in many ways. Capture the Flag (CTF) challenges serve as benchmarks for assessing the automated task-planning abilities of LLM agents. We introduce the D-CIPHER multi-agent LLM framework for collaborative CTF challenge solving.
  arXiv  Detail & Related papers  (2025-02-15T23:43:18Z)
• GUI Agents with Foundation Models: A Comprehensive Survey [52.991688542729385]
  This survey consolidates recent research on (M)LLM-based GUI agents. We highlight key innovations in data, frameworks, and applications. We hope this paper will inspire further developments in the field of (M)LLM-based GUI agents.
  arXiv  Detail & Related papers  (2024-11-07T17:28:10Z)
• SPA-Bench: A Comprehensive Benchmark for SmartPhone Agent Evaluation [89.24729958546168]
  We present SPA-Bench, a comprehensive SmartPhone Agent Benchmark designed to evaluate (M)LLM-based agents. SPA-Bench offers three key contributions: A diverse set of tasks covering system and third-party apps in both English and Chinese, focusing on features commonly used in daily routines. A novel evaluation pipeline that automatically assesses agent performance across multiple dimensions, encompassing seven metrics related to task completion and resource consumption.
  arXiv  Detail & Related papers  (2024-10-19T17:28:48Z)
• Textualized Agent-Style Reasoning for Complex Tasks by Multiple Round LLM Generation [49.27250832754313]
  We present AgentCOT, a llm-based autonomous agent framework. At each step, AgentCOT selects an action and executes it to yield an intermediate result with supporting evidence. We introduce two new strategies to enhance the performance of AgentCOT.
  arXiv  Detail & Related papers  (2024-09-19T02:20:06Z)
• LLM-Agent-UMF: LLM-based Agent Unified Modeling Framework for Seamless Integration of Multi Active/Passive Core-Agents [0.0]
  We propose a novel LLM-based Agent Unified Modeling Framework (LLM-Agent-UMF) Our framework distinguishes between the different components of an LLM-based agent, setting LLMs and tools apart from a new element, the core-agent. We evaluate our framework by applying it to thirteen state-of-the-art agents, thereby demonstrating its alignment with their functionalities.
  arXiv  Detail & Related papers  (2024-09-17T17:54:17Z)
• AutoSafeCoder: A Multi-Agent Framework for Securing LLM Code Generation through Static Analysis and Fuzz Testing [6.334110674473677]
  Existing approaches often rely on a single agent for code generation, which struggles to produce secure, vulnerability-free code. We propose AutoSafeCoder, a multi-agent framework that leverages LLM-driven agents for code generation, vulnerability analysis, and security enhancement through continuous collaboration. Our contribution focuses on ensuring the safety of multi-agent code generation by integrating dynamic and static testing in an iterative process during code generation.
  arXiv  Detail & Related papers  (2024-09-16T21:15:56Z)
• Hacking, The Lazy Way: LLM Augmented Pentesting [0.0]
  "LLM Augmented Pentesting" is demonstrated through a tool named "Pentest Copilot" Our research includes a "chain of thought" mechanism to streamline token usage and boost performance. We propose a novel file analysis approach, enabling LLMs to understand files.
  arXiv  Detail & Related papers  (2024-09-14T17:40:35Z)
• MMAU: A Holistic Benchmark of Agent Capabilities Across Diverse Domains [54.117238759317004]
  Massive Multitask Agent Understanding (MMAU) benchmark features comprehensive offline tasks that eliminate the need for complex environment setups. It evaluates models across five domains, including Tool-use, Directed Acyclic Graph (DAG) QA, Data Science and Machine Learning coding, Contest-level programming and Mathematics. With a total of 20 meticulously designed tasks encompassing over 3K distinct prompts, MMAU provides a comprehensive framework for evaluating the strengths and limitations of LLM agents.
  arXiv  Detail & Related papers  (2024-07-18T00:58:41Z)
• Dissecting Adversarial Robustness of Multimodal LM Agents [70.2077308846307]
  We manually create 200 targeted adversarial tasks and evaluation scripts in a realistic threat model on top of VisualWebArena. We find that we can successfully break latest agents that use black-box frontier LMs, including those that perform reflection and tree search. We also use ARE to rigorously evaluate how the robustness changes as new components are added.
  arXiv  Detail & Related papers  (2024-06-18T17:32:48Z)
• Alibaba LingmaAgent: Improving Automated Issue Resolution via Comprehensive Repository Exploration [64.19431011897515]
  This paper presents Alibaba LingmaAgent, a novel Automated Software Engineering method designed to comprehensively understand and utilize whole software repositories for issue resolution. Our approach introduces a top-down method to condense critical repository information into a knowledge graph, reducing complexity, and employs a Monte Carlo tree search based strategy. In production deployment and evaluation at Alibaba Cloud, LingmaAgent automatically resolved 16.9% of in-house issues faced by development engineers, and solved 43.3% of problems after manual intervention.
  arXiv  Detail & Related papers  (2024-06-03T15:20:06Z)
• SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineering [79.07755560048388]
  SWE-agent is a system that facilitates LM agents to autonomously use computers to solve software engineering tasks. SWE-agent's custom agent-computer interface (ACI) significantly enhances an agent's ability to create and edit code files, navigate entire repositories, and execute tests and other programs. We evaluate SWE-agent on SWE-bench and HumanEvalFix, achieving state-of-the-art performance on both with a pass@1 rate of 12.5% and 87.7%, respectively.
  arXiv  Detail & Related papers  (2024-05-06T17:41:33Z)
• Understanding the Weakness of Large Language Model Agents within a Complex Android Environment [21.278266207772756]
  Large language models (LLMs) have empowered intelligent agents to execute intricate tasks within domain-specific software such as browsers and games. LLMs face three primary challenges when applied to general-purpose software systems like operating systems. These challenges motivate AndroidArena, an environment and benchmark designed to evaluate LLM agents on a modern operating system.
  arXiv  Detail & Related papers  (2024-02-09T18:19:25Z)
• Language Agent Tree Search Unifies Reasoning Acting and Planning in Language Models [31.509994889286183]
  We introduce Language Agent Tree Search (LATS) -- the first general framework that synergizes the capabilities of language models (LMs) in reasoning, acting, and planning. A key feature of our approach is the incorporation of an environment for external feedback, which offers a more deliberate and adaptive problem-solving mechanism. LATS achieves state-of-the-art pass@1 accuracy (92.7%) for programming on HumanEval with GPT-4 and demonstrates gradient-free performance (average score of 75.9) comparable to gradient-based fine-tuning for web navigation on WebShop with GPT
  arXiv  Detail & Related papers  (2023-10-06T17:55:11Z)
• Enhancing Trust in LLM-Based AI Automation Agents: New Considerations and Future Challenges [2.6212127510234797]
  In the field of process automation, a new generation of AI-based agents has emerged, enabling the execution of complex tasks. This paper analyzes the main aspects of trust in AI agents discussed in existing literature, and identifies specific considerations and challenges relevant to this new generation of automation agents.
  arXiv  Detail & Related papers  (2023-08-10T07:12:11Z)
• Trojaning Language Models for Fun and Profit [53.45727748224679]
  TROJAN-LM is a new class of trojaning attacks in which maliciously crafted LMs trigger host NLP systems to malfunction. By empirically studying three state-of-the-art LMs in a range of security-critical NLP tasks, we demonstrate that TROJAN-LM possesses the following properties.
  arXiv  Detail & Related papers  (2020-08-01T18:22:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.