MobileSafetyBench: Evaluating Safety of Autonomous Agents in Mobile Device Control

• URL: http://arxiv.org/abs/2410.17520v2
• Date: Tue, 10 Dec 2024 11:56:09 GMT
• Title: MobileSafetyBench: Evaluating Safety of Autonomous Agents in Mobile Device Control
• Authors: Juyong Lee, Dongyoon Hahm, June Suk Choi, W. Bradley Knox, Kimin Lee,
• Abstract summary: We introduce MobileSafetyBench, a benchmark designed to evaluate the safety of mobile device-control agents. We develop a diverse set of tasks involving interactions with various mobile applications, including messaging and banking applications. Our experiments demonstrate that baseline agents, based on state-of-the-art LLMs, often fail to effectively prevent harm while performing the tasks.
• Score: 20.796190000442053
• License:
• Abstract: Autonomous agents powered by large language models (LLMs) show promising potential in assistive tasks across various domains, including mobile device control. As these agents interact directly with personal information and device settings, ensuring their safe and reliable behavior is crucial to prevent undesirable outcomes. However, no benchmark exists for standardized evaluation of the safety of mobile device-control agents. In this work, we introduce MobileSafetyBench, a benchmark designed to evaluate the safety of device-control agents within a realistic mobile environment based on Android emulators. We develop a diverse set of tasks involving interactions with various mobile applications, including messaging and banking applications, challenging agents with managing risks encompassing misuse and negative side effects. These tasks include tests to evaluate the safety of agents in daily scenarios as well as their robustness against indirect prompt injection attacks. Our experiments demonstrate that baseline agents, based on state-of-the-art LLMs, often fail to effectively prevent harm while performing the tasks. To mitigate these safety concerns, we propose a prompting method that encourages agents to prioritize safety considerations. While this method shows promise in promoting safer behaviors, there is still considerable room for improvement to fully earn user trust. This highlights the urgent need for continued research to develop more robust safety mechanisms in mobile environments. We open-source our benchmark at: https://mobilesafetybench.github.io/.

Related papers

• AGrail: A Lifelong Agent Guardrail with Effective and Adaptive Safety Detection [47.83354878065321]
  We propose AGrail, a lifelong guardrail to enhance agent safety. AGrail features adaptive safety check generation, effective safety check optimization, and tool compatibility and flexibility.
  arXiv  Detail & Related papers  (2025-02-17T05:12:33Z)
• AgentGuard: Repurposing Agentic Orchestrator for Safety Evaluation of Tool Orchestration [0.3222802562733787]
  AgentGuard is a framework to autonomously discover and validate unsafe tool-use. It generates safety constraints to confine the behaviors of agents, achieving the baseline of safety guarantee. The framework operates through four phases: identifying unsafe, validating them in real-world execution, generating safety constraints, and validating constraint efficacy.
  arXiv  Detail & Related papers  (2025-02-13T23:00:33Z)
• Agent-SafetyBench: Evaluating the Safety of LLM Agents [72.92604341646691]
  We introduce Agent-SafetyBench, a comprehensive benchmark to evaluate the safety of large language models (LLMs) Agent-SafetyBench encompasses 349 interaction environments and 2,000 test cases, evaluating 8 categories of safety risks and covering 10 common failure modes frequently encountered in unsafe interactions. Our evaluation of 16 popular LLM agents reveals a concerning result: none of the agents achieves a safety score above 60%.
  arXiv  Detail & Related papers  (2024-12-19T02:35:15Z)
• ST-WebAgentBench: A Benchmark for Evaluating Safety and Trustworthiness in Web Agents [3.09793323158304]
  We present STWebAgentBench, a benchmark designed to evaluate web agents safety and trustworthiness across six critical dimensions. This benchmark is grounded in a detailed framework that defines safe and trustworthy (ST) agent behavior. We open-source this benchmark and invite the community to contribute, with the goal of fostering a new generation of safer, more trustworthy AI agents.
  arXiv  Detail & Related papers  (2024-10-09T09:13:38Z)
• SCANS: Mitigating the Exaggerated Safety for LLMs via Safety-Conscious Activation Steering [56.92068213969036]
  Safety alignment is indispensable for Large Language Models (LLMs) to defend threats from malicious instructions. Recent researches reveal safety-aligned LLMs prone to reject benign queries due to the exaggerated safety issue. We propose a Safety-Conscious Activation Steering (SCANS) method to mitigate the exaggerated safety concerns.
  arXiv  Detail & Related papers  (2024-08-21T10:01:34Z)
• On Prompt-Driven Safeguarding for Large Language Models [172.13943777203377]
  We find that in the representation space, the input queries are typically moved by safety prompts in a "higher-refusal" direction. Inspired by these findings, we propose a method for safety prompt optimization, namely DRO. Treating a safety prompt as continuous, trainable embeddings, DRO learns to move the queries' representations along or opposite the refusal direction, depending on their harmfulness.
  arXiv  Detail & Related papers  (2024-01-31T17:28:24Z)
• Testing Language Model Agents Safely in the Wild [19.507292491433738]
  We propose a framework for conducting safe autonomous agent tests on the open internet. Agent actions are audited by a context-sensitive monitor that enforces a stringent safety boundary. Using an adversarial simulated agent, we measure its ability to identify and stop unsafe situations.
  arXiv  Detail & Related papers  (2023-11-17T14:06:05Z)
• Safety Margins for Reinforcement Learning [53.10194953873209]
  We show how to leverage proxy criticality metrics to generate safety margins. We evaluate our approach on learned policies from APE-X and A3C within an Atari environment.
  arXiv  Detail & Related papers  (2023-07-25T16:49:54Z)
• Safe Reinforcement Learning via Curriculum Induction [94.67835258431202]
  In safety-critical applications, autonomous agents may need to learn in an environment where mistakes can be very costly. Existing safe reinforcement learning methods make an agent rely on priors that let it avoid dangerous situations. This paper presents an alternative approach inspired by human teaching, where an agent learns under the supervision of an automatic instructor.
  arXiv  Detail & Related papers  (2020-06-22T10:48:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.