Related papers: Generating Robot Constitutions & Benchmarks for Semantic Safety

Generating Robot Constitutions & Benchmarks for Semantic Safety

URL: http://arxiv.org/abs/2503.08663v1
Date: Tue, 11 Mar 2025 17:50:47 GMT
Title: Generating Robot Constitutions & Benchmarks for Semantic Safety
Authors: Pierre Sermanet, Anirudha Majumdar, Alex Irpan, Dmitry Kalashnikov, Vikas Sindhwani,
Abstract summary: We release the ASIMOV Benchmark for evaluating semantic safety of robot brains.<n>We develop a framework to automatically generate robot constitutions from real-world data.<n>We propose a novel auto-amending process that is able to introduce nuances in written rules of behavior.
Score: 22.889717765617394
License: http://creativecommons.org/licenses/by/4.0/
Abstract: Until recently, robotics safety research was predominantly about collision avoidance and hazard reduction in the immediate vicinity of a robot. Since the advent of large vision and language models (VLMs), robots are now also capable of higher-level semantic scene understanding and natural language interactions with humans. Despite their known vulnerabilities (e.g. hallucinations or jail-breaking), VLMs are being handed control of robots capable of physical contact with the real world. This can lead to dangerous behaviors, making semantic safety for robots a matter of immediate concern. Our contributions in this paper are two fold: first, to address these emerging risks, we release the ASIMOV Benchmark, a large-scale and comprehensive collection of datasets for evaluating and improving semantic safety of foundation models serving as robot brains. Our data generation recipe is highly scalable: by leveraging text and image generation techniques, we generate undesirable situations from real-world visual scenes and human injury reports from hospitals. Secondly, we develop a framework to automatically generate robot constitutions from real-world data to steer a robot's behavior using Constitutional AI mechanisms. We propose a novel auto-amending process that is able to introduce nuances in written rules of behavior; this can lead to increased alignment with human preferences on behavior desirability and safety. We explore trade-offs between generality and specificity across a diverse set of constitutions of different lengths, and demonstrate that a robot is able to effectively reject unconstitutional actions. We measure a top alignment rate of 84.3% on the ASIMOV Benchmark using generated constitutions, outperforming no-constitution baselines and human-written constitutions. Data is available at asimov-benchmark.github.io

Related papers

SciFi-Benchmark: How Would AI-Powered Robots Behave in Science Fiction Literature? [20.51881907653089]
We generate a benchmark spanning the key moments in 824 pieces of science fiction literature. We use a LLM's recollection of each key moment to generate questions in similar situations. We then measure an approximation of how well models align with human values on a set of human-voted answers.
arXiv Detail & Related papers (2025-03-12T16:35:51Z)
GNN-based Decentralized Perception in Multirobot Systems for Predicting Worker Actions [12.260881600042374]
This paper introduces a perception framework that enables mobile robots to understand and share information about human actions in a decentralized way.<n>A swarm-inspired decision-making process is used to ensure all robots agree on a unified interpretation of the human's actions.
arXiv Detail & Related papers (2025-01-08T00:06:38Z)
$π_0$: A Vision-Language-Action Flow Model for General Robot Control [77.32743739202543]
We propose a novel flow matching architecture built on top of a pre-trained vision-language model (VLM) to inherit Internet-scale semantic knowledge. We evaluate our model in terms of its ability to perform tasks in zero shot after pre-training, follow language instructions from people, and its ability to acquire new skills via fine-tuning.
arXiv Detail & Related papers (2024-10-31T17:22:30Z)
Know your limits! Optimize the robot's behavior through self-awareness [11.021217430606042]
Recent human-robot imitation algorithms focus on following a reference human motion with high precision. We introduce a deep-learning model that anticipates the robot's performance when imitating a given reference. Our Self-AWare model (SAW) ranks potential robot behaviors based on various criteria, such as fall likelihood, adherence to the reference motion, and smoothness.
arXiv Detail & Related papers (2024-09-16T14:14:58Z)
GRUtopia: Dream General Robots in a City at Scale [65.08318324604116]
This paper introduces project GRUtopia, the first simulated interactive 3D society designed for various robots. GRScenes includes 100k interactive, finely annotated scenes, which can be freely combined into city-scale environments. GRResidents is a Large Language Model (LLM) driven Non-Player Character (NPC) system that is responsible for social interaction.
arXiv Detail & Related papers (2024-07-15T17:40:46Z)
HumanoidBench: Simulated Humanoid Benchmark for Whole-Body Locomotion and Manipulation [50.616995671367704]
We present a high-dimensional, simulated robot learning benchmark, HumanoidBench, featuring a humanoid robot equipped with dexterous hands. Our findings reveal that state-of-the-art reinforcement learning algorithms struggle with most tasks, whereas a hierarchical learning approach achieves superior performance when supported by robust low-level policies.
arXiv Detail & Related papers (2024-03-15T17:45:44Z)
RoboScript: Code Generation for Free-Form Manipulation Tasks across Real and Simulation [77.41969287400977]
This paper presents textbfRobotScript, a platform for a deployable robot manipulation pipeline powered by code generation. We also present a benchmark for a code generation benchmark for robot manipulation tasks in free-form natural language. We demonstrate the adaptability of our code generation framework across multiple robot embodiments, including the Franka and UR5 robot arms.
arXiv Detail & Related papers (2024-02-22T15:12:00Z)
Cybersecurity and Embodiment Integrity for Modern Robots: A Conceptual Framework [3.29295880899738]
We show how cyberattacks on different devices can have radically different consequences on the robot's ability to complete its tasks. We also claim that modern robots should have self-awareness for what it concerns such aspects. We show that achieving these propositions requires that robots possess at least three properties that conceptually link devices and tasks.
arXiv Detail & Related papers (2024-01-15T15:46:38Z)
ImitationNet: Unsupervised Human-to-Robot Motion Retargeting via Shared Latent Space [9.806227900768926]
This paper introduces a novel deep-learning approach for human-to-robot motion. Our method does not require paired human-to-robot data, which facilitates its translation to new robots. Our model outperforms existing works regarding human-to-robot similarity in terms of efficiency and precision.
arXiv Detail & Related papers (2023-09-11T08:55:04Z)
Robot Learning with Sensorimotor Pre-training [98.7755895548928]
We present a self-supervised sensorimotor pre-training approach for robotics. Our model, called RPT, is a Transformer that operates on sequences of sensorimotor tokens. We find that sensorimotor pre-training consistently outperforms training from scratch, has favorable scaling properties, and enables transfer across different tasks, environments, and robots.
arXiv Detail & Related papers (2023-06-16T17:58:10Z)
Self-Improving Robots: End-to-End Autonomous Visuomotor Reinforcement Learning [54.636562516974884]
In imitation and reinforcement learning, the cost of human supervision limits the amount of data that robots can be trained on. In this work, we propose MEDAL++, a novel design for self-improving robotic systems. The robot autonomously practices the task by learning to both do and undo the task, simultaneously inferring the reward function from the demonstrations.
arXiv Detail & Related papers (2023-03-02T18:51:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.