LLM-Driven Robots Risk Enacting Discrimination, Violence, and Unlawful Actions

• URL: http://arxiv.org/abs/2406.08824v1
• Date: Thu, 13 Jun 2024 05:31:49 GMT
• Title: LLM-Driven Robots Risk Enacting Discrimination, Violence, and Unlawful Actions
• Authors: Rumaisa Azeem, Andrew Hundt, Masoumeh Mansouri, Martim Brandão,
• Abstract summary: Research has raised concerns about the potential for Large Language Models to produce discriminatory outcomes and unsafe behaviors in real-world robot experiments and applications. We conduct an HRI-based evaluation of discrimination and safety criteria on several highly-rated LLMs. Our results underscore the urgent need for systematic, routine, and comprehensive risk assessments and assurances to improve outcomes.
• Score: 3.1247504290622214
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Members of the Human-Robot Interaction (HRI) and Artificial Intelligence (AI) communities have proposed Large Language Models (LLMs) as a promising resource for robotics tasks such as natural language interactions, doing household and workplace tasks, approximating `common sense reasoning', and modeling humans. However, recent research has raised concerns about the potential for LLMs to produce discriminatory outcomes and unsafe behaviors in real-world robot experiments and applications. To address these concerns, we conduct an HRI-based evaluation of discrimination and safety criteria on several highly-rated LLMs. Our evaluation reveals that LLMs currently lack robustness when encountering people across a diverse range of protected identity characteristics (e.g., race, gender, disability status, nationality, religion, and their intersections), producing biased outputs consistent with directly discriminatory outcomes -- e.g. `gypsy' and `mute' people are labeled untrustworthy, but not `european' or `able-bodied' people. Furthermore, we test models in settings with unconstrained natural language (open vocabulary) inputs, and find they fail to act safely, generating responses that accept dangerous, violent, or unlawful instructions -- such as incident-causing misstatements, taking people's mobility aids, and sexual predation. Our results underscore the urgent need for systematic, routine, and comprehensive risk assessments and assurances to improve outcomes and ensure LLMs only operate on robots when it is safe, effective, and just to do so. Data and code will be made available.

Related papers

• Persuasion with Large Language Models: a Survey [49.86930318312291]
  Large Language Models (LLMs) have created new disruptive possibilities for persuasive communication. In areas such as politics, marketing, public health, e-commerce, and charitable giving, such LLM Systems have already achieved human-level or even super-human persuasiveness. Our survey suggests that the current and future potential of LLM-based persuasion poses profound ethical and societal risks.
  arXiv  Detail & Related papers  (2024-11-11T10:05:52Z)
• Generative LLM Powered Conversational AI Application for Personalized Risk Assessment: A Case Study in COVID-19 [6.367429891237191]
  Large language models (LLMs) have shown remarkable capabilities in various natural language tasks. This work demonstrates a new LLM-powered disease risk assessment approach via streaming human-AI conversation.
  arXiv  Detail & Related papers  (2024-09-23T13:55:13Z)
• BadRobot: Jailbreaking Embodied LLMs in the Physical World [20.96351292684658]
  Embodied AI represents systems where AI is integrated into physical entities. Large Language Model (LLM) exhibits powerful language understanding abilities. We introduce BadRobot, a novel attack paradigm aiming to make embodied LLMs violate safety and ethical constraints through typical voice-based user-system interactions.
  arXiv  Detail & Related papers  (2024-07-16T13:13:16Z)
• Unveiling the Misuse Potential of Base Large Language Models via In-Context Learning [61.2224355547598]
  Open-sourcing of large language models (LLMs) accelerates application development, innovation, and scientific progress. Our investigation exposes a critical oversight in this belief. By deploying carefully designed demonstrations, our research demonstrates that base LLMs could effectively interpret and execute malicious instructions.
  arXiv  Detail & Related papers  (2024-04-16T13:22:54Z)
• ALERT: A Comprehensive Benchmark for Assessing Large Language Models' Safety through Red Teaming [64.86326523181553]
  ALERT is a large-scale benchmark to assess safety based on a novel fine-grained risk taxonomy. It aims to identify vulnerabilities, inform improvements, and enhance the overall safety of the language models.
  arXiv  Detail & Related papers  (2024-04-06T15:01:47Z)
• Leveraging Large Language Models in Human-Robot Interaction: A Critical Analysis of Potential and Pitfalls [0.0]
  Large language models (LLM) and vision language models (VLM) present unprecedented opportunities and challenges for socially assistive robots (SARs) We conduct a meta-study of more than 250 papers exploring major robots in HRI research and significant applications of SARs, emphasizing education, healthcare, and entertainment while addressing societal norms and issues like trust, bias, and ethics that the robot developers must address. We outline a pathway for the responsible and effective adoption of LLM or VLM into SARs, and we close our discussion by offering caution regarding this deployment.
  arXiv  Detail & Related papers  (2024-03-26T15:36:40Z)
• The Wolf Within: Covert Injection of Malice into MLLM Societies via an MLLM Operative [55.08395463562242]
  Multimodal Large Language Models (MLLMs) are constantly defining the new boundary of Artificial General Intelligence (AGI) Our paper explores a novel vulnerability in MLLM societies - the indirect propagation of malicious content.
  arXiv  Detail & Related papers  (2024-02-20T23:08:21Z)
• Highlighting the Safety Concerns of Deploying LLMs/VLMs in Robotics [54.57914943017522]
  We highlight the critical issues of robustness and safety associated with integrating large language models (LLMs) and vision-language models (VLMs) into robotics applications.
  arXiv  Detail & Related papers  (2024-02-15T22:01:45Z)
• Relying on the Unreliable: The Impact of Language Models' Reluctance to Express Uncertainty [53.336235704123915]
  We investigate how LMs incorporate confidence in responses via natural language and how downstream users behave in response to LM-articulated uncertainties. We find that LMs are reluctant to express uncertainties when answering questions even when they produce incorrect responses. We test the risks of LM overconfidence by conducting human experiments and show that users rely heavily on LM generations. Lastly, we investigate the preference-annotated datasets used in post training alignment and find that humans are biased against texts with uncertainty.
  arXiv  Detail & Related papers  (2024-01-12T18:03:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.