ShieldLM: Empowering LLMs as Aligned, Customizable and Explainable Safety Detectors

• URL: http://arxiv.org/abs/2402.16444v2
• Date: Tue, 05 Nov 2024 02:13:59 GMT
• Title: ShieldLM: Empowering LLMs as Aligned, Customizable and Explainable Safety Detectors
• Authors: Zhexin Zhang, Yida Lu, Jingyuan Ma, Di Zhang, Rui Li, Pei Ke, Hao Sun, Lei Sha, Zhifang Sui, Hongning Wang, Minlie Huang,
• Abstract summary: ShieldLM is a safety detector for Large Language Models (LLMs) that aligns with common safety standards. We show that ShieldLM surpasses strong baselines across four test sets, showcasing remarkable customizability and explainability.
• Score: 90.73444232283371
• License:
• Abstract: The safety of Large Language Models (LLMs) has gained increasing attention in recent years, but there still lacks a comprehensive approach for detecting safety issues within LLMs' responses in an aligned, customizable and explainable manner. In this paper, we propose ShieldLM, an LLM-based safety detector, which aligns with common safety standards, supports customizable detection rules, and provides explanations for its decisions. To train ShieldLM, we compile a large bilingual dataset comprising 14,387 query-response pairs, annotating the safety of responses based on various safety standards. Through extensive experiments, we demonstrate that ShieldLM surpasses strong baselines across four test sets, showcasing remarkable customizability and explainability. Besides performing well on standard detection datasets, ShieldLM has also been shown to be effective as a safety evaluator for advanced LLMs. ShieldLM is released at \url{https://github.com/thu-coai/ShieldLM} to support accurate and explainable safety detection under various safety standards.

Related papers

• VLM-Guard: Safeguarding Vision-Language Models via Fulfilling Safety Alignment Gap [51.287157951953226]
  Vision language models (VLMs) come with increased safety concerns. VLMs can be built upon LLMs that have textual safety alignment, but it is easily undermined when the vision modality is integrated. We propose VLM-Guard, an inference-time intervention strategy that leverages the LLM component of a VLM as supervision for the safety alignment of the VLM.
  arXiv  Detail & Related papers  (2025-02-14T08:44:43Z)
• ASTRAL: Automated Safety Testing of Large Language Models [6.1050306667733185]
  Large Language Models (LLMs) have recently gained attention due to their ability to understand and generate sophisticated human-like content. We present ASTRAL, a tool that automates the generation and execution of test cases (i.e., prompts) for testing the safety of LLMs.
  arXiv  Detail & Related papers  (2025-01-28T18:25:11Z)
• Safety Layers in Aligned Large Language Models: The Key to LLM Security [43.805905164456846]
  Internal parameters in aligned LLMs can be vulnerable to security degradation when subjected to fine-tuning attacks. Our work uncovers the mechanism behind security in aligned LLMs at the parameter level, identifying a small set of contiguous layers in the middle of the model. We propose a novel fine-tuning approach, Safely Partial- Fine-Tuning (SPPFT), that fixes the gradient of the safety layers during fine-tuning to address the security degradation.
  arXiv  Detail & Related papers  (2024-08-30T04:35:59Z)
• SCANS: Mitigating the Exaggerated Safety for LLMs via Safety-Conscious Activation Steering [56.92068213969036]
  Safety alignment is indispensable for Large Language Models (LLMs) to defend threats from malicious instructions. Recent researches reveal safety-aligned LLMs prone to reject benign queries due to the exaggerated safety issue. We propose a Safety-Conscious Activation Steering (SCANS) method to mitigate the exaggerated safety concerns.
  arXiv  Detail & Related papers  (2024-08-21T10:01:34Z)
• MoGU: A Framework for Enhancing Safety of Open-Sourced LLMs While Preserving Their Usability [25.750371424096436]
  Large Language Models (LLMs) are increasingly deployed in various applications. Our research finds that existing defense strategies lead LLMs to predominantly adopt a rejection-oriented stance. We introduce the MoGU framework, designed to enhance LLMs' safety while preserving their usability.
  arXiv  Detail & Related papers  (2024-05-23T12:19:59Z)
• Towards Comprehensive Post Safety Alignment of Large Language Models via Safety Patching [74.62818936088065]
  textscSafePatching is a novel framework for comprehensive PSA. textscSafePatching achieves a more comprehensive PSA than baseline methods. textscSafePatching demonstrates its superiority in continual PSA scenarios.
  arXiv  Detail & Related papers  (2024-05-22T16:51:07Z)
• Eyes Closed, Safety On: Protecting Multimodal LLMs via Image-to-Text Transformation [98.02846901473697]
  We propose ECSO (Eyes Closed, Safety On), a training-free protecting approach that exploits the inherent safety awareness of MLLMs. ECSO generates safer responses via adaptively transforming unsafe images into texts to activate the intrinsic safety mechanism of pre-aligned LLMs.
  arXiv  Detail & Related papers  (2024-03-14T17:03:04Z)
• Safety of Multimodal Large Language Models on Images and Texts [33.97489213223888]
  In this paper, we systematically survey current efforts on the evaluation, attack, and defense of MLLMs' safety on images and text. We review the evaluation datasets and metrics for measuring the safety of MLLMs. Next, we comprehensively present attack and defense techniques related to MLLMs' safety.
  arXiv  Detail & Related papers  (2024-02-01T05:57:10Z)
• SafetyBench: Evaluating the Safety of Large Language Models [54.878612385780805]
  SafetyBench is a comprehensive benchmark for evaluating the safety of Large Language Models (LLMs) It comprises 11,435 diverse multiple choice questions spanning across 7 distinct categories of safety concerns. Our tests over 25 popular Chinese and English LLMs in both zero-shot and few-shot settings reveal a substantial performance advantage for GPT-4 over its counterparts.
  arXiv  Detail & Related papers  (2023-09-13T15:56:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.