HarmLevelBench: Evaluating Harm-Level Compliance and the Impact of Quantization on Model Alignment

• URL: http://arxiv.org/abs/2411.06835v1
• Date: Mon, 11 Nov 2024 10:02:49 GMT
• Title: HarmLevelBench: Evaluating Harm-Level Compliance and the Impact of Quantization on Model Alignment
• Authors: Yannis Belkhiter, Giulio Zizzo, Sergio Maffeis,
• Abstract summary: This paper aims to address gaps in the current literature on jailbreaking techniques and the evaluation of LLM vulnerabilities. Our contributions include the creation of a novel dataset designed to assess the harmfulness of model outputs across multiple harm levels. We provide a comprehensive benchmark of state-of-the-art jailbreaking attacks, specifically targeting the Vicuna 13B v1.5 model.
• Score: 1.8843687952462742
• License:
• Abstract: With the introduction of the transformers architecture, LLMs have revolutionized the NLP field with ever more powerful models. Nevertheless, their development came up with several challenges. The exponential growth in computational power and reasoning capabilities of language models has heightened concerns about their security. As models become more powerful, ensuring their safety has become a crucial focus in research. This paper aims to address gaps in the current literature on jailbreaking techniques and the evaluation of LLM vulnerabilities. Our contributions include the creation of a novel dataset designed to assess the harmfulness of model outputs across multiple harm levels, as well as a focus on fine-grained harm-level analysis. Using this framework, we provide a comprehensive benchmark of state-of-the-art jailbreaking attacks, specifically targeting the Vicuna 13B v1.5 model. Additionally, we examine how quantization techniques, such as AWQ and GPTQ, influence the alignment and robustness of models, revealing trade-offs between enhanced robustness with regards to transfer attacks and potential increases in vulnerability on direct ones. This study aims to demonstrate the influence of harmful input queries on the complexity of jailbreaking techniques, as well as to deepen our understanding of LLM vulnerabilities and improve methods for assessing model robustness when confronted with harmful content, particularly in the context of compression strategies.

Related papers

• Outside the Comfort Zone: Analysing LLM Capabilities in Software Vulnerability Detection [9.652886240532741]
  This paper thoroughly analyses large language models' capabilities in detecting vulnerabilities within source code. We evaluate the performance of six open-source models that are specifically trained for vulnerability detection against six general-purpose LLMs.
  arXiv  Detail & Related papers  (2024-08-29T10:00:57Z)
• ShieldGemma: Generative AI Content Moderation Based on Gemma [49.91147965876678]
  ShieldGemma is a suite of safety content moderation models built upon Gemma2. Models provide robust, state-of-the-art predictions of safety risks across key harm types.
  arXiv  Detail & Related papers  (2024-07-31T17:48:14Z)
• Threat Modelling and Risk Analysis for Large Language Model (LLM)-Powered Applications [0.0]
  Large Language Models (LLMs) have revolutionized various applications by providing advanced natural language processing capabilities. This paper explores the threat modeling and risk analysis specifically tailored for LLM-powered applications.
  arXiv  Detail & Related papers  (2024-06-16T16:43:58Z)
• Defending Large Language Models Against Attacks With Residual Stream Activation Analysis [0.0]
  Large Language Models (LLMs) are vulnerable to adversarial threats. This paper presents an innovative defensive strategy, given white box access to an LLM. We apply a novel methodology for analyzing distinctive activation patterns in the residual streams for attack prompt classification.
  arXiv  Detail & Related papers  (2024-06-05T13:06:33Z)
• Unveiling the Misuse Potential of Base Large Language Models via In-Context Learning [61.2224355547598]
  Open-sourcing of large language models (LLMs) accelerates application development, innovation, and scientific progress. Our investigation exposes a critical oversight in this belief. By deploying carefully designed demonstrations, our research demonstrates that base LLMs could effectively interpret and execute malicious instructions.
  arXiv  Detail & Related papers  (2024-04-16T13:22:54Z)
• Fine-Tuning, Quantization, and LLMs: Navigating Unintended Outcomes [0.0]
  Large Language Models (LLMs) have gained widespread adoption across various domains, including chatbots and auto-task completion agents. These models are susceptible to safety vulnerabilities such as jailbreaking, prompt injection, and privacy leakage attacks. This study investigates the impact of these modifications on LLM safety, a critical consideration for building reliable and secure AI systems.
  arXiv  Detail & Related papers  (2024-04-05T20:31:45Z)
• RigorLLM: Resilient Guardrails for Large Language Models against Undesired Content [62.685566387625975]
  Current mitigation strategies, while effective, are not resilient under adversarial attacks. This paper introduces Resilient Guardrails for Large Language Models (RigorLLM), a novel framework designed to efficiently moderate harmful and unsafe inputs.
  arXiv  Detail & Related papers  (2024-03-19T07:25:02Z)
• Breaking Down the Defenses: A Comparative Survey of Attacks on Large Language Models [18.624280305864804]
  Large Language Models (LLMs) have become a cornerstone in the field of Natural Language Processing (NLP) This paper presents a comprehensive survey of the various forms of attacks targeting LLMs. We delve into topics such as adversarial attacks that aim to manipulate model outputs, data poisoning that affects model training, and privacy concerns related to training data exploitation.
  arXiv  Detail & Related papers  (2024-03-03T04:46:21Z)
• Data Poisoning for In-context Learning [49.77204165250528]
  In-context learning (ICL) has been recognized for its innovative ability to adapt to new tasks. This paper delves into the critical issue of ICL's susceptibility to data poisoning attacks. We introduce ICLPoison, a specialized attacking framework conceived to exploit the learning mechanisms of ICL.
  arXiv  Detail & Related papers  (2024-02-03T14:20:20Z)
• Improving robustness of jet tagging algorithms with adversarial training [56.79800815519762]
  We investigate the vulnerability of flavor tagging algorithms via application of adversarial attacks. We present an adversarial training strategy that mitigates the impact of such simulated attacks.
  arXiv  Detail & Related papers  (2022-03-25T19:57:19Z)
• ML-Doctor: Holistic Risk Assessment of Inference Attacks Against Machine Learning Models [64.03398193325572]
  Inference attacks against Machine Learning (ML) models allow adversaries to learn about training data, model parameters, etc. We concentrate on four attacks - namely, membership inference, model inversion, attribute inference, and model stealing. Our analysis relies on a modular re-usable software, ML-Doctor, which enables ML model owners to assess the risks of deploying their models.
  arXiv  Detail & Related papers  (2021-02-04T11:35:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.