ELITE: Enhanced Language-Image Toxicity Evaluation for Safety

• URL: http://arxiv.org/abs/2502.04757v2
• Date: Mon, 10 Feb 2025 04:39:28 GMT
• Title: ELITE: Enhanced Language-Image Toxicity Evaluation for Safety
• Authors: Wonjun Lee, Doehyeon Lee, Eugene Choi, Sangyoon Yu, Ashkan Yousefpour, Haon Park, Bumsub Ham, Suhyun Kim,
• Abstract summary: Current Vision Language Models (VLMs) remain vulnerable to malicious prompts that induce harmful outputs.<n>Existing benchmarks have low levels of harmfulness, ambiguous data, and limited diversity in image-text pair combinations.<n>We propose the ELITE benchmark, a high-quality safety evaluation benchmark for VLMs, underpinned by our enhanced evaluation method, the ELITE evaluator.
• Score: 22.371913404553545
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Current Vision Language Models (VLMs) remain vulnerable to malicious prompts that induce harmful outputs. Existing safety benchmarks for VLMs primarily rely on automated evaluation methods, but these methods struggle to detect implicit harmful content or produce inaccurate evaluations. Therefore, we found that existing benchmarks have low levels of harmfulness, ambiguous data, and limited diversity in image-text pair combinations. To address these issues, we propose the ELITE benchmark, a high-quality safety evaluation benchmark for VLMs, underpinned by our enhanced evaluation method, the ELITE evaluator. The ELITE evaluator explicitly incorporates a toxicity score to accurately assess harmfulness in multimodal contexts, where VLMs often provide specific, convincing, but unharmful descriptions of images. We filter out ambiguous and low-quality image-text pairs from existing benchmarks using the ELITE evaluator and generate diverse combinations of safe and unsafe image-text pairs. Our experiments demonstrate that the ELITE evaluator achieves superior alignment with human evaluations compared to prior automated methods, and the ELITE benchmark offers enhanced benchmark quality and diversity. By introducing ELITE, we pave the way for safer, more robust VLMs, contributing essential tools for evaluating and mitigating safety risks in real-world applications.

Related papers

• Automated Safety Benchmarking: A Multi-agent Pipeline for LVLMs [61.01470415470677]
  Large vision-language models (LVLMs) exhibit remarkable capabilities in cross-modal tasks but face significant safety challenges.<n>Existing benchmarks are hindered by their labor-intensive construction process, static complexity, and limited discriminative power.<n>We propose VLSafetyBencher, the first automated system for LVLM safety benchmarking.
  arXiv  Detail & Related papers  (2026-01-27T11:51:30Z)
• Value-Aligned Prompt Moderation via Zero-Shot Agentic Rewriting for Safe Image Generation [11.663809872664103]
  Current defenses struggle to align outputs with human values without sacrificing generation quality or incurring high costs.<n>We introduce VALOR, a zero-shot agentic framework for safer and more helpful text-to-image generation.<n>VALOR integrates layered prompt analysis with human-aligned value reasoning.
  arXiv  Detail & Related papers  (2025-11-12T09:52:47Z)
• ROSE: Toward Reality-Oriented Safety Evaluation of Large Language Models [60.28667314609623]
  Large Language Models (LLMs) are increasingly deployed as black-box components in real-world applications.<n>We propose Reality-Oriented Safety Evaluation (ROSE), a novel framework that uses multi-objective reinforcement learning to fine-tune an adversarial LLM.
  arXiv  Detail & Related papers  (2025-06-17T10:55:17Z)
• HoliSafe: Holistic Safety Benchmarking and Modeling with Safety Meta Token for Vision-Language Model [52.72318433518926]
  Existing safety-tuning datasets and benchmarks only partially consider how image-text interactions can yield harmful content.<n>We introduce a holistic safety dataset and benchmark, HoliSafe, that spans all five safe/unsafe image-text combinations.<n>We propose SafeLLaVA, a novel VLM augmented with a learnable safety meta token and a dedicated safety head.
  arXiv  Detail & Related papers  (2025-06-05T07:26:34Z)
• $\texttt{SAGE}$: A Generic Framework for LLM Safety Evaluation [9.935219917903858]
  This paper introduces the $texttSAGE$ (Safety AI Generic Evaluation) framework. $texttSAGE$ is an automated modular framework designed for customized and dynamic harm evaluations. Our experiments with multi-turn conversational evaluations revealed a concerning finding that harm steadily increases with conversation length.
  arXiv  Detail & Related papers  (2025-04-28T11:01:08Z)
• MEQA: A Meta-Evaluation Framework for Question & Answer LLM Benchmarks [0.0]
  We propose MEQA, a framework for the meta-evaluation of question and answer (QA) benchmarks. We demonstrate this approach on cybersecurity benchmarks, using human and LLM evaluators. We motivate our choice of test domain by AI models' dual nature as powerful defensive tools and security threats.
  arXiv  Detail & Related papers  (2025-04-18T19:01:53Z)
• A Framework for Evaluating Vision-Language Model Safety: Building Trust in AI for Public Sector Applications [0.0]
  This paper introduces a novel framework to quantify adversarial risks in Vision-Language Models (VLMs) We analyze model performance under Gaussian, salt-and-pepper, and uniform noise, identifying misclassification thresholds and deriving composite noise patches and saliency patterns that highlight vulnerable regions. We propose a new Vulnerability Score that combines the impact of random noise and adversarial attacks, providing a comprehensive metric for evaluating model robustness.
  arXiv  Detail & Related papers  (2025-02-22T21:33:26Z)
• Retention Score: Quantifying Jailbreak Risks for Vision Language Models [60.48306899271866]
  Vision-Language Models (VLMs) are integrated with Large Language Models (LLMs) to enhance multi-modal machine learning capabilities. This paper aims to assess the resilience of VLMs against jailbreak attacks that can compromise model safety compliance and result in harmful outputs. To evaluate a VLM's ability to maintain its robustness against adversarial input perturbations, we propose a novel metric called the textbfRetention Score.
  arXiv  Detail & Related papers  (2024-12-23T13:05:51Z)
• The Vulnerability of Language Model Benchmarks: Do They Accurately Reflect True LLM Performance? [1.3810901729134184]
  Large Language Models (LLMs) excel at standardized tests while failing to demonstrate genuine language understanding and adaptability.<n>Our systematic analysis of NLP evaluation frameworks reveals pervasive vulnerabilities across the evaluation spectrum.<n>We lay the groundwork for new evaluation methods that resist manipulation, minimize data contamination, and assess domain-specific tasks.
  arXiv  Detail & Related papers  (2024-12-02T20:49:21Z)
• SafeBench: A Safety Evaluation Framework for Multimodal Large Language Models [75.67623347512368]
  We propose toolns, a comprehensive framework designed for conducting safety evaluations of MLLMs. Our framework consists of a comprehensive harmful query dataset and an automated evaluation protocol. Based on our framework, we conducted large-scale experiments on 15 widely-used open-source MLLMs and 6 commercial MLLMs.
  arXiv  Detail & Related papers  (2024-10-24T17:14:40Z)
• SeCodePLT: A Unified Platform for Evaluating the Security of Code GenAI [58.29510889419971]
  Existing benchmarks for evaluating the security risks and capabilities of code-generating large language models (LLMs) face several key limitations.<n>We introduce a general and scalable benchmark construction framework that begins with manually validated, high-quality seed examples and expands them via targeted mutations.<n>Applying this framework to Python, C/C++, and Java, we build SeCodePLT, a dataset of more than 5.9k samples spanning 44 CWE-based risk categories and three security capabilities.
  arXiv  Detail & Related papers  (2024-10-14T21:17:22Z)
• SAFETY-J: Evaluating Safety with Critique [24.723999605458832]
  We introduce SAFETY-J, a bilingual generative safety evaluator for English and Chinese with critique-based judgment. We establish an automated meta-evaluation benchmark that objectively assesses the quality of critiques with minimal human intervention. Our evaluations demonstrate that SAFETY-J provides more nuanced and accurate safety evaluations, thereby enhancing both critique quality and predictive reliability in complex content scenarios.
  arXiv  Detail & Related papers  (2024-07-24T08:04:00Z)
• SORRY-Bench: Systematically Evaluating Large Language Model Safety Refusal Behaviors [64.9938658716425]
  Existing evaluations of large language models' (LLMs) ability to recognize and reject unsafe user requests face three limitations. First, existing methods often use coarse-grained of unsafe topics, and are over-representing some fine-grained topics. Second, linguistic characteristics and formatting of prompts are often overlooked, like different languages, dialects, and more -- which are only implicitly considered in many evaluations. Third, existing evaluations rely on large LLMs for evaluation, which can be expensive.
  arXiv  Detail & Related papers  (2024-06-20T17:56:07Z)
• ALERT: A Comprehensive Benchmark for Assessing Large Language Models' Safety through Red Teaming [64.86326523181553]
  ALERT is a large-scale benchmark to assess safety based on a novel fine-grained risk taxonomy. It aims to identify vulnerabilities, inform improvements, and enhance the overall safety of the language models.
  arXiv  Detail & Related papers  (2024-04-06T15:01:47Z)
• Flames: Benchmarking Value Alignment of LLMs in Chinese [86.73527292670308]
  This paper proposes a value alignment benchmark named Flames. It encompasses both common harmlessness principles and a unique morality dimension that integrates specific Chinese values. Our findings indicate that all the evaluated LLMs demonstrate relatively poor performance on Flames.
  arXiv  Detail & Related papers  (2023-11-12T17:18:21Z)
• Safety Assessment of Chinese Large Language Models [51.83369778259149]
  Large language models (LLMs) may generate insulting and discriminatory content, reflect incorrect social values, and may be used for malicious purposes. To promote the deployment of safe, responsible, and ethical AI, we release SafetyPrompts including 100k augmented prompts and responses by LLMs.
  arXiv  Detail & Related papers  (2023-04-20T16:27:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.