SPA-VL: A Comprehensive Safety Preference Alignment Dataset for Vision Language Model

• URL: http://arxiv.org/abs/2406.12030v1
• Date: Mon, 17 Jun 2024 18:57:37 GMT
• Title: SPA-VL: A Comprehensive Safety Preference Alignment Dataset for Vision Language Model
• Authors: Yongting Zhang, Lu Chen, Guodong Zheng, Yifeng Gao, Rui Zheng, Jinlan Fu, Zhenfei Yin, Senjie Jin, Yu Qiao, Xuanjing Huang, Feng Zhao, Tao Gui, Jing Shao,
• Abstract summary: We propose a Safety Preference Alignment dataset for Vision Language Models named SPA-VL. In terms of breadth, SPA-VL covers 6 harmfulness domains, 13 categories, and 53 subcategories, and contains 100,788 samples of the quadruple (question, image, chosen response, rejected response) The experimental results indicate that models trained with alignment techniques on the SPA-VL dataset exhibit substantial improvements in harmlessness and helpfulness while maintaining core capabilities.
• Score: 77.86593720792986
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The emergence of Vision Language Models (VLMs) has brought unprecedented advances in understanding multimodal information. The combination of textual and visual semantics in VLMs is highly complex and diverse, making the safety alignment of these models challenging. Furthermore, due to the limited study on the safety alignment of VLMs, there is a lack of large-scale, high-quality datasets. To address these limitations, we propose a Safety Preference Alignment dataset for Vision Language Models named SPA-VL. In terms of breadth, SPA-VL covers 6 harmfulness domains, 13 categories, and 53 subcategories, and contains 100,788 samples of the quadruple (question, image, chosen response, rejected response). In terms of depth, the responses are collected from 12 open- (e.g., QwenVL) and closed-source (e.g., Gemini) VLMs to ensure diversity. The experimental results indicate that models trained with alignment techniques on the SPA-VL dataset exhibit substantial improvements in harmlessness and helpfulness while maintaining core capabilities. SPA-VL, as a large-scale, high-quality, and diverse dataset, represents a significant milestone in ensuring that VLMs achieve both harmlessness and helpfulness. We have made our code https://github.com/EchoseChen/SPA-VL-RLHF and SPA-VL dataset url https://huggingface.co/datasets/sqrti/SPA-VL publicly available.

Related papers

• Scalable Vision Language Model Training via High Quality Data Curation [10.121967684111445]
  We introduce an open-source vision language model (VLM) series achieving state-of-the-art (SOTA) performance in 2B and 8B parameters. The following three key improvements contribute to SAILVL's leading performance.
  arXiv  Detail & Related papers  (2025-01-10T13:27:04Z)
• VLsI: Verbalized Layers-to-Interactions from Large to Small Vision Language Models [63.27511432647797]
  We propose VLsI: Verbalized Layers-to-Interactions, a new VLM family in 2B and 7B model sizes. We validate VLsI across ten challenging vision-language benchmarks, achieving notable performance gains (11.0% for 2B and 17.4% for 7B) over GPT-4V.
  arXiv  Detail & Related papers  (2024-12-02T18:58:25Z)
• FedMLLM: Federated Fine-tuning MLLM on Multimodal Heterogeneity Data [64.50893177169996]
  Fine-tuning Multimodal Large Language Models (MLLMs) with Federated Learning (FL) allows for expanding the training data scope by including private data sources. We introduce a benchmark for evaluating various downstream tasks in the federated fine-tuning of MLLMs within multimodal heterogeneous scenarios. We develop a general FedMLLM framework that integrates four representative FL methods alongside two modality-agnostic strategies.
  arXiv  Detail & Related papers  (2024-11-22T04:09:23Z)
• Membership Inference Attacks against Large Vision-Language Models [40.996912464828696]
  Large vision-language models (VLLMs) exhibit promising capabilities for processing multi-modal tasks across various application scenarios. Their emergence also raises significant data security concerns, given the potential inclusion of sensitive information, such as private photos and medical records. Detecting inappropriately used data in VLLMs remains a critical and unresolved issue.
  arXiv  Detail & Related papers  (2024-11-05T08:35:08Z)
• NVLM: Open Frontier-Class Multimodal LLMs [64.00053046838225]
  We introduce NVLM 1.0, a family of frontier-class multimodal large language models (LLMs) that achieve state-of-the-art results on vision-language tasks. We propose a novel architecture that enhances both training efficiency and multimodal reasoning capabilities. We develop production-grade multimodality for the NVLM-1.0 models, enabling them to excel in vision-language tasks.
  arXiv  Detail & Related papers  (2024-09-17T17:59:06Z)
• Concept-skill Transferability-based Data Selection for Large Vision-Language Models [56.0725292404808]
  We introduce COINCIDE, an effective and scalable data selection technique for training vision-language models. We cluster the training data using internal activations from a small model, which identifies concept-skill compositions needed by a target LVLM. Experiments demonstrate that COINCIDE achieves superior performance and data selection efficiency against 8 strong baselines.
  arXiv  Detail & Related papers  (2024-06-16T16:15:20Z)
• B-AVIBench: Towards Evaluating the Robustness of Large Vision-Language Model on Black-box Adversarial Visual-Instructions [73.97665608366447]
  Large Vision-Language Models (LVLMs) have shown significant progress in responding well to visual-instructions from users. These instructions, encompassing images and text, are susceptible to both intentional and inadvertent attacks. We introduce B-AVIBench, a framework designed to analyze the robustness of LVLMs when facing various Black-box Adrial Visual-Instructions.
  arXiv  Detail & Related papers  (2024-03-14T12:51:07Z)
• Safety Fine-Tuning at (Almost) No Cost: A Baseline for Vision Large Language Models [39.56233272612982]
  Current vision large language models (VLLMs) exhibit remarkable capabilities yet are prone to generate harmful content and are vulnerable to jailbreaking attacks. Our initial analysis finds that this is due to the presence of harmful data during vision-language instruction fine-tuning. To address this issue, we first curate a vision-language safe instruction-following dataset VLGuard covering various harmful categories.
  arXiv  Detail & Related papers  (2024-02-03T16:43:42Z)
• Teaching Structured Vision&Language Concepts to Vision&Language Models [46.344585368641006]
  We introduce the collective notion of Structured Vision&Language Concepts (SVLC) SVLC includes object attributes, relations, and states which are present in the text and visible in the image. We propose a more elegant data-driven approach for enhancing VL models' understanding of SVLCs.
  arXiv  Detail & Related papers  (2022-11-21T18:54:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.