Extracting Training Data from Document-Based VQA Models

• URL: http://arxiv.org/abs/2407.08707v1
• Date: Thu, 11 Jul 2024 17:44:41 GMT
• Title: Extracting Training Data from Document-Based VQA Models
• Authors: Francesco Pinto, Nathalie Rauschmayr, Florian Tramèr, Philip Torr, Federico Tombari,
• Abstract summary: Vision-Language Models (VLMs) have made remarkable progress in document-based Visual Question Answering (i.e., responding to queries about the contents of an input document provided as an image) We show these models can memorise responses for training samples and regurgitate them even when the relevant visual information has been removed. This includes Personal Identifiable Information repeated once in the training set, indicating these models could divulge sensitive information and therefore pose a privacy risk.
• Score: 67.1470112451617
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Vision-Language Models (VLMs) have made remarkable progress in document-based Visual Question Answering (i.e., responding to queries about the contents of an input document provided as an image). In this work, we show these models can memorize responses for training samples and regurgitate them even when the relevant visual information has been removed. This includes Personal Identifiable Information (PII) repeated once in the training set, indicating these models could divulge memorised sensitive information and therefore pose a privacy risk. We quantitatively measure the extractability of information in controlled experiments and differentiate between cases where it arises from generalization capabilities or from memorization. We further investigate the factors that influence memorization across multiple state-of-the-art models and propose an effective heuristic countermeasure that empirically prevents the extractability of PII.

Related papers

• Extracting Training Data from Unconditional Diffusion Models [76.85077961718875]
  diffusion probabilistic models (DPMs) are being employed as mainstream models for generative artificial intelligence (AI) We aim to establish a theoretical understanding of memorization in DPMs with 1) a memorization metric for theoretical analysis, 2) an analysis of conditional memorization with informative and random labels, and 3) two better evaluation metrics for measuring memorization. Based on the theoretical analysis, we propose a novel data extraction method called textbfSurrogate condItional Data Extraction (SIDE) that leverages a trained on generated data as a surrogate condition to extract training data directly from unconditional diffusion models.
  arXiv  Detail & Related papers  (2024-06-18T16:20:12Z)
• Enhancing Large Vision Language Models with Self-Training on Image Comprehension [131.14381425260706]
  We introduce Self-Training on Image (STIC), which emphasizes a self-training approach specifically for image comprehension. First, the model self-constructs a preference for image descriptions using unlabeled images. To further self-improve reasoning on the extracted visual information, we let the model reuse a small portion of existing instruction-tuning data.
  arXiv  Detail & Related papers  (2024-05-30T05:53:49Z)
• Deep Variational Privacy Funnel: General Modeling with Applications in Face Recognition [3.351714665243138]
  We develop a method for privacy-preserving representation learning using an end-to-end training framework. We apply our model to state-of-the-art face recognition systems.
  arXiv  Detail & Related papers  (2024-01-26T11:32:53Z)
• Revisiting Self-supervised Learning of Speech Representation from a Mutual Information Perspective [68.20531518525273]
  We take a closer look into existing self-supervised methods of speech from an information-theoretic perspective. We use linear probes to estimate the mutual information between the target information and learned representations. We explore the potential of evaluating representations in a self-supervised fashion, where we estimate the mutual information between different parts of the data without using any labels.
  arXiv  Detail & Related papers  (2024-01-16T21:13:22Z)
• SoK: Memorisation in machine learning [5.563171090433323]
  Quantifying the impact of individual data samples on machine learning models is an open research problem. In this work we unify a broad range of previous definitions and perspectives on memorisation in ML. We discuss their interplay with model generalisation and their implications of these phenomena on data privacy.
  arXiv  Detail & Related papers  (2023-11-06T12:59:18Z)
• Assessing Privacy Risks in Language Models: A Case Study on Summarization Tasks [65.21536453075275]
  We focus on the summarization task and investigate the membership inference (MI) attack. We exploit text similarity and the model's resistance to document modifications as potential MI signals. We discuss several safeguards for training summarization models to protect against MI attacks and discuss the inherent trade-off between privacy and utility.
  arXiv  Detail & Related papers  (2023-10-20T05:44:39Z)
• Can Pre-trained Vision and Language Models Answer Visual Information-Seeking Questions? [50.29862466940209]
  We introduce InfoSeek, a visual question answering dataset tailored for information-seeking questions. We analyze various pre-trained visual question answering models and gain insights into their characteristics. We show that accurate visual entity recognition can be used to improve performance on InfoSeek by retrieving relevant documents.
  arXiv  Detail & Related papers  (2023-02-23T00:33:54Z)
• Leveraging Adversarial Examples to Quantify Membership Information Leakage [30.55736840515317]
  We develop a novel approach to address the problem of membership inference in pattern recognition models. We argue that this quantity reflects the likelihood of belonging to the training data. Our method performs comparable or even outperforms state-of-the-art strategies.
  arXiv  Detail & Related papers  (2022-03-17T19:09:38Z)
• Reducing Overlearning through Disentangled Representations by Suppressing Unknown Tasks [8.517620051440005]
  Existing deep learning approaches for learning visual features tend to overlearn and extract more information than what is required for the task at hand. From a privacy preservation perspective, the input visual information is not protected from the model. We propose a model-agnostic solution for reducing model overlearning by suppressing all the unknown tasks.
  arXiv  Detail & Related papers  (2020-05-20T17:31:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.