FP-VEC: Fingerprinting Large Language Models via Efficient Vector Addition

• URL: http://arxiv.org/abs/2409.08846v1
• Date: Fri, 13 Sep 2024 14:04:39 GMT
• Title: FP-VEC: Fingerprinting Large Language Models via Efficient Vector Addition
• Authors: Zhenhua Xu, Wenpeng Xing, Zhebo Wang, Chang Hu, Chen Jie, Meng Han,
• Abstract summary: We introduce FP-VEC, a pilot study on using fingerprint vectors as an efficient fingerprinting method for Large Language Models. Our approach generates a fingerprint vector that represents a confidential signature embedded in the model, allowing the same fingerprint to be seamlessly incorporated into an unlimited number of LLMs. Results on several LLMs show that FP-VEC is lightweight by running on CPU-only devices for fingerprinting, scalable with a single training and unlimited fingerprinting process, and preserves the model's normal behavior.
• Score: 11.885529039351217
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Training Large Language Models (LLMs) requires immense computational power and vast amounts of data. As a result, protecting the intellectual property of these models through fingerprinting is essential for ownership authentication. While adding fingerprints to LLMs through fine-tuning has been attempted, it remains costly and unscalable. In this paper, we introduce FP-VEC, a pilot study on using fingerprint vectors as an efficient fingerprinting method for LLMs. Our approach generates a fingerprint vector that represents a confidential signature embedded in the model, allowing the same fingerprint to be seamlessly incorporated into an unlimited number of LLMs via vector addition. Results on several LLMs show that FP-VEC is lightweight by running on CPU-only devices for fingerprinting, scalable with a single training and unlimited fingerprinting process, and preserves the model's normal behavior. The project page is available at https://fingerprintvector.github.io .

Related papers

• UTF:Undertrained Tokens as Fingerprints A Novel Approach to LLM Identification [23.164580168870682]
  Fingerprinting large language models (LLMs) is essential for verifying model ownership, ensuring authenticity, and preventing misuse. In this paper, we introduce a novel and efficient approach to fingerprinting LLMs by leveraging under-trained tokens. Our method has minimal overhead and impact on model's performance, and does not require white-box access to target model's ownership identification.
  arXiv  Detail & Related papers  (2024-10-16T07:36:57Z)
• Instructional Fingerprinting of Large Language Models [57.72356846657551]
  We present a pilot study on fingerprinting Large language models (LLMs) as a form of very lightweight instruction tuning. Results on 11 popularly-used LLMs showed that this approach is lightweight and does not affect the normal behavior of the model. It also prevents publisher overclaim, maintains robustness against fingerprint guessing and parameter-efficient training, and supports multi-stage fingerprinting akin to MIT License.
  arXiv  Detail & Related papers  (2024-01-21T09:51:45Z)
• HuRef: HUman-REadable Fingerprint for Large Language Models [44.9820558213721]
  HuRef is a human-readable fingerprint for large language models. It uniquely identifies the base model without interfering with training or exposing model parameters to the public.
  arXiv  Detail & Related papers  (2023-12-08T05:01:47Z)
• Robust Retraining-free GAN Fingerprinting via Personalized Normalization [21.63902009635896]
  The proposed method can embed different fingerprints inside the GAN by just changing the input of the ParamGen Nets. The performance of the proposed method in terms of robustness against both model-level and image-level attacks is superior to the state-of-the-art.
  arXiv  Detail & Related papers  (2023-11-09T16:09:12Z)
• Language models are weak learners [71.33837923104808]
  We show that prompt-based large language models can operate effectively as weak learners. We incorporate these models into a boosting approach, which can leverage the knowledge within the model to outperform traditional tree-based boosting. Results illustrate the potential for prompt-based LLMs to function not just as few-shot learners themselves, but as components of larger machine learning pipelines.
  arXiv  Detail & Related papers  (2023-06-25T02:39:19Z)
• PrintsGAN: Synthetic Fingerprint Generator [39.804969475699345]
  PrintsGAN is a synthetic fingerprint generator capable of generating unique fingerprints along with multiple impressions for a given fingerprint. We show the utility of the PrintsGAN generated by training a deep network to extract a fixed-length embedding from a fingerprint.
  arXiv  Detail & Related papers  (2022-01-10T22:25:10Z)
• SignBERT: Pre-Training of Hand-Model-Aware Representation for Sign Language Recognition [94.30084702921529]
  Hand gesture serves as a critical role in sign language. Current deep-learning-based sign language recognition methods may suffer insufficient interpretability. We introduce the first self-supervised pre-trainable SignBERT with incorporated hand prior for SLR.
  arXiv  Detail & Related papers  (2021-10-11T16:18:09Z)
• Responsible Disclosure of Generative Models Using Scalable Fingerprinting [70.81987741132451]
  Deep generative models have achieved a qualitatively new level of performance. There are concerns on how this technology can be misused to spoof sensors, generate deep fakes, and enable misinformation at scale. Our work enables a responsible disclosure of such state-of-the-art generative models, that allows researchers and companies to fingerprint their models.
  arXiv  Detail & Related papers  (2020-12-16T03:51:54Z)
• Latent Fingerprint Registration via Matching Densely Sampled Points [100.53031290339483]
  Existing latent fingerprint registration approaches are mainly based on establishing correspondences between minutiae. We propose a non-minutia latent fingerprint registration method which estimates the spatial transformation between a pair of fingerprints. The proposed method achieves the state-of-the-art registration performance, especially under challenging conditions.
  arXiv  Detail & Related papers  (2020-05-12T15:51:59Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.