Auditing Data Provenance in LLM Fine-tuning via Intrinsic Distributional Fingerprints

• URL: http://arxiv.org/abs/2608.02154v1
• Date: Mon, 03 Aug 2026 12:35:51 GMT
• Title: Auditing Data Provenance in LLM Fine-tuning via Intrinsic Distributional Fingerprints
• Abstract summary: We propose textitDistribution Provenance Audit (DPA), a post-hoc framework for auditing data IP infringement in Large Language Models (LLMs)<n>DPA is grounded in a critical insight: regardless of fine-tuning tactics to evade provenance, the practical necessity of maintaining utility constrains the model to preserve the fundamental intersection of semantic substance and lexical form.
• Score: 12.090394120714402
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The proliferation of customized Large Language Models (LLMs) poses critical risks of Data Intellectual Property (Data IP) infringement via unauthorized fine-tuning on proprietary data. Existing audit techniques are limited, as they require intervention during data preparation or training and remain fragile under malicious obfuscations such as data paraphrasing and knowledge distillation. We propose \textit{Distribution Provenance Audit (DPA)}, a post-hoc framework for auditing data IP infringement in LLM fine-tuning under black-box and malicious settings. DPA is grounded in a critical insight: regardless of fine-tuning tactics to evade provenance, the practical necessity of maintaining utility constrains the model to preserve the fundamental intersection of semantic substance and lexical form. Accordingly, DPA captures this persistent lexical-semantic intersection as intrinsic distributional fingerprints. The framework formulates the audit as a statistical hypothesis test, effectively quantifying these fingerprints via unbiased output sampling to reliably reject the null hypothesis of non-usage. Extensive experiments on medical and legal fine-tuning tasks show that DPA consistently outperforms existing baselines, remaining robust against adversarial trainers employing paraphrasing and knowledge distillation. We further highlight a fundamental dual-use tension: the same high-fidelity distributional fingerprints enabling reliable auditing may also facilitate privacy attacks.

Related papers

• Phantoms and Disclosures: a Causal Framework for Auditing Synthetic Data [56.86147283213652]
  We present a customizable empirical auditing framework designed to detect and explain data disclosures.<n>Our framework introduces a mechanism to distinguish between "true disclosures"-where the system directly reproduces a user's information-and "phantom disclosures"-where the system incidentally generates a user's data.
  arXiv  Detail & Related papers  (2026-06-15T16:54:02Z)
• Alignment Imprint: Zero-Shot AI-Generated Text Detection via Provable Preference Discrepancy [51.887915969023965]
  We show that modern Large Language Models (LLMs) undergo alignment, leaving a measurable distributional imprint.<n>To mitigate the instability in high-entropy regions, we introduce Log-likelihood Alignment Preference Discrepancy (LAPD)<n>LAPD is a standardized information-weighted statistic based on alignment imprint.
  arXiv  Detail & Related papers  (2026-04-18T09:12:24Z)
• Gaming the Judge: Unfaithful Chain-of-Thought Can Undermine Agent Evaluation [76.5533899503582]
  Large language models (LLMs) are increasingly used as judges to evaluate agent performance.<n>We show this paradigm implicitly assumes that the agent's chain-of-thought (CoT) reasoning faithfully reflects both its internal reasoning and the underlying environment state.<n>We demonstrate that manipulated reasoning alone can inflate false positive rates of state-of-the-art VLM judges by up to 90% across 800 trajectories spanning diverse web tasks.
  arXiv  Detail & Related papers  (2026-01-21T06:07:43Z)
• SWAP: Towards Copyright Auditing of Soft Prompts via Sequential Watermarking [58.475471437150674]
  We propose sequential watermarking for soft prompts (SWAP)<n>SWAP encodes watermarks through a specific order of defender-specified out-of-distribution classes.<n>Experiments on 11 datasets demonstrate SWAP's effectiveness, harmlessness, and robustness against potential adaptive attacks.
  arXiv  Detail & Related papers  (2025-11-05T13:48:48Z)
• Copyright Infringement Detection in Text-to-Image Diffusion Models via Differential Privacy [23.262369771803364]
  We formalize the concept of copyright infringement and its detection from the perspective of Differential Privacy (DP)<n>We propose D-Plus-Minus (DPM), a novel post-hoc detection framework that identifies copyright infringement in text-to-image diffusion models.<n>Our results demonstrate that DPM reliably detects infringement content without requiring access to the original training dataset or text prompts.
  arXiv  Detail & Related papers  (2025-09-27T00:38:12Z)
• CertDW: Towards Certified Dataset Ownership Verification via Conformal Prediction [48.82467166657901]
  We propose the first certified dataset watermark (i.e., CertDW) and CertDW-based certified dataset ownership verification method.<n>Inspired by conformal prediction, we introduce two statistical measures, including principal probability (PP) and watermark robustness (WR)<n>We prove there exists a provable lower bound between PP and WR, enabling ownership verification when a suspicious model's WR value significantly exceeds the PP values of benign models trained on watermark-free datasets.
  arXiv  Detail & Related papers  (2025-06-16T07:17:23Z)
• Hide in Plain Sight: Clean-Label Backdoor for Auditing Membership Inference [16.893873979953593]
  We propose a novel clean-label backdoor-based approach for stealthy data auditing. Our approach employs an optimal trigger generated by a shadow model that mimics target model's behavior. The proposed method enables robust data auditing through blackbox access, achieving high attack success rates across diverse datasets.
  arXiv  Detail & Related papers  (2024-11-24T20:56:18Z)
• Certifiably Byzantine-Robust Federated Conformal Prediction [49.23374238798428]
  We introduce a novel framework Rob-FCP, which executes robust federated conformal prediction effectively countering malicious clients. We empirically demonstrate the robustness of Rob-FCP against diverse proportions of malicious clients under a variety of Byzantine attacks.
  arXiv  Detail & Related papers  (2024-06-04T04:43:30Z)
• On the Robustness of Dataset Inference [21.321310557323383]
  Machine learning (ML) models are costly to train as they can require a significant amount of data, computational resources and technical expertise. Ownership verification techniques allow the victims of model stealing attacks to demonstrate that a suspect model was in fact stolen from theirs. A fingerprinting technique, dataset inference (DI), has been shown to offer better robustness and efficiency than prior methods.
  arXiv  Detail & Related papers  (2022-10-24T22:17:55Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.