PrivAudit: A Dual-Lens Auditing Framework for Website Privacy Practices under the CCPA

• URL: http://arxiv.org/abs/2609.09697v1
• Date: Wed, 09 Sep 2026 04:31:30 GMT
• Title: PrivAudit: A Dual-Lens Auditing Framework for Website Privacy Practices under the CCPA
• Abstract summary: Five years after the enforcement of the California Consumer Privacy Act (CCPA), understanding how website privacy practices evolve at scale in response to regulation remains a key challenge for researchers and regulators.<n>We present PrivAudit, an automated auditing framework that adopts a duallens approach to capture: (1) privacy disclosures through large language model-based analysis of privacy policies grounded in CCPA provisions, and (2) user-observable data collection behavior through automated browser measurements of cookie writes under diverse privacy configurations.
• Score: 1.8964171459664039
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Five years after the enforcement of the California Consumer Privacy Act (CCPA), understanding how website privacy practices evolve at scale in response to regulation remains a key challenge for both researchers and regulators. Prior work and regulatory efforts have focused on manual and case-specific enforcement, but there remain no scalable approaches to systematically audit two key user-facing facets of websites that are crucial signals for the CCPA: privacy disclosures and front-end user tracking behavior. In this paper, we present PrivAudit, an automated auditing framework that adopts a dual-lens approach to capture: (1) privacy disclosures through large language model-based analysis of privacy policies grounded in CCPA provisions, and (2) user-observable data collection behavior through automated browser measurements of cookie writes under diverse privacy configurations. We apply PrivAudit to 998 websites and report two broad findings. The law is associated with stronger privacy disclosures: CCPA-subject policies are more likely to disclose opt-out mechanisms, data-sharing practices, and user rights. On the other hand, cookie-based tracking remains pervasive, with both CCPA-subject and not-subject websites setting a total of 6,392 targeting cookies, 49% of which are third-party writes. Moreover, cookies show limited-to-moderate responsiveness to privacy signals and consent choices, even when websites claim to honor them in their disclosures. Our results highlight the need for multi-layered and scalable auditing approaches that combine policy analysis with behavioral evidence. PrivAudit can support these auditing workflows at scale by generating actionable signals and patterns for further manual review. We open-source PrivAudit and are engaging with regulators to support auditing in practice.

Related papers

• MAGPIE: A dataset for Multi-AGent contextual PrIvacy Evaluation [54.410825977390274]
  Existing benchmarks to evaluate contextual privacy in LLM-agents primarily assess single-turn, low-complexity tasks.<n>We first present a benchmark - MAGPIE comprising 158 real-life high-stakes scenarios across 15 domains.<n>We then evaluate the current state-of-the-art LLMs on their understanding of contextually private data and their ability to collaborate without violating user privacy.
  arXiv  Detail & Related papers  (2025-06-25T18:04:25Z)
• PriveShield: Enhancing User Privacy Using Automatic Isolated Profiles in Browsers [3.9251831157293515]
  PriveShield is a light-weight privacy mechanism that disrupts the information gathering cycle.<n>Our evaluation results show that our extension is effective in preventing retargeted ads in 91% of those scenarios.
  arXiv  Detail & Related papers  (2025-01-03T20:29:33Z)
• On the Differential Privacy and Interactivity of Privacy Sandbox Reports [78.85958224681858]
  The Privacy Sandbox initiative from Google includes APIs for enabling privacy-preserving advertising functionalities.<n>We provide an abstract model for analyzing the privacy of these APIs and show that they satisfy a formal DP guarantee.
  arXiv  Detail & Related papers  (2024-12-22T08:22:57Z)
• Dual-Technique Privacy & Security Analysis for E-Commerce Websites Through Automated and Manual Implementation [2.7039386580759666]
  38.5% of the websites deployed over 50 cookies per session, many of which were categorized as unnecessary or unclear in function. Our manual assessment uncovered critical gaps in standard security practices, including the absence of mandatory multi-factor authentication and breach notification protocols. Based on these findings, we recommend targeted improvements to privacy policies, enhanced transparency in cookie usage, and the implementation of stronger authentication protocols.
  arXiv  Detail & Related papers  (2024-10-19T03:25:48Z)
• C3PA: An Open Dataset of Expert-Annotated and Regulation-Aware Privacy Policies to Enable Scalable Regulatory Compliance Audits [7.1195014414194695]
  C3PA is the first regulation-aware dataset of expert-annotated privacy policies. It contains over 48K expert-labeled privacy policy text segments associated with responses to CCPA-specific disclosure mandates.
  arXiv  Detail & Related papers  (2024-10-04T21:04:39Z)
• PrivacyLens: Evaluating Privacy Norm Awareness of Language Models in Action [54.11479432110771]
  PrivacyLens is a novel framework designed to extend privacy-sensitive seeds into expressive vignettes and further into agent trajectories.<n>We instantiate PrivacyLens with a collection of privacy norms grounded in privacy literature and crowdsourced seeds.<n>State-of-the-art LMs, like GPT-4 and Llama-3-70B, leak sensitive information in 25.68% and 38.69% of cases, even when prompted with privacy-enhancing instructions.
  arXiv  Detail & Related papers  (2024-08-29T17:58:38Z)
• Mind the Privacy Unit! User-Level Differential Privacy for Language Model Fine-Tuning [62.224804688233]
  differential privacy (DP) offers a promising solution by ensuring models are 'almost indistinguishable' with or without any particular privacy unit. We study user-level DP motivated by applications where it necessary to ensure uniform privacy protection across users.
  arXiv  Detail & Related papers  (2024-06-20T13:54:32Z)
• Evaluating Google's Protected Audience Protocol [7.737740676767729]
  Google has proposed the Privacy Sandbox initiative to enable ad targeting without third-party cookies. This work focuses on analyzing linkage privacy risks for the reporting mechanisms proposed in the Protected Audience proposal.
  arXiv  Detail & Related papers  (2024-05-13T18:28:56Z)
• Tight Auditing of Differentially Private Machine Learning [77.38590306275877]
  For private machine learning, existing auditing mechanisms are tight. They only give tight estimates under implausible worst-case assumptions. We design an improved auditing scheme that yields tight privacy estimates for natural (not adversarially crafted) datasets.
  arXiv  Detail & Related papers  (2023-02-15T21:40:33Z)
• Having your Privacy Cake and Eating it Too: Platform-supported Auditing of Social Media Algorithms for Public Interest [70.02478301291264]
  Social media platforms curate access to information and opportunities, and so play a critical role in shaping public discourse. Prior studies have used black-box methods to show that these algorithms can lead to biased or discriminatory outcomes. We propose a new method for platform-supported auditing that can meet the goals of the proposed legislation.
  arXiv  Detail & Related papers  (2022-07-18T17:32:35Z)
• Fighting the Fog: Evaluating the Clarity of Privacy Disclosures in the Age of CCPA [29.56312492076473]
  Vagueness and ambiguity in privacy policies threaten the ability of consumers to make informed choices about how businesses collect, use, and share personal information. The California Consumer Privacy Act (CCPA) of 2018 was intended to provide Californian consumers with more control by mandating that businesses clearly disclose their data practices. Our results suggest that CCPA's mandates for privacy disclosures, as currently implemented, have not yet yielded the level of clarity they were designed to deliver.
  arXiv  Detail & Related papers  (2021-09-28T15:40:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.