Big Help or Big Brother? Auditing Tracking, Profiling, and Personalization in Generative AI Assistants

• URL: http://arxiv.org/abs/2503.16586v2
• Date: Tue, 10 Jun 2025 12:45:18 GMT
• Title: Big Help or Big Brother? Auditing Tracking, Profiling, and Personalization in Generative AI Assistants
• Abstract summary: Generative AI (GenAI) browser assistants integrate powerful capabilities of GenAI in web browsers to provide rich experiences such as question answering, content summarization, and agentic navigation.<n>These assistants, available today as browser extensions, can not only track detailed browsing activity such as search and click data, but can also autonomously perform tasks such as filling forms, raising significant privacy concerns.<n>We study their ability to profile users and personalize their responses based on explicit or inferred demographic attributes and interests of users.
• Score: 10.38225151849918
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Generative AI (GenAI) browser assistants integrate powerful capabilities of GenAI in web browsers to provide rich experiences such as question answering, content summarization, and agentic navigation. These assistants, available today as browser extensions, can not only track detailed browsing activity such as search and click data, but can also autonomously perform tasks such as filling forms, raising significant privacy concerns. It is crucial to understand the design and operation of GenAI browser extensions, including how they collect, store, process, and share user data. To this end, we study their ability to profile users and personalize their responses based on explicit or inferred demographic attributes and interests of users. We perform network traffic analysis and use a novel prompting framework to audit tracking, profiling, and personalization by the ten most popular GenAI browser assistant extensions. We find that instead of relying on local in-browser models, these assistants largely depend on server-side APIs, which can be auto-invoked without explicit user interaction. When invoked, they collect and share webpage content, often the full HTML DOM and sometimes even the user's form inputs, with their first-party servers. Some assistants also share identifiers and user prompts with third-party trackers such as Google Analytics. The collection and sharing continues even if a webpage contains sensitive information such as health or personal information such as name or SSN entered in a web form. We find that several GenAI browser assistants infer demographic attributes such as age, gender, income, and interests and use this profile--which carries across browsing contexts--to personalize responses. In summary, our work shows that GenAI browser assistants can and do collect personal and sensitive information for profiling and personalization with little to no safeguards.

Related papers

• Quick Build, Careful Check? Generative AI Use in Hackathons [83.0493885475068]
  We report an exploratory interview study with participants at an AI-themed hackathon in central Europe.<n>We found that participants used GenAI for purposes beyond coding, including learning unfamiliar topics, brainstorming, and preparing documentation.<n>We also found that all participants converged on an unwritten practice of checking GenAI output before using it, yet their ability to actually verify was constrained by time pressure and limited domain knowledge.
  arXiv  Detail & Related papers  (2026-07-31T08:59:18Z)
• Do Generative AI Assistants Respect robots.txt? Tracing Web Access Beyond Visible Answers [0.3499870393443268]
  AI assistants increasingly retrieve web content at inference time to provide fresh and grounded answers.<n>It remains unclear whether these search-augmented capabilities respect website-owner restrictions expressed through robots$.$txt.<n>We present a controlled empirical study of ten widely used AI assistants with advertised web-search capabilities.
  arXiv  Detail & Related papers  (2026-07-16T00:45:06Z)
• Developer Experience with AI Coding Agents: HTTP Behavioral Signatures in Documentation Portals [0.0]
  The rapid adoption of AI coding agents and AI assistant web services is fundamentally changing how developers discover, consume, and interact with technical documentation.<n>This paper studies that transformation across three interconnected dimensions: documentation accessibility, content analytics, and feedback systems.<n>We present an empirical study of HTTP request fingerprints from nine AI coding agents (Aider, Antigravity, Claude Code, Cline, Cursor, Junie, OpenCode, VS Code, and Windsurf) and six AI assistant services (ChatGPT, Claude, Google Gemini, Google NotebookLM, MistralAI, and Perplexity) accessing a live developer documentation endpoint.
  arXiv  Detail & Related papers  (2026-04-02T21:54:07Z)
• In-Browser Agents for Search Assistance [2.50369129460887]
  A tension exists between the demand for AI assistance in web search and the need for user data privacy.<n>We present a browser extension that provides a viable in-browser alternative.<n>Our results show that this privacy-preserving approach is highly effective at adapting to individual user behavior.
  arXiv  Detail & Related papers  (2026-01-14T23:18:54Z)
• Nested Browser-Use Learning for Agentic Information Seeking [60.775556172513014]
  Information-seeking (IS) agents have achieved strong performance across a range of wide and deep search tasks, yet their tool use remains largely restricted to API-level snippet retrieval and URL-based page fetching.<n>We propose Nested Browser-Use Learning (NestBrowse), which introduces a minimal and complete browser-action framework that decouples interaction control from page exploration through a nested structure.
  arXiv  Detail & Related papers  (2025-12-29T17:59:14Z)
• Creating General User Models from Computer Use [62.91116265732001]
  This paper presents an architecture for a general user model (GUM) that learns about you by observing any interaction you have with your computer.<n>The GUM takes as input any unstructured observation of a user (e.g., device screenshots) and constructs confidence-weighted propositions that capture user knowledge and preferences.
  arXiv  Detail & Related papers  (2025-05-16T04:00:31Z)
• AgentDAM: Privacy Leakage Evaluation for Autonomous Web Agents [75.85554113398626]
  We develop a benchmark called AgentDAM to evaluate how well existing and future AI agents can limit processing of potentially private information. Our benchmark simulates realistic web interaction scenarios and is adaptable to all existing web navigation agents.
  arXiv  Detail & Related papers  (2025-03-12T19:30:31Z)
• AIDetection: A Generative AI Detection Tool for Educators Using Syntactic Matching of Common ASCII Characters As Potential 'AI Traces' Within Users' Internet Browser [0.0]
  AIDetection.info employs a syntactic-based approach to identify common traces left by generative AI models. The tool scans documents in bulk for potential AI artifacts, as well as AI citations and acknowledgments, and provides a visual summary with downloadable Excel and CSV reports.
  arXiv  Detail & Related papers  (2025-03-12T15:53:58Z)
• PersonaBench: Evaluating AI Models on Understanding Personal Information through Accessing (Synthetic) Private User Data [76.21047984886273]
  Personalization is critical in AI assistants, particularly in the context of private AI models that work with individual users.<n>Due to the sensitive nature of such data, there are no publicly available datasets that allow us to assess an AI model's ability to understand users.<n>We introduce a synthetic data generation pipeline that creates diverse, realistic user profiles and private documents simulating human activities.
  arXiv  Detail & Related papers  (2025-02-28T00:43:35Z)
• GOD model: Privacy Preserved AI School for Personal Assistant [3.3015224434662396]
  We introduce the Guardian of Data (GOD), a secure, privacy-preserving framework for training and evaluating AI assistants on-device.<n>GOD measures how well assistants can anticipate user needs-such as suggesting gifts-while protecting user data and autonomy.
  arXiv  Detail & Related papers  (2025-02-24T20:30:17Z)
• Beyond Browsing: API-Based Web Agents [58.39129004543844]
  API-based agents outperform web browsing agents in experiments on WebArena. Hybrid Agents out-perform both others nearly uniformly across tasks. Results strongly suggest that when APIs are available, they present an attractive alternative to relying on web browsing alone.
  arXiv  Detail & Related papers  (2024-10-21T19:46:06Z)
• How Unique is Whose Web Browser? The role of demographics in browser fingerprinting among US users [50.699390248359265]
  Browser fingerprinting can be used to identify and track users across the Web, even without cookies. This technique and resulting privacy risks have been studied for over a decade. We provide a first-of-its-kind dataset to enable further research.
  arXiv  Detail & Related papers  (2024-10-09T14:51:58Z)
• GenAIPABench: A Benchmark for Generative AI-based Privacy Assistants [1.2642388972233845]
  This study introduces GenAIPABench, a benchmark for evaluating Generative AI-based Privacy Assistants (GenAIPAs) GenAIPABench includes: 1) A set of questions about privacy policies and data protection regulations, with annotated answers for various organizations and regulations; 2) Metrics to assess the accuracy, relevance, and consistency of responses; and 3) A tool for generating prompts to introduce privacy documents and varied privacy questions to test system robustness. We evaluated three leading genAI systems ChatGPT-4, Bard, and Bing AI using GenAIPABench to gauge their effectiveness as GenAIPAs.
  arXiv  Detail & Related papers  (2023-09-10T21:15:42Z)
• What Do End-Users Really Want? Investigation of Human-Centered XAI for Mobile Health Apps [69.53730499849023]
  We present a user-centered persona concept to evaluate explainable AI (XAI) Results show that users' demographics and personality, as well as the type of explanation, impact explanation preferences. Our insights bring an interactive, human-centered XAI closer to practical application.
  arXiv  Detail & Related papers  (2022-10-07T12:51:27Z)
• Masked LARk: Masked Learning, Aggregation and Reporting worKflow [6.484847460164177]
  Many web advertising data flows involve passive cross-site tracking of users. Most browsers are moving towards removal of 3PC in subsequent browser iterations. We propose a new proposal, called Masked LARk, for aggregation of user engagement measurement and model training.
  arXiv  Detail & Related papers  (2021-10-27T21:59:37Z)
• TIPRDC: Task-Independent Privacy-Respecting Data Crowdsourcing Framework for Deep Learning with Anonymized Intermediate Representations [49.20701800683092]
  We present TIPRDC, a task-independent privacy-respecting data crowdsourcing framework with anonymized intermediate representation. The goal of this framework is to learn a feature extractor that can hide the privacy information from the intermediate representations; while maximally retaining the original information embedded in the raw data for the data collector to accomplish unknown learning tasks.
  arXiv  Detail & Related papers  (2020-05-23T06:21:26Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.