Privacy Aware Experiments without Cookies

• URL: http://arxiv.org/abs/2211.03758v1
• Date: Thu, 3 Nov 2022 23:44:01 GMT
• Title: Privacy Aware Experiments without Cookies
• Abstract summary: Two brands want to jointly test alternate web experiences for their customers with an A/B test. Such collaborative tests are today enabled using textitthird-party cookies, where each brand has information on the identity of visitors to another website. With the imminent elimination of third-party cookies, such A/B tests will become untenable. We propose a two-stage experimental design, where the two brands only need to agree on high-level aggregate parameters of the experiment to test the alternate experiences.
• Score: 13.488622527004082
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Consider two brands that want to jointly test alternate web experiences for their customers with an A/B test. Such collaborative tests are today enabled using \textit{third-party cookies}, where each brand has information on the identity of visitors to another website. With the imminent elimination of third-party cookies, such A/B tests will become untenable. We propose a two-stage experimental design, where the two brands only need to agree on high-level aggregate parameters of the experiment to test the alternate experiences. Our design respects the privacy of customers. We propose an estimater of the Average Treatment Effect (ATE), show that it is unbiased and theoretically compute its variance. Our demonstration describes how a marketer for a brand can design such an experiment and analyze the results. On real and simulated data, we show that the approach provides valid estimate of the ATE with low variance and is robust to the proportion of visitors overlapping across the brands.

Related papers

• Experimentation Accelerator: Interpretable Insights and Creative Recommendations for A/B Testing with Content-Aware ranking [28.316205612655228]
  We present a unified framework to prioritize which variants to test, (ii) explain why winners win, and (iii) surface targeted opportunities for new, higher-potential variants.<n>These components have been built into a real Adobe product, called textitExperimentation Accelerator, to provide AI-based insights and opportunities to scale experimentation for customers.
  arXiv  Detail & Related papers  (2026-02-14T19:22:58Z)
• Addressing Personalized Bias for Unbiased Learning to Rank [56.663619153713434]
  Unbiased learning to rank (ULTR) aims to learn unbiased ranking models from biased user behavior logs.<n>We propose a novel user-aware inverse-propensity-score estimator for learning-to-rank objectives.
  arXiv  Detail & Related papers  (2025-08-28T14:01:31Z)
• Label Inference Attacks against Federated Unlearning [52.102277522089814]
  Federated Unlearning (FU) has emerged as a promising solution to respond to the right to be forgotten of clients.<n>We introduce and analyze a new privacy threat against FU and propose a novel label inference attack, ULIA.
  arXiv  Detail & Related papers  (2025-08-09T02:38:24Z)
• Unbiased Learning to Rank with Query-Level Click Propensity Estimation: Beyond Pointwise Observation and Relevance [74.43264459255121]
  In real-world scenarios, users often click only one or two results after examining multiple relevant options. We propose a query-level click propensity model to capture the probability that users will click on different result lists. Our method introduces a Dual Inverse Propensity Weighting mechanism to address both relevance saturation and position bias.
  arXiv  Detail & Related papers  (2025-02-17T03:55:51Z)
• Prioritized Ranking Experimental Design Using Recommender Systems in Two-Sided Platforms [0.0]
  We propose a novel design to mitigate the interference bias in estimating the total average treatment effect (TATE) of item-side interventions in online two-sided marketplaces. Our Two-Sided Prioritized Ranking (TSPR) design uses the recommender system as an instrument for experimentation.
  arXiv  Detail & Related papers  (2025-02-13T22:48:09Z)
• Fact Checking Beyond Training Set [64.88575826304024]
  We show that the retriever-reader suffers from performance deterioration when it is trained on labeled data from one domain and used in another domain. We propose an adversarial algorithm to make the retriever component robust against distribution shift. We then construct eight fact checking scenarios from these datasets, and compare our model to a set of strong baseline models.
  arXiv  Detail & Related papers  (2024-03-27T15:15:14Z)
• FedFixer: Mitigating Heterogeneous Label Noise in Federated Learning [15.382625332503125]
  Federated Learning (FL) heavily depends on label quality for its performance. The high loss incurred by client-specific samples in heterogeneous label noise poses challenges for distinguishing between client-specific and noisy label samples. We propose FedFixer, where the personalized model is introduced to cooperate with the global model to effectively select clean client-specific samples.
  arXiv  Detail & Related papers  (2024-03-25T09:24:05Z)
• AiGen-FoodReview: A Multimodal Dataset of Machine-Generated Restaurant Reviews and Images on Social Media [57.70351255180495]
  AiGen-FoodReview is a dataset of 20,144 restaurant review-image pairs divided into authentic and machine-generated. We explore unimodal and multimodal detection models, achieving 99.80% multimodal accuracy with FLAVA. The paper contributes by open-sourcing the dataset and releasing fake review detectors, recommending its use in unimodal and multimodal fake review detection tasks, and evaluating linguistic and visual features in synthetic versus authentic data.
  arXiv  Detail & Related papers  (2024-01-16T20:57:36Z)
• Delving into Identify-Emphasize Paradigm for Combating Unknown Bias [52.76758938921129]
  We propose an effective bias-conflicting scoring method (ECS) to boost the identification accuracy. We also propose gradient alignment (GA) to balance the contributions of the mined bias-aligned and bias-conflicting samples. Experiments are conducted on multiple datasets in various settings, demonstrating that the proposed solution can mitigate the impact of unknown biases.
  arXiv  Detail & Related papers  (2023-02-22T14:50:24Z)
• Debiasing Recommendation by Learning Identifiable Latent Confounders [49.16119112336605]
  Confounding bias arises due to the presence of unmeasured variables that can affect both a user's exposure and feedback. Existing methods either (1) make untenable assumptions about these unmeasured variables or (2) directly infer latent confounders from users' exposure. We propose a novel method, i.e., identifiable deconfounder (iDCF), which leverages a set of proxy variables to resolve the aforementioned non-identification issue.
  arXiv  Detail & Related papers  (2023-02-10T05:10:26Z)
• Assign Experiment Variants at Scale in Online Controlled Experiments [1.9205538784019935]
  Online controlled experiments (A/B tests) have become the gold standard for learning the impact of new product features in technology companies. Technology companies run A/B tests at scale -- hundreds if not thousands of A/B tests concurrently, each with millions of users. We present a novel assignment algorithm and statistical tests to validate the randomized assignments.
  arXiv  Detail & Related papers  (2022-12-17T00:45:12Z)
• Fair Effect Attribution in Parallel Online Experiments [57.13281584606437]
  A/B tests serve the purpose of reliably identifying the effect of changes introduced in online services. It is common for online platforms to run a large number of simultaneous experiments by splitting incoming user traffic randomly. Despite a perfect randomization between different groups, simultaneous experiments can interact with each other and create a negative impact on average population outcomes.
  arXiv  Detail & Related papers  (2022-10-15T17:15:51Z)
• Clustering-based Imputation for Dropout Buyers in Large-scale Online Experimentation [4.753069295451989]
  In online experimentation, appropriate metrics (e.g., purchase) provide strong evidence to support hypotheses and enhance the decision-making process. In this work, we introduce the concept of dropout buyers and categorize users with incomplete metric values into two groups: visitors and dropout buyers. For the analysis of incomplete metrics, we propose a clustering-based imputation method using $k$-nearest neighbors.
  arXiv  Detail & Related papers  (2022-09-09T01:05:53Z)
• Are Missing Links Predictable? An Inferential Benchmark for Knowledge Graph Completion [79.07695173192472]
  InferWiki improves upon existing benchmarks in inferential ability, assumptions, and patterns. Each testing sample is predictable with supportive data in the training set. In experiments, we curate two settings of InferWiki varying in sizes and structures, and apply the construction process on CoDEx as comparative datasets.
  arXiv  Detail & Related papers  (2021-08-03T09:51:15Z)
• Comparison Lift: Bandit-based Experimentation System for Online Advertising [10.499720681099452]
  Comparison Lift is an experimentation-as-a-service (E) application for testing online advertising audiences and creatives at JD.com. Unlike many other E tools that focus primarily on fixed sample A/B testing, Comparison Lift deploys a custom bandit-based experimentation algorithm. We estimate that the adaptive design in the product has generated 27% more clicks on average during testing compared to a fixed sample A/B design.
  arXiv  Detail & Related papers  (2020-09-16T19:08:44Z)
• Two-Sample Testing on Ranked Preference Data and the Role of Modeling Assumptions [57.77347280992548]
  In this paper, we design two-sample tests for pairwise comparison data and ranking data. Our test requires essentially no assumptions on the distributions. By applying our two-sample test on real-world pairwise comparison data, we conclude that ratings and rankings provided by people are indeed distributed differently.
  arXiv  Detail & Related papers  (2020-06-21T20:51:09Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.