The Price of Isolation: Estimating the Ecosystem Cost of Symmetric Two-Sided A/B Testing

• URL: http://arxiv.org/abs/2608.04432v1
• Date: Wed, 05 Aug 2026 04:22:05 GMT
• Title: The Price of Isolation: Estimating the Ecosystem Cost of Symmetric Two-Sided A/B Testing
• Abstract summary: symmetric two-sided isolation is widely used for creator-side and cold-start experiments.<n>Isolation thins each viewer's candidate catalog, and intuition suggests the resulting engagement cost should fade as the platform grows.<n>We show that, in an order-statistics model of engagement, whether this intuition holds depends on the upper tail of match quality.
• Score: 8.026599954667267
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: On two-sided content platforms, symmetric two-sided isolation (assigning matched fractions of creators and viewers to isolated treatment and control submarkets) is widely used for creator-side and cold-start experiments because it removes cross-arm marketplace interference. Isolation, however, thins each viewer's candidate catalog, and intuition suggests the resulting engagement cost should fade as the platform grows: a small fraction of a vast catalog is still vast. We show that, in an order-statistics model of engagement, whether this intuition holds depends on the upper tail of match quality. Extreme-value theory yields tail-class loss laws with a sharp dichotomy: for light or bounded tails the loss vanishes as the candidate pool grows, whereas under heavy tails it converges to a size-independent constant, so expanding the candidate pool, even by orders of magnitude, does not asymptotically eliminate the cost. Evidence from two production experiments on a platform with millions of active creators is consistent with this picture: a pure A/A traffic sweep reveals a measurable, depth-graded engagement cost; a one-sided catalog ablation independently shows that per-viewer thinning contributes to the loss; and a tail index calibrated on the small exploration pool predicts an effect consistent with the one observed in the far larger full-catalog ablation. Isolation thus carries a price that experimenters should budget for, like any other cost. We give practitioners a preflight procedure that estimates it before launch, sizes traffic accordingly, and recommends a fallback design when the predicted cost exceeds a chosen tolerance.

Related papers

• Across-Design Uncertainty in Short Pricing Panels: Inference and Identification [0.0]
  We evaluate the inferential consequences of sparsity in a synthetic data-generating process.<n>Across-design component accounts for 97.6% of estimation error variance for a gradient-boosted specification.
  arXiv  Detail & Related papers  (2026-08-21T17:40:31Z)
• Cost-Aware Recovery-Pathway Identification and Bayesian Optimization for Autonomous Materials Discovery [0.0]
  We formulate a sequential decision problem with a discrete pathway-identification stage and a continuous within-pathway optimization stage.<n>Our implementation, Coactive learning, combines a cost-sensitive Bayesian hypothesis-discrimination policy motivated by EC2.<n>We evaluate the method on synthetic benchmarks constrained by selected results reported for PNNL's CICERO selective-precipitation study.
  arXiv  Detail & Related papers  (2026-07-26T23:56:58Z)
• Best-Arm Identification with Generative Proxy [10.674965991983974]
  We study fixed-confidence best-arm identification in which each costly reward pull is paired with a cheap but correlated proxy score.<n>We propose PROBE, a phase-elimination algorithm that maintains an upper certificate on the residual variance with an ordinary least squares fit.<n>We prove that PROBE is $$-PAC and attains the known-correlation oracle sample complexity up to a constant multiplicative factor and a constant additive calibration cost.
  arXiv  Detail & Related papers  (2026-07-08T00:41:43Z)
• Observationally Informed Adaptive Causal Experimental Design [55.998153710215654]
  We propose Active Residual Learning, a new paradigm that leverages the observational model as a foundational prior.<n>This approach shifts the experimental focus from learning target causal quantities from scratch to efficiently estimating the residuals required to correct observational bias.<n> Experiments on synthetic and semi-synthetic benchmarks demonstrate that R-Design significantly outperforms baselines.
  arXiv  Detail & Related papers  (2026-03-04T06:52:37Z)
• Sharp Convergence Rates for Masked Diffusion Models [53.117058231393834]
  We develop a total-variation based analysis for the Euler method that overcomes limitations.<n>Our results relax assumptions on score estimation, improve parameter dependencies, and establish convergence guarantees.<n>Overall, our analysis introduces a direct TV-based error decomposition along the CTMC trajectory and a decoupling-based path-wise analysis for FHS.
  arXiv  Detail & Related papers  (2026-02-26T00:47:51Z)
• When Fusion Helps and When It Breaks: View-Aligned Robustness in Same-Source Financial Imaging [4.857561672498819]
  We study same-source multi-view learning and adversarial robustness for next-day direction prediction with financial image representations.<n>We construct two window-aligned views from each rolling window: an OHLCV-rendered price/volume chart and a technical-indicator matrix.<n>We evaluate test-time L-infinity perturbations using FGSM and PGD under two threat scenarios: view-constrained attacks that perturb one view and joint attacks that perturb both.
  arXiv  Detail & Related papers  (2026-02-11T16:45:23Z)
• LLM Collusion [5.363252654303049]
  Large language models (LLMs) can facilitate collusion in a duopoly when both sellers rely on the same pre-trained model.<n>We show that configuring LLMs for robustness and robustness can induce collusion via a phase transition.
  arXiv  Detail & Related papers  (2026-01-03T20:38:21Z)
• ZIP-RC: Optimizing Test-Time Compute via Zero-Overhead Joint Reward-Cost Prediction [57.799425838564]
  We present ZIP-RC, an adaptive inference method that equips models with zero-overhead inference-time predictions of reward and cost.<n> ZIP-RC improves accuracy by up to 12% over majority voting at equal or lower average cost.
  arXiv  Detail & Related papers  (2025-12-01T09:44:31Z)
• Estimating Treatment Effects under Recommender Interference: A Structured Neural Networks Approach [32.55211271796683]
  We show that commonly adopted difference-in-means estimators can lead to severely biased estimates due to recommender interference.<n>We propose a recommender choice model'' that explicitly represents the interference pathway.<n>We validate our method with a large-scale field experiment on Weixin short-video platform.
  arXiv  Detail & Related papers  (2024-06-20T14:53:26Z)
• Mutual Wasserstein Discrepancy Minimization for Sequential Recommendation [82.0801585843835]
  We propose a novel self-supervised learning framework based on Mutual WasserStein discrepancy minimization MStein for the sequential recommendation. We also propose a novel contrastive learning loss based on Wasserstein Discrepancy Measurement.
  arXiv  Detail & Related papers  (2023-01-28T13:38:48Z)
• Breaking the Sample Complexity Barrier to Regret-Optimal Model-Free Reinforcement Learning [52.76230802067506]
  A novel model-free algorithm is proposed to minimize regret in episodic reinforcement learning. The proposed algorithm employs an em early-settled reference update rule, with the aid of two Q-learning sequences. The design principle of our early-settled variance reduction method might be of independent interest to other RL settings.
  arXiv  Detail & Related papers  (2021-10-09T21:13:48Z)
• LSDAT: Low-Rank and Sparse Decomposition for Decision-based Adversarial Attack [74.5144793386864]
  LSDAT crafts perturbations in the low-dimensional subspace formed by the sparse component of the input sample and that of an adversarial sample. LSD works directly in the image pixel domain to guarantee that non-$ell$ constraints, such as sparsity, are satisfied.
  arXiv  Detail & Related papers  (2021-03-19T13:10:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.