Unintended Consequences of Recommender System Interventions: Evidence from a Field Experiment

• URL: http://arxiv.org/abs/2606.08265v1
• Date: Sat, 06 Jun 2026 17:21:22 GMT
• Title: Unintended Consequences of Recommender System Interventions: Evidence from a Field Experiment
• Abstract summary: We show that user-facing interventions can effectively retrain the underlying algorithm.<n>Our findings challenge standard evaluation metrics in platform governance and social responsibility initiatives.
• Score: 0.4375619650153184
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Platform content interventions in recommendation systems are typically evaluated as static "nudges", ignoring that the systems adaptively learn from the resulting user behavior. We investigate this dynamic through a large-scale field experiment on a short-video platform. The experiment involves a "sleep reminder" campaign designed to reduce late-night usage. Paradoxically, the intervention increased late-night engagement by 14.75% and overall platform usage by 2.18%, and the effects persisted for weeks even after the experiment. We explain this through a forced-exploration mechanism, showing that by revealing high latent demand for the promoted content, the intervention triggers a recommendation policy update that routine user behavior would not produce. The data generated by the intervention induced the algorithm to update its post-campaign policy, reinforcing the very engagement loops the campaign aimed to mitigate. Our findings demonstrate that user-facing interventions can effectively retrain the underlying algorithm, triggering durable, system-wide shifts in content distribution that challenge standard evaluation metrics in platform governance and social responsibility initiatives.

Related papers

• A Self-Triggered Agentic Push Recommendation System [77.13502692161426]
  Push notification is a critical recommendation scenario on large-scale platforms.<n>We propose a proactive, Self-Triggered End-to-end Agentic Push Recommendation System.<n>STEPS is already fully deployed at Douyin with over 1 billion users.
  arXiv  Detail & Related papers  (2026-08-03T09:20:53Z)
• The Illusion of Intervention: Your LLM-Simulated Experiment is an Observational Study [56.649987197786096]
  Large language models (LLMs) show potential as simulators of human behavior.<n>LLMs can induce unintended shifts in latent user attributes, causing user drift.<n>We formalize the confounding or selection bias that can arise due to user drift.
  arXiv  Detail & Related papers  (2026-05-20T06:09:41Z)
• Retentive Relevance: Capturing Long-Term User Value in Recommendation Systems [29.596401271139797]
  We introduce Retentive Relevance, a novel content-level survey-based feedback measure.<n>Retentive Relevance directly assesses users' intent to return to the platform for similar content.<n>We show that Retentive Relevance significantly outperforms both engagement signals and other survey measures in predicting next-day retention.
  arXiv  Detail & Related papers  (2025-10-08T23:38:57Z)
• Finding Interest Needle in Popularity Haystack: Improving Retrieval by Modeling Item Exposure [8.3095709445007]
  We introduce an exposure-aware retrieval scoring approach, which explicitly models item exposure probability and adjusts retrieval-stage ranking at inference time.<n>We validate our approach through online A/B experiments in a real-world video recommendation system, demonstrating a 25% increase in uniquely retrieved items and a 40% reduction in the dominance of over-popular content.<n>Our results establish a scalable, deployable solution for mitigating popularity bias at the retrieval stage, offering a new paradigm for bias-aware personalization.
  arXiv  Detail & Related papers  (2025-03-31T00:04:01Z)
• Long-Sequence Recommendation Models Need Decoupled Embeddings [49.410906935283585]
  We identify and characterize a neglected deficiency in existing long-sequence recommendation models.<n>A single set of embeddings struggles with learning both attention and representation, leading to interference between these two processes.<n>We propose the Decoupled Attention and Representation Embeddings (DARE) model, where two distinct embedding tables are learned separately to fully decouple attention and representation.
  arXiv  Detail & Related papers  (2024-10-03T15:45:15Z)
• Algorithmic Drift: A Simulation Framework to Study the Effects of Recommender Systems on User Preferences [7.552217586057245]
  We propose a simulation framework that mimics user-recommender system interactions in a long-term scenario. We introduce two novel metrics for quantifying the algorithm's impact on user preferences, specifically in terms of drift over time.
  arXiv  Detail & Related papers  (2024-09-24T21:54:22Z)
• System-2 Recommenders: Disentangling Utility and Engagement in Recommendation Systems via Temporal Point-Processes [80.97898201876592]
  We propose a generative model in which past content interactions impact the arrival rates of users based on a self-exciting Hawkes process. We show analytically that given samples it is possible to disentangle System-1 and System-2 and allow content optimization based on user utility.
  arXiv  Detail & Related papers  (2024-05-29T18:19:37Z)
• Measuring Strategization in Recommendation: Users Adapt Their Behavior to Shape Future Content [66.71102704873185]
  We test for user strategization by conducting a lab experiment and survey. We find strong evidence of strategization across outcome metrics, including participants' dwell time and use of "likes" Our findings suggest that platforms cannot ignore the effect of their algorithms on user behavior.
  arXiv  Detail & Related papers  (2024-05-09T07:36:08Z)
• Beyond Trial-and-Error: Predicting User Abandonment After a Moderation Intervention [0.6918368994425961]
  Current content moderation follows a reactive, trial-and-error approach.<n>We introduce a proactive, predictive approach that enables moderators to anticipate the impact of their actions before implementation.<n>We study the reactions of 16,540 users to a massive ban of online communities on Reddit.
  arXiv  Detail & Related papers  (2024-04-23T08:52:41Z)
• Breaking Feedback Loops in Recommender Systems with Causal Inference [99.22185950608838]
  Recent work has shown that feedback loops may compromise recommendation quality and homogenize user behavior. We propose the Causal Adjustment for Feedback Loops (CAFL), an algorithm that provably breaks feedback loops using causal inference. We show that CAFL improves recommendation quality when compared to prior correction methods.
  arXiv  Detail & Related papers  (2022-07-04T17:58:39Z)
• Long-term Dynamics of Fairness Intervention in Connection Recommender Systems [5.048563042541915]
  We study a connection recommender system patterned after the systems employed by web-scale social networks. We find that, although seemingly fair in aggregate, common exposure and utility parity interventions fail to mitigate amplification of biases in the long term.
  arXiv  Detail & Related papers  (2022-03-30T16:27:48Z)
• Sayer: Using Implicit Feedback to Optimize System Policies [63.992191765269396]
  We develop a methodology that leverages implicit feedback to evaluate and train new system policies. Sayer builds on two ideas from reinforcement learning to leverage data collected by an existing policy. We show that Sayer can evaluate arbitrary policies accurately, and train new policies that outperform the production policies.
  arXiv  Detail & Related papers  (2021-10-28T04:16:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.