Understanding Endogenous Data Drift in Adaptive Models with Recourse-Seeking Users

• URL: http://arxiv.org/abs/2503.09658v2
• Date: Sun, 02 Nov 2025 16:58:23 GMT
• Title: Understanding Endogenous Data Drift in Adaptive Models with Recourse-Seeking Users
• Authors: Bo-Yi Liu, Zhi-Xuan Liu, Kuan Lun Chen, Shih-Yu Tsai, Jie Gao, Hao-Tsung Yang,
• Abstract summary: We study user strategic behaviors and their interactions with decision-making systems under resource constraints and competitive dynamics.<n>We propose two methods--Fair-top-k and Dynamic Continual Learning--which significantly reduce recourse cost and improve model robustness.<n>Our findings draw connections to economic theories, highlighting how algorithmic decision-making can unintentionally reinforce a higher standard and generate endogenous barriers to entry.
• Score: 6.782864450313782
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Deep learning models are widely used in decision-making and recommendation systems, where they typically rely on the assumption of a static data distribution between training and deployment. However, real-world deployment environments often violate this assumption. Users who receive negative outcomes may adapt their features to meet model criteria, i.e., recourse action. These adaptive behaviors create shifts in the data distribution and when models are retrained on this shifted data, a feedback loop emerges: user behavior influences the model, and the updated model in turn reshapes future user behavior. Despite its importance, this bidirectional interaction between users and models has received limited attention. In this work, we develop a general framework to model user strategic behaviors and their interactions with decision-making systems under resource constraints and competitive dynamics. Both the theoretical and empirical analyses show that user recourse behavior tends to push logistic and MLP models toward increasingly higher decision standards, resulting in higher recourse costs and less reliable recourse actions over time. To mitigate these challenges, we propose two methods--Fair-top-k and Dynamic Continual Learning (DCL)--which significantly reduce recourse cost and improve model robustness. Our findings draw connections to economic theories, highlighting how algorithmic decision-making can unintentionally reinforce a higher standard and generate endogenous barriers to entry.

Related papers

• Pre-training for Recommendation Unlearning [14.514770044236375]
  UnlearnRec is a model-agnostic pre-training paradigm that prepares systems for efficient unlearning operations.<n>Our method delivers exceptional unlearning effectiveness while providing more than 10x speedup compared to retraining approaches.
  arXiv  Detail & Related papers  (2025-05-28T17:57:11Z)
• Anticipating Gaming to Incentivize Improvement: Guiding Agents in (Fair) Strategic Classification [6.660458629649826]
  We explore individuals' choice between genuinely improving their qualifications ('improvement'') vs. attempting to deceive the algorithm.<n>We formulate these interactions as a Stackelberg game, where a firm deploys a (fair) classifier, and individuals strategically respond.
  arXiv  Detail & Related papers  (2025-05-08T18:47:23Z)
• Reinforcement Learning for Machine Learning Model Deployment: Evaluating Multi-Armed Bandits in ML Ops Environments [0.0]
  We investigate whether reinforcement learning (RL)-based model management can manage deployment decisions more effectively.<n>Our approach enables more adaptive production environments by continuously evaluating deployed models and rolling back underperforming ones in real-time.<n>Our findings suggest that RL-based model management can improve automation, reduce reliance on manual interventions, and mitigate risks associated with post-deployment model failures.
  arXiv  Detail & Related papers  (2025-03-28T16:42:21Z)
• Information-Seeking Decision Strategies Mitigate Risk in Dynamic, Uncertain Environments [0.0]
  We compare the performance of normative reward- and information-seeking strategies in a foraging task. We find subtle disparities in the actions they take, resulting in meaningful performance differences. Our findings support the adaptive value of information-seeking behaviors that can mitigate risk with minimal reward loss.
  arXiv  Detail & Related papers  (2025-03-24T19:55:41Z)
• Global-Decision-Focused Neural ODEs for Proactive Grid Resilience Management [50.34345101758248]
  We propose predict-all-then-optimize-globally (PATOG), a framework that integrates outage prediction with globally optimized interventions. Our approach ensures spatially and temporally coherent decision-making, improving both predictive accuracy and operational efficiency. Experiments on synthetic and real-world datasets demonstrate significant improvements in outage prediction consistency and grid resilience.
  arXiv  Detail & Related papers  (2025-02-25T16:15:35Z)
• Learning from Reward-Free Offline Data: A Case for Planning with Latent Dynamics Models [79.2162092822111]
  We systematically evaluate reinforcement learning (RL) and control-based methods on a suite of navigation tasks.<n>We employ a latent dynamics model using the Joint Embedding Predictive Architecture (JEPA) and employ it for planning.<n>Our results show that model-free RL benefits most from large amounts of high-quality data, whereas model-based planning generalizes better to unseen layouts.
  arXiv  Detail & Related papers  (2025-02-20T18:39:41Z)
• The Edge-of-Reach Problem in Offline Model-Based Reinforcement Learning [31.8260779160424]
  We investigate how popular algorithms perform as the learned dynamics model is improved.<n>We propose Reach-Aware Learning (RAVL), a simple and robust method that directly addresses the edge-of-reach problem.
  arXiv  Detail & Related papers  (2024-02-19T20:38:00Z)
• Deep autoregressive density nets vs neural ensembles for model-based offline reinforcement learning [2.9158689853305693]
  We consider a model-based reinforcement learning algorithm that infers the system dynamics from the available data and performs policy optimization on imaginary model rollouts. This approach is vulnerable to exploiting model errors which can lead to catastrophic failures on the real system. We show that better performance can be obtained with a single well-calibrated autoregressive model on the D4RL benchmark.
  arXiv  Detail & Related papers  (2024-02-05T10:18:15Z)
• Secrets of RLHF in Large Language Models Part II: Reward Modeling [134.97964938009588]
  We introduce a series of novel methods to mitigate the influence of incorrect and ambiguous preferences in the dataset. We also introduce contrastive learning to enhance the ability of reward models to distinguish between chosen and rejected responses.
  arXiv  Detail & Related papers  (2024-01-11T17:56:59Z)
• Dealing with uncertainty: balancing exploration and exploitation in deep recurrent reinforcement learning [0.0]
  Incomplete knowledge of the environment leads an agent to make decisions under uncertainty. One of the major dilemmas in Reinforcement Learning (RL) where an autonomous agent has to balance two contrasting needs in making its decisions. We show that adaptive methods better approximate the trade-off between exploration and exploitation.
  arXiv  Detail & Related papers  (2023-10-12T13:45:33Z)
• Interpretable Imitation Learning with Dynamic Causal Relations [65.18456572421702]
  We propose to expose captured knowledge in the form of a directed acyclic causal graph. We also design this causal discovery process to be state-dependent, enabling it to model the dynamics in latent causal graphs. The proposed framework is composed of three parts: a dynamic causal discovery module, a causality encoding module, and a prediction module, and is trained in an end-to-end manner.
  arXiv  Detail & Related papers  (2023-09-30T20:59:42Z)
• Causal Disentangled Variational Auto-Encoder for Preference Understanding in Recommendation [50.93536377097659]
  This paper introduces the Causal Disentangled Variational Auto-Encoder (CaD-VAE), a novel approach for learning causal disentangled representations from interaction data in recommender systems. The approach utilizes structural causal models to generate causal representations that describe the causal relationship between latent factors.
  arXiv  Detail & Related papers  (2023-04-17T00:10:56Z)
• Causal Strategic Classification: A Tale of Two Shifts [11.929584800629675]
  We show how strategic behavior and causal effects underlie two complementing forms of distribution shift. We propose a learning algorithm that balances these two forces and over time, and permits end-to-end training.
  arXiv  Detail & Related papers  (2023-02-13T11:35:59Z)
• When to Update Your Model: Constrained Model-based Reinforcement Learning [50.74369835934703]
  We propose a novel and general theoretical scheme for a non-decreasing performance guarantee of model-based RL (MBRL) Our follow-up derived bounds reveal the relationship between model shifts and performance improvement. A further example demonstrates that learning models from a dynamically-varying number of explorations benefit the eventual returns.
  arXiv  Detail & Related papers  (2022-10-15T17:57:43Z)
• Preference Enhanced Social Influence Modeling for Network-Aware Cascade Prediction [59.221668173521884]
  We propose a novel framework to promote cascade size prediction by enhancing the user preference modeling. Our end-to-end method makes the user activating process of information diffusion more adaptive and accurate.
  arXiv  Detail & Related papers  (2022-04-18T09:25:06Z)
• Learning Self-Modulating Attention in Continuous Time Space with Applications to Sequential Recommendation [102.24108167002252]
  We propose a novel attention network, named self-modulating attention, that models the complex and non-linearly evolving dynamic user preferences. We empirically demonstrate the effectiveness of our method on top-N sequential recommendation tasks, and the results on three large-scale real-world datasets show that our model can achieve state-of-the-art performance.
  arXiv  Detail & Related papers  (2022-03-30T03:54:11Z)
• Learning from Heterogeneous Data Based on Social Interactions over Graphs [58.34060409467834]
  This work proposes a decentralized architecture, where individual agents aim at solving a classification problem while observing streaming features of different dimensions. We show that the. strategy enables the agents to learn consistently under this highly-heterogeneous setting. We show that the. strategy enables the agents to learn consistently under this highly-heterogeneous setting.
  arXiv  Detail & Related papers  (2021-12-17T12:47:18Z)
• Towards Robust and Adaptive Motion Forecasting: A Causal Representation Perspective [72.55093886515824]
  We introduce a causal formalism of motion forecasting, which casts the problem as a dynamic process with three groups of latent variables. We devise a modular architecture that factorizes the representations of invariant mechanisms and style confounders to approximate a causal graph. Experiment results on synthetic and real datasets show that our three proposed components significantly improve the robustness and reusability of the learned motion representations.
  arXiv  Detail & Related papers  (2021-11-29T18:59:09Z)
• Revisiting Design Choices in Model-Based Offline Reinforcement Learning [39.01805509055988]
  Offline reinforcement learning enables agents to leverage large pre-collected datasets of environment transitions to learn control policies. This paper compares and designs novel protocols to investigate their interaction with other hyper parameters, such as the number of models, or imaginary rollout horizon.
  arXiv  Detail & Related papers  (2021-10-08T13:51:34Z)
• Online Learning Demands in Max-min Fairness [91.37280766977923]
  We describe mechanisms for the allocation of a scarce resource among multiple users in a way that is efficient, fair, and strategy-proof. The mechanism is repeated for multiple rounds and a user's requirements can change on each round. At the end of each round, users provide feedback about the allocation they received, enabling the mechanism to learn user preferences over time.
  arXiv  Detail & Related papers  (2020-12-15T22:15:20Z)
• Coordinated Online Learning for Multi-Agent Systems with Coupled Constraints and Perturbed Utility Observations [91.02019381927236]
  We introduce a novel method to steer the agents toward a stable population state, fulfilling the given resource constraints. The proposed method is a decentralized resource pricing method based on the resource loads resulting from the augmentation of the game's Lagrangian.
  arXiv  Detail & Related papers  (2020-10-21T10:11:17Z)
• On the model-based stochastic value gradient for continuous reinforcement learning [50.085645237597056]
  We show that simple model-based agents can outperform state-of-the-art model-free agents in terms of both sample-efficiency and final reward. Our findings suggest that model-based policy evaluation deserves closer attention.
  arXiv  Detail & Related papers  (2020-08-28T17:58:29Z)
• Counterfactual Learning of Stochastic Policies with Continuous Actions [42.903292639112536]
  We introduce a modelling strategy based on a joint kernel embedding of contexts and actions. We empirically show that the optimization aspect of counterfactual learning is important. We propose an evaluation protocol for offline policies in real-world logged systems.
  arXiv  Detail & Related papers  (2020-04-22T07:42:30Z)
• Causal Strategic Linear Regression [5.672132510411465]
  In many predictive decision-making scenarios, such as credit scoring and academic testing, a decision-maker must construct a model that accounts for agents' propensity to "game" the decision rule. We join concurrent work in modeling agents' outcomes as a function of their changeable attributes. We provide efficient algorithms for learning decision rules that optimize three distinct decision-maker objectives.
  arXiv  Detail & Related papers  (2020-02-24T03:57:22Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.