RIE-Greedy: Regularization-Induced Exploration for Contextual Bandits

• URL: http://arxiv.org/abs/2603.11276v1
• Date: Wed, 11 Mar 2026 20:11:01 GMT
• Title: RIE-Greedy: Regularization-Induced Exploration for Contextual Bandits
• Abstract summary: Real-world contextual bandit problems with complex reward models are often tackled with iteratively trained models, such as boosting trees.<n>Existing approaches rely on sophisticated assumptions or intractable procedures that are hard to verify and implement in practice.<n>In this work, we explore the use of an exploration-free (pure-greedy) action selection strategy, that exploits the randomness inherent in model fitting process as an intrinsic source of exploration.
• Score: 4.282654316889406
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Real-world contextual bandit problems with complex reward models are often tackled with iteratively trained models, such as boosting trees. However, it is difficult to directly apply simple and effective exploration strategies--such as Thompson Sampling or UCB--on top of those black-box estimators. Existing approaches rely on sophisticated assumptions or intractable procedures that are hard to verify and implement in practice. In this work, we explore the use of an exploration-free (pure-greedy) action selection strategy, that exploits the randomness inherent in model fitting process as an intrinsic source of exploration. More specifically, we note that the stochasticity in cross-validation based regularization process can naturally induce Thompson Sampling-like exploration. We show that this regularization-induced exploration is theoretically equivalent to Thompson Sampling in the two-armed bandit case and empirically leads to reliable exploration in large-scale business environments compared to benchmark methods such as epsilon-greedy and other state-of-the-art approaches. Overall, our work reveals how regularized estimator training itself can induce effective exploration, offering both theoretical insight and practical guidance for contextual bandit design.

Related papers

• DiPO: Disentangled Perplexity Policy Optimization for Fine-grained Exploration-Exploitation Trade-Off [87.58233482504308]
  Reinforcement Learning with Verifiable Rewards (RLVR) has catalyzed significant advances in the reasoning capabilities of Large Language Models (LLMs)<n>This paper fully analyze the exploration and exploitation dilemma of extremely hard and easy samples during the training and propose a new fine-grained trade-off mechanism.
  arXiv  Detail & Related papers  (2026-04-15T14:12:10Z)
• Think Longer to Explore Deeper: Learn to Explore In-Context via Length-Incentivized Reinforcement Learning [53.58654277639939]
  In-context exploration is the intrinsic ability to generate, verify, and refine hypotheses within a single continuous context.<n>We propose Length-Incentivized Exploration, which explicitly encourages models to explore more.<n>Our method achieves an average improvement of 4.4% on in-domain tasks and a 2.7% gain on out-of-domain benchmarks.
  arXiv  Detail & Related papers  (2026-02-12T09:24:32Z)
• Consistency Is Not Always Correct: Towards Understanding the Role of Exploration in Post-Training Reasoning [75.79451512757844]
  Foundation models exhibit broad knowledge but limited task-specific reasoning.<n> RLVR and inference scaling motivate post-training strategies such as RLVR and inference scaling.<n>We show that RLVR induces a squeezing effect, reducing reasoning entropy and forgetting some correct paths.
  arXiv  Detail & Related papers  (2025-11-10T18:25:26Z)
• Infrequent Exploration in Linear Bandits [39.8867004581646]
  We introduce a simple and practical framework, INFEX, explicitly designed for infrequent exploration.<n> INFEX executes a base exploratory policy according to a given schedule while predominantly choosing greedy actions in between.<n>Our theoretical analysis demonstrates that INFEX achieves instance-dependent regret matching standard provably efficient algorithms.
  arXiv  Detail & Related papers  (2025-10-29T22:25:43Z)
• Robust Optimization with Diffusion Models for Green Security [49.68562792424776]
  In green security, defenders must forecast adversarial behavior, such as poaching, illegal logging, and illegal fishing, to plan effective patrols.<n>We propose a conditional diffusion model for adversary behavior modeling, leveraging its strong distribution-fitting capabilities.<n>We introduce a mixed strategy of mixed strategies and employ a twisted Sequential Monte Carlo (SMC) sampler for accurate sampling.
  arXiv  Detail & Related papers  (2025-02-19T05:30:46Z)
• Random Latent Exploration for Deep Reinforcement Learning [71.88709402926415]
  We introduce Random Latent Exploration (RLE), a simple yet effective exploration strategy in reinforcement learning (RL)<n>On average, RLE outperforms noise-based methods, which perturb the agent's actions, and bonus-based exploration, which rewards the agent for attempting novel behaviors.<n>RLE is as simple as noise-based methods, as it avoids complex bonus calculations but retains the deep exploration benefits of bonus-based methods.
  arXiv  Detail & Related papers  (2024-07-18T17:55:22Z)
• Adaptive Experimentation When You Can't Experiment [55.86593195947978]
  This paper introduces the emphconfounded pure exploration transductive linear bandit (textttCPET-LB) problem. Online services can employ a properly randomized encouragement that incentivizes users toward a specific treatment.
  arXiv  Detail & Related papers  (2024-06-15T20:54:48Z)
• Latent State Marginalization as a Low-cost Approach for Improving Exploration [79.12247903178934]
  We propose the adoption of latent variable policies within the MaxEnt framework. We show that latent variable policies naturally emerges under the use of world models with a latent belief state. We experimentally validate our method on continuous control tasks, showing that effective marginalization can lead to better exploration and more robust training.
  arXiv  Detail & Related papers  (2022-10-03T15:09:12Z)
• SEREN: Knowing When to Explore and When to Exploit [14.188362393915432]
  We introduce Sive Reinforcement Exploration Network (SEREN) that poses the exploration-exploitation trade-off as a game. Using a form of policies known as impulse control, switcher is able to determine the best set of states to switch to the exploration policy. We prove that SEREN converges quickly and induces a natural schedule towards pure exploitation.
  arXiv  Detail & Related papers  (2022-05-30T12:44:56Z)
• GuideBoot: Guided Bootstrap for Deep Contextual Bandits [14.983677719946666]
  We introduce Guided Bootstrap (GuideBoot) for contextual bandit algorithms. GuideBoot provides explicit guidance to the exploration behavior by training multiple models over both real samples and noisy samples with fake labels. The proposed method is efficient as it can make decisions on-the-fly by utilizing only one randomly chosen model.
  arXiv  Detail & Related papers  (2021-07-18T07:53:04Z)
• Deep Bandits Show-Off: Simple and Efficient Exploration with Deep Networks [14.178899938667161]
  We introduce Sample Average Uncertainty (SAU), a simple and efficient uncertainty measure for contextual bandits. Because of its simplicity SAU can be seamlessly applied to deep contextual bandits as a very scalable drop-in replacement for epsilon-greedy exploration.
  arXiv  Detail & Related papers  (2021-05-10T21:45:01Z)
• Intrinsic Exploration as Multi-Objective RL [29.124322674133]
  Intrinsic motivation enables reinforcement learning (RL) agents to explore when rewards are very sparse. We propose a framework based on multi-objective RL where both exploration and exploitation are being optimized as separate objectives. This formulation brings the balance between exploration and exploitation at a policy level, resulting in advantages over traditional methods.
  arXiv  Detail & Related papers  (2020-04-06T02:37:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.