A Comparative Study of Bayesian Contextual Bandits for Real-Time Warehouse Sorter Optimization

• URL: http://arxiv.org/abs/2606.23977v1
• Date: Mon, 22 Jun 2026 22:07:36 GMT
• Title: A Comparative Study of Bayesian Contextual Bandits for Real-Time Warehouse Sorter Optimization
• Abstract summary: In this study, we use an inbound receiving sorter at a high-volume e-commerce warehouse as our primary use case.<n>To address this real-time sorter diversion optimization challenge, we conducted a comparative study of three candidate hybrid machine learning frameworks.<n>Our results demonstrate that while tree-based reward models offer slightly better predictive power, the BCB framework achieved overall higher performance with 2.03% reward uplift over the baseline.
• Score: 1.8807636852384417
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Efficient sorter diversion control of automated material handling systems (MHS) is critical for optimizing operational efficiency in large-scale warehouse environments. In this study, we use an inbound receiving sorter at a high-volume e-commerce warehouse as our primary use case, where the sorter diversion system relies on cost functions with static weight configurations that fail to adapt to highly dynamic system contexts, such as volume mode, congestion level, equipment physical status, and upstream/downstream dependencies. To address this real-time sorter diversion optimization challenge, we conducted a comparative study of three candidate hybrid machine learning frameworks: Linear Regression with Gradient Descent Optimization (LR+GDO), XGBoost with Bayesian Optimization (XGB+BO), and Bayesian Contextual Bandits (BCB). Model training and evaluation were enabled by leveraging a high-fidelity physics-aware emulator to overcome the cold-start problem and allow a safe transition from offline to online learning. We performed comprehensive evaluations including reward model predictive accuracy, contextual sensitivity, action distribution, and projected reward uplift. Our results demonstrate that while tree-based reward models offer slightly better predictive power, the BCB framework achieved overall higher performance with 2.03% reward uplift over the heuristic baseline. Furthermore, BCB exhibits several superior characteristics, such as its decisive time-optimal policy backed by Bang-Bang control theory, continuous online learning capability, strategic balance between exploration and exploitation, and significantly shorter inference latency. These results demonstrate the potential of the BCB framework for real-time control optimization in large-scale warehouse environments, motivating further investigation toward operational deployment.

Related papers

• Reinforcement Learning-Guided Evolutionary Policy Optimization for Preference-Adjustable Heterogeneous Agile Earth Observation Satellite Scheduling [50.176885352114674]
  Heterogeneous agile Earth observation satellite (AEOS) scheduling requires task selection, satellite assignment, and observation sequencing.<n>This paper proposes an evolutionary policy optimization framework for heterogeneous AEOS scheduling with preference-adjustable weighted objectives.
  arXiv  Detail & Related papers  (2026-08-25T12:16:28Z)
• ClawGym II: Exploring Black-Box RL on Agent Harness [82.92963070856416]
  Agent harnesses have substantially improved performance on long-horizon tasks by coordinating agent interactions with the environment.<n>We present a unified black-box RL framework for stable and scalable optimization of general agents through complex harnesses.
  arXiv  Detail & Related papers  (2026-08-17T16:53:03Z)
• Toward an Energy-Optimized Operation of Data Centers Located in Wind Farms Using Reinforcement Learning [39.799207552858114]
  Reinforcement Learning is an online controller for curtailment-aware workload shifting in high-performance computing data centers.<n>We introduce a fixed-day simulation framework with synthetic wind and price signals and delayed completion feedback.<n>We evaluate two complementary countermeasures: optimization-based Learning and potential-based Reward Shaping.
  arXiv  Detail & Related papers  (2026-06-29T13:59:33Z)
• Long Live The Balance: Information Bottleneck Driven Tree-based Policy Optimization [18.398968714754474]
  We introduce IB-Score, a novel metric that evaluates policy's exploration-exploitation balance by quantifying the trade-off between step-level reasoning diversity and mutual information shared with the correct answer.<n>We propose Information Bottleneck-driven Tree-based Policy Optimization (IB-TPO), a principled framework that formulates IB-Score as a fine-grained optimization objective.<n>Our method significantly outperforms GRPO baseline by 2.9% to 3.6% and also outperforms other state-of-the-art online RL approaches.
  arXiv  Detail & Related papers  (2026-05-27T08:01:42Z)
• A Systematic Post-Train Framework for Video Generation [76.26555417456773]
  Large-scale video diffusion models have demonstrated impressive capabilities in generating high-resolution and semantically rich content.<n>A significant gap remains between their pretraining performance and real-world deployment requirements due to critical issues such as prompt sensitivity, temporal inconsistency, and prohibitive inference costs.<n>We propose a comprehensive post-training framework that systematically aligns pretrained models with user intentions through four synergistic stages.
  arXiv  Detail & Related papers  (2026-04-28T09:34:51Z)
• Scalable AI Inference: Performance Analysis and Optimization of AI Model Serving [0.9167082845109437]
  This study investigates the performance and optimization of a BentoML-based AI inference system for scalable model serving developed in collaboration with graphworks.ai.<n>The study examines how latency and throughput scale under varying workloads, how optimizations at the runtime, service, and deployment levels affect response time, and how deployment in a single-node K3s cluster influences resilience during disruptions.
  arXiv  Detail & Related papers  (2026-04-22T10:39:14Z)
• VBO-MI: A Fully Gradient-Based Bayesian Optimization Framework Using Variational Mutual Information Estimation [1.0829694003408499]
  VBO-MI is a fully gradient-based BO framework that leverages recent advances in variational mutual information estimation.<n>We evaluate our method on a diverse suite of benchmarks, including high-dimensional synthetic functions and complex real-world tasks.
  arXiv  Detail & Related papers  (2026-01-13T03:07:52Z)
• Momentum-constrained Hybrid Heuristic Trajectory Optimization Framework with Residual-enhanced DRL for Visually Impaired Scenarios [4.735413508037063]
  This paper proposes a momentum-constrained hybrid trajectory optimization framework (MHHTOF) tailored for assistive navigation in visually impaired scenarios.<n>It integrates trajectory sampling generation, optimization and evaluation with residual deep reinforcement learning (DRL)<n> Experimental results demonstrate that the proposed LSTM-BResPPO achieves significantly faster convergence, attaining stable policy performance in approximately half the training required by the PPO.
  arXiv  Detail & Related papers  (2025-09-19T04:33:39Z)
• LAPSO: A Unified Optimization View for Learning-Augmented Power System Operations [3.754570687412345]
  This paper proposes a holistic framework of Learning-Augmented Power System Operations (LAPSO)<n>LAPSO is centered on the operation stage and aims to break the boundary between temporally siloed power system tasks.<n>A dedicated Python package-lapso is introduced to automatically augment existing power system optimization models with learnable components.
  arXiv  Detail & Related papers  (2025-05-08T13:00:24Z)
• On the Role of Feedback in Test-Time Scaling of Agentic AI Workflows [71.92083784393418]
  Agentic AI (systems that autonomously plan and act) are becoming widespread, yet their task success rate on complex tasks remains low.<n>Inference-time alignment relies on three components: sampling, evaluation, and feedback.<n>We introduce Iterative Agent Decoding (IAD), a procedure that repeatedly inserts feedback extracted from different forms of critiques.
  arXiv  Detail & Related papers  (2025-04-02T17:40:47Z)
• Model-based Causal Bayesian Optimization [74.78486244786083]
  We introduce the first algorithm for Causal Bayesian Optimization with Multiplicative Weights (CBO-MW) We derive regret bounds for CBO-MW that naturally depend on graph-related quantities. Our experiments include a realistic demonstration of how CBO-MW can be used to learn users' demand patterns in a shared mobility system.
  arXiv  Detail & Related papers  (2023-07-31T13:02:36Z)
• End-to-End Meta-Bayesian Optimisation with Transformer Neural Processes [52.818579746354665]
  This paper proposes the first end-to-end differentiable meta-BO framework that generalises neural processes to learn acquisition functions via transformer architectures. We enable this end-to-end framework with reinforcement learning (RL) to tackle the lack of labelled acquisition data.
  arXiv  Detail & Related papers  (2023-05-25T10:58:46Z)
• When Demonstrations Meet Generative World Models: A Maximum Likelihood Framework for Offline Inverse Reinforcement Learning [62.00672284480755]
  This paper aims to recover the structure of rewards and environment dynamics that underlie observed actions in a fixed, finite set of demonstrations from an expert agent. Accurate models of expertise in executing a task has applications in safety-sensitive applications such as clinical decision making and autonomous driving.
  arXiv  Detail & Related papers  (2023-02-15T04:14:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.