Deep Controlled Learning for Inventory Control

• URL: http://arxiv.org/abs/2011.15122v7
• Date: Tue, 03 Jun 2025 13:35:58 GMT
• Title: Deep Controlled Learning for Inventory Control
• Authors: Tarkan Temizöz, Christina Imdahl, Remco Dijkman, Douniel Lamghari-Idrissi, Willem van Jaarsveld,
• Abstract summary: The application of Deep Reinforcement Learning (DRL) to inventory management is an emerging field.<n>Traditional DRL algorithms, originally developed for diverse domains such as game-playing and robotics, may not be well-suited for the specific challenges posed by inventory management.<n>We propose Deep Learning (DCL), a new DRL algorithm designed for highly numerical problems.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The application of Deep Reinforcement Learning (DRL) to inventory management is an emerging field. However, traditional DRL algorithms, originally developed for diverse domains such as game-playing and robotics, may not be well-suited for the specific challenges posed by inventory management. Consequently, these algorithms often fail to outperform established heuristics; for instance, no existing DRL approach consistently surpasses the capped base-stock policy in lost sales inventory control. This highlights a critical gap in the practical application of DRL to inventory management: the highly stochastic nature of inventory problems requires tailored solutions. In response, we propose Deep Controlled Learning (DCL), a new DRL algorithm designed for highly stochastic problems. DCL is based on approximate policy iteration and incorporates an efficient simulation mechanism, combining Sequential Halving with Common Random Numbers. Our numerical studies demonstrate that DCL consistently outperforms state-of-the-art heuristics and DRL algorithms across various inventory settings, including lost sales, perishable inventory systems, and inventory systems with random lead times. DCL achieves lower average costs in all test cases while maintaining an optimality gap of no more than 0.2\%. Remarkably, this performance is achieved using the same hyperparameter set across all experiments, underscoring the robustness and generalizability of our approach. These findings contribute to the ongoing exploration of tailored DRL algorithms for inventory management, providing a foundation for further research and practical application in this area.

Related papers

• Structure-Informed Deep Reinforcement Learning for Inventory Management [8.697068617006964]
  This paper investigates the application of Deep Reinforcement Learning to classical inventory management problems.<n>We apply a DRL algorithm based on DirectBackprop to several fundamental inventory management scenarios.<n>We show that our generic DRL implementation performs competitively against or outperforms established benchmarks and distributions.
  arXiv  Detail & Related papers  (2025-07-29T17:41:45Z)
• Deep RL Dual Sourcing Inventory Management with Supply and Capacity Risk Awareness [4.583289433858458]
  We study how to efficiently apply reinforcement learning (RL) for solving large-scale optimization problems by leveraging intervention models.<n>We demonstrate our approach on a challenging real-world application, the multi-sourcing multi-period inventory management problem in supply chain optimization.
  arXiv  Detail & Related papers  (2025-07-19T02:44:45Z)
• Classical and Deep Reinforcement Learning Inventory Control Policies for Pharmaceutical Supply Chains with Perishability and Non-Stationarity [1.0124625066746595]
  We study inventory control policies for pharmaceutical supply chains, addressing challenges such as perishability, yield uncertainty, and non-stationary demand.<n>We benchmark three policies--order-up-to (OUT), projected inventory level (PIL), and deep reinforcement learning (DRL)
  arXiv  Detail & Related papers  (2025-01-18T22:40:33Z)
• Zero-shot Generalization in Inventory Management: Train, then Estimate and Decide [0.0]
  Deploying deep reinforcement learning (DRL) in real-world inventory management presents challenges. These challenges highlight a research gap, suggesting a need for a unifying framework to model and solve sequential decision-making under parameter uncertainty. We address this by exploring an underexplored area of DRL for inventory management: training generally capable agents (GCAs) under zero-shot generalization (ZSG)
  arXiv  Detail & Related papers  (2024-11-01T11:20:05Z)
• Comparison of Model Predictive Control and Proximal Policy Optimization for a 1-DOF Helicopter System [0.7499722271664147]
  This study conducts a comparative analysis of Model Predictive Control (MPC) and Proximal Policy Optimization (PPO), a Deep Reinforcement Learning (DRL) algorithm, applied to a Quanser Aero 2 system. PPO excels in rise-time and adaptability, making it a promising approach for applications requiring rapid response and adaptability.
  arXiv  Detail & Related papers  (2024-08-28T08:35:34Z)
• How Can LLM Guide RL? A Value-Based Approach [68.55316627400683]
  Reinforcement learning (RL) has become the de facto standard practice for sequential decision-making problems by improving future acting policies with feedback. Recent developments in large language models (LLMs) have showcased impressive capabilities in language understanding and generation, yet they fall short in exploration and self-improvement capabilities. We develop an algorithm named LINVIT that incorporates LLM guidance as a regularization factor in value-based RL, leading to significant reductions in the amount of data needed for learning.
  arXiv  Detail & Related papers  (2024-02-25T20:07:13Z)
• Deep reinforcement learning for machine scheduling: Methodology, the state-of-the-art, and future directions [2.4541568670428915]
  Machine scheduling aims to optimize job assignments to machines while adhering to manufacturing rules and job specifications. Deep Reinforcement Learning (DRL), a key component of artificial general intelligence, has shown promise in various domains like gaming and robotics. This paper offers a comprehensive review and comparison of DRL-based approaches, highlighting their methodology, applications, advantages, and limitations.
  arXiv  Detail & Related papers  (2023-10-04T22:45:09Z)
• Efficient Deep Reinforcement Learning Requires Regulating Overfitting [91.88004732618381]
  We show that high temporal-difference (TD) error on the validation set of transitions is the main culprit that severely affects the performance of deep RL algorithms. We show that a simple online model selection method that targets the validation TD error is effective across state-based DMC and Gym tasks.
  arXiv  Detail & Related papers  (2023-04-20T17:11:05Z)
• Benchmarking Actor-Critic Deep Reinforcement Learning Algorithms for Robotics Control with Action Constraints [9.293472255463454]
  This study presents a benchmark for evaluating action-constrained reinforcement learning (RL) algorithms. We evaluate existing algorithms and their novel variants across multiple robotics control environments.
  arXiv  Detail & Related papers  (2023-04-18T05:45:09Z)
• Train Hard, Fight Easy: Robust Meta Reinforcement Learning [78.16589993684698]
  A major challenge of reinforcement learning (RL) in real-world applications is the variation between environments, tasks or clients. Standard MRL methods optimize the average return over tasks, but often suffer from poor results in tasks of high risk or difficulty. In this work, we define a robust MRL objective with a controlled level. The data inefficiency is addressed via the novel Robust Meta RL algorithm (RoML)
  arXiv  Detail & Related papers  (2023-01-26T14:54:39Z)
• A Tutorial on Meta-Reinforcement Learning [69.76165430793571]
  We cast the development of better RL algorithms as a machine learning problem itself in a process called meta-RL.<n>We discuss how, at a high level, meta-RL research can be clustered based on the presence of a task distribution and the learning budget available for each individual task.<n>We conclude by presenting the open problems on the path to making meta-RL part of the standard toolbox for a deep RL practitioner.
  arXiv  Detail & Related papers  (2023-01-19T12:01:41Z)
• LCRL: Certified Policy Synthesis via Logically-Constrained Reinforcement Learning [78.2286146954051]
  LCRL implements model-free Reinforcement Learning (RL) algorithms over unknown Decision Processes (MDPs) We present case studies to demonstrate the applicability, ease of use, scalability, and performance of LCRL.
  arXiv  Detail & Related papers  (2022-09-21T13:21:00Z)
• A Transferable and Automatic Tuning of Deep Reinforcement Learning for Cost Effective Phishing Detection [21.481974148873807]
  Many challenging real-world problems require the deployment of ensembles multiple complementary learning models. Deep Reinforcement Learning (DRL) offers a cost-effective alternative, where detectors are dynamically chosen based on the output of their predecessors.
  arXiv  Detail & Related papers  (2022-09-19T14:09:07Z)
• Towards Deployment-Efficient Reinforcement Learning: Lower Bound and Optimality [141.89413461337324]
  Deployment efficiency is an important criterion for many real-world applications of reinforcement learning (RL) We propose a theoretical formulation for deployment-efficient RL (DE-RL) from an "optimization with constraints" perspective.
  arXiv  Detail & Related papers  (2022-02-14T01:31:46Z)
• Math Programming based Reinforcement Learning for Multi-Echelon Inventory Management [1.9161790404101895]
  Reinforcement learning has lead to considerable break-throughs in diverse areas such as robotics, games and many others. But the application to RL in complex real-world decision making problems remains limited. These characteristics make the problem considerably harder to solve for existing RL methods that rely on enumeration techniques to solve per step action problems. We show that a properly selected discretization of the underlying uncertain distribution can yield near optimal actor policy even with very few samples from the underlying uncertainty. We find that PARL outperforms commonly used base stock by 44.7% and the best performing RL method by up to 12.1% on average
  arXiv  Detail & Related papers  (2021-12-04T01:40:34Z)
• Towards Standardizing Reinforcement Learning Approaches for Stochastic Production Scheduling [77.34726150561087]
  reinforcement learning can be used to solve scheduling problems. Existing studies rely on (sometimes) complex simulations for which the code is unavailable. There is a vast array of RL designs to choose from. standardization of model descriptions - both production setup and RL design - and validation scheme are a prerequisite.
  arXiv  Detail & Related papers  (2021-04-16T16:07:10Z)
• Critic Regularized Regression [70.8487887738354]
  We propose a novel offline RL algorithm to learn policies from data using a form of critic-regularized regression (CRR) We find that CRR performs surprisingly well and scales to tasks with high-dimensional state and action spaces.
  arXiv  Detail & Related papers  (2020-06-26T17:50:26Z)
• Guided Constrained Policy Optimization for Dynamic Quadrupedal Robot Locomotion [78.46388769788405]
  We introduce guided constrained policy optimization (GCPO), an RL framework based upon our implementation of constrained policy optimization (CPPO) We show that guided constrained RL offers faster convergence close to the desired optimum resulting in an optimal, yet physically feasible, robotic control behavior without the need for precise reward function tuning.
  arXiv  Detail & Related papers  (2020-02-22T10:15:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.