Comparing Deep Reinforcement Learning Algorithms in Two-Echelon Supply Chains

• URL: http://arxiv.org/abs/2204.09603v3
• Date: Fri, 17 Nov 2023 16:42:46 GMT
• Title: Comparing Deep Reinforcement Learning Algorithms in Two-Echelon Supply Chains
• Authors: Francesco Stranieri and Fabio Stella
• Abstract summary: We analyze and compare the performance of state-of-the-art deep reinforcement learning algorithms for solving the supply chain inventory management problem. This study provides detailed insight into the design and development of an open-source software library that provides a customizable environment for solving the supply chain inventory management problem.
• Score: 1.4685355149711299
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: In this study, we analyze and compare the performance of state-of-the-art deep reinforcement learning algorithms for solving the supply chain inventory management problem. This complex sequential decision-making problem consists of determining the optimal quantity of products to be produced and shipped across different warehouses over a given time horizon. In particular, we present a mathematical formulation of a two-echelon supply chain environment with stochastic and seasonal demand, which allows managing an arbitrary number of warehouses and product types. Through a rich set of numerical experiments, we compare the performance of different deep reinforcement learning algorithms under various supply chain structures, topologies, demands, capacities, and costs. The results of the experimental plan indicate that deep reinforcement learning algorithms outperform traditional inventory management strategies, such as the static (s, Q)-policy. Furthermore, this study provides detailed insight into the design and development of an open-source software library that provides a customizable environment for solving the supply chain inventory management problem using a wide range of data-driven approaches.

Related papers

• Enhancing Supply Chain Visibility with Knowledge Graphs and Large Language Models [49.898152180805454]
  This paper presents a novel framework leveraging Knowledge Graphs (KGs) and Large Language Models (LLMs) to enhance supply chain visibility. Our zero-shot, LLM-driven approach automates the extraction of supply chain information from diverse public sources. With high accuracy in NER and RE tasks, it provides an effective tool for understanding complex, multi-tiered supply networks.
  arXiv  Detail & Related papers  (2024-08-05T17:11:29Z)
• Distributionally Robust Model-based Reinforcement Learning with Large State Spaces [55.14361269378122]
  Three major challenges in reinforcement learning are the complex dynamical systems with large state spaces, the costly data acquisition processes, and the deviation of real-world dynamics from the training environment deployment. We study distributionally robust Markov decision processes with continuous state spaces under the widely used Kullback-Leibler, chi-square, and total variation uncertainty sets. We propose a model-based approach that utilizes Gaussian Processes and the maximum variance reduction algorithm to efficiently learn multi-output nominal transition dynamics.
  arXiv  Detail & Related papers  (2023-09-05T13:42:11Z)
• MARLIM: Multi-Agent Reinforcement Learning for Inventory Management [1.1470070927586016]
  This paper presents a novel reinforcement learning framework called MARLIM to address the inventory management problem. Within this context, controllers are developed through single or multiple agents in a cooperative setting. Numerical experiments on real data demonstrate the benefits of reinforcement learning methods over traditional baselines.
  arXiv  Detail & Related papers  (2023-08-03T09:31:45Z)
• Enhancing Human-like Multi-Modal Reasoning: A New Challenging Dataset and Comprehensive Framework [51.44863255495668]
  Multimodal reasoning is a critical component in the pursuit of artificial intelligence systems that exhibit human-like intelligence. We present Multi-Modal Reasoning(COCO-MMR) dataset, a novel dataset that encompasses an extensive collection of open-ended questions. We propose innovative techniques, including multi-hop cross-modal attention and sentence-level contrastive learning, to enhance the image and text encoders.
  arXiv  Detail & Related papers  (2023-07-24T08:58:25Z)
• Multi-Agent Reinforcement Learning with Shared Resources for Inventory Management [62.23979094308932]
  In our setting, the constraint on the shared resources (such as the inventory capacity) couples the otherwise independent control for each SKU. We formulate the problem with this structure as Shared-Resource Game (SRSG)and propose an efficient algorithm called Context-aware Decentralized PPO (CD-PPO) Through extensive experiments, we demonstrate that CD-PPO can accelerate the learning procedure compared with standard MARL algorithms.
  arXiv  Detail & Related papers  (2022-12-15T09:35:54Z)
• No-Regret Learning in Two-Echelon Supply Chain with Unknown Demand Distribution [48.27759561064771]
  We consider the two-echelon supply chain model introduced in [Cachon and Zipkin, 1999] under two different settings. We design algorithms that achieve favorable guarantees for both regret and convergence to the optimal inventory decision in both settings. Our algorithms are based on Online Gradient Descent and Online Newton Step, together with several new ingredients specifically designed for our problem.
  arXiv  Detail & Related papers  (2022-10-23T08:45:39Z)
• A Multi-label Continual Learning Framework to Scale Deep Learning Approaches for Packaging Equipment Monitoring [57.5099555438223]
  We study multi-label classification in the continual scenario for the first time. We propose an efficient approach that has a logarithmic complexity with regard to the number of tasks. We validate our approach on a real-world multi-label Forecasting problem from the packaging industry.
  arXiv  Detail & Related papers  (2022-08-08T15:58:39Z)
• Learning General Inventory Management Policy for Large Supply Chain Network [2.4660652494309936]
  This study proposes a reinforcement learning-based warehouse inventory management algorithm. It can be used for supply chain systems where both the number of products and retailers are large. Our experiments on both real and artificial data demonstrate that our algorithm with approximated simulation can successfully handle large supply chain networks.
  arXiv  Detail & Related papers  (2022-04-28T09:43:47Z)
• Math Programming based Reinforcement Learning for Multi-Echelon Inventory Management [1.9161790404101895]
  Reinforcement learning has lead to considerable break-throughs in diverse areas such as robotics, games and many others. But the application to RL in complex real-world decision making problems remains limited. These characteristics make the problem considerably harder to solve for existing RL methods that rely on enumeration techniques to solve per step action problems. We show that a properly selected discretization of the underlying uncertain distribution can yield near optimal actor policy even with very few samples from the underlying uncertainty. We find that PARL outperforms commonly used base stock by 44.7% and the best performing RL method by up to 12.1% on average
  arXiv  Detail & Related papers  (2021-12-04T01:40:34Z)
• Reinforcement Learning for Multi-Product Multi-Node Inventory Management in Supply Chains [17.260459603456745]
  This paper describes the application of reinforcement learning (RL) to multi-product inventory management in supply chains. Experiments show that the proposed approach is able to handle a multi-objective reward comprised of maximising product sales and minimising wastage of perishable products.
  arXiv  Detail & Related papers  (2020-06-07T04:02:59Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.