Hybrid intelligence for dynamic job-shop scheduling with deep
reinforcement learning and attention mechanism
- URL: http://arxiv.org/abs/2201.00548v1
- Date: Mon, 3 Jan 2022 09:38:13 GMT
- Title: Hybrid intelligence for dynamic job-shop scheduling with deep
reinforcement learning and attention mechanism
- Authors: Yunhui Zeng, Zijun Liao, Yuanzhi Dai, Rong Wang, Xiu Li, Bo Yuan
- Abstract summary: We formulate the DJSP as a Markov decision process (MDP) to be tackled by reinforcement learning (RL)
We propose a flexible hybrid framework that takes disjunctive graphs as states and a set of general dispatching rules as the action space with minimum prior domain knowledge.
We present Gymjsp, a public benchmark based on the well-known OR-Library, to provide a standardized off-the-shelf facility for RL and DJSP research communities.
- Score: 28.28095225164155
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: The dynamic job-shop scheduling problem (DJSP) is a class of scheduling tasks
that specifically consider the inherent uncertainties such as changing order
requirements and possible machine breakdown in realistic smart manufacturing
settings. Since traditional methods cannot dynamically generate effective
scheduling strategies in face of the disturbance of environments, we formulate
the DJSP as a Markov decision process (MDP) to be tackled by reinforcement
learning (RL). For this purpose, we propose a flexible hybrid framework that
takes disjunctive graphs as states and a set of general dispatching rules as
the action space with minimum prior domain knowledge. The attention mechanism
is used as the graph representation learning (GRL) module for the feature
extraction of states, and the double dueling deep Q-network with prioritized
replay and noisy networks (D3QPN) is employed to map each state to the most
appropriate dispatching rule. Furthermore, we present Gymjsp, a public
benchmark based on the well-known OR-Library, to provide a standardized
off-the-shelf facility for RL and DJSP research communities. Comprehensive
experiments on various DJSP instances confirm that our proposed framework is
superior to baseline algorithms with smaller makespan across all instances and
provide empirical justification for the validity of the various components in
the hybrid framework.
Related papers
- Online Frequency Scheduling by Learning Parallel Actions [5.9838600557884805]
Frequency resources need to be assigned to a set of users while allowing for concurrent transmissions in the same sub-band.
Traditional methods are insufficient to cope with all the involved constraints and uncertainties.
We propose a scheduler based on action-branching over sub-bands, which is a deep Q-learning architecture with parallel decision capabilities.
arXiv Detail & Related papers (2024-06-07T16:14:51Z) - Intelligent Hybrid Resource Allocation in MEC-assisted RAN Slicing Network [72.2456220035229]
We aim to maximize the SSR for heterogeneous service demands in the cooperative MEC-assisted RAN slicing system.
We propose a recurrent graph reinforcement learning (RGRL) algorithm to intelligently learn the optimal hybrid RA policy.
arXiv Detail & Related papers (2024-05-02T01:36:13Z) - Intent-Aware DRL-Based Uplink Dynamic Scheduler for 5G-NR [30.146175299047325]
We investigate the problem of supporting Industrial Internet of Things user equipment (IIoT UEs) with intent (i.e., requested quality of service (QoS)) and random traffic arrival.
A deep reinforcement learning (DRL) based centralized dynamic scheduler for time-frequency resources is proposed to learn how to schedule the available communication resources.
arXiv Detail & Related papers (2024-03-27T08:57:15Z) - Learning-enabled Flexible Job-shop Scheduling for Scalable Smart
Manufacturing [11.509669981978874]
In smart manufacturing systems, flexible job-shop scheduling with transportation constraints is essential to optimize solutions for maximizing productivity.
Recent developments in deep reinforcement learning (DRL)-based methods for FJSPT have encountered a scale generalization challenge.
We introduce a novel graph-based DRL method, named the Heterogeneous Graph Scheduler (HGS)
arXiv Detail & Related papers (2024-02-14T06:49:23Z) - On the Role of Server Momentum in Federated Learning [85.54616432098706]
We propose a general framework for server momentum, that (a) covers a large class of momentum schemes that are unexplored in federated learning (FL)
We provide rigorous convergence analysis for the proposed framework.
arXiv Detail & Related papers (2023-12-19T23:56:49Z) - Action-Quantized Offline Reinforcement Learning for Robotic Skill
Learning [68.16998247593209]
offline reinforcement learning (RL) paradigm provides recipe to convert static behavior datasets into policies that can perform better than the policy that collected the data.
In this paper, we propose an adaptive scheme for action quantization.
We show that several state-of-the-art offline RL methods such as IQL, CQL, and BRAC improve in performance on benchmarks when combined with our proposed discretization scheme.
arXiv Detail & Related papers (2023-10-18T06:07:10Z) - Flexible Job Shop Scheduling via Dual Attention Network Based
Reinforcement Learning [73.19312285906891]
In flexible job shop scheduling problem (FJSP), operations can be processed on multiple machines, leading to intricate relationships between operations and machines.
Recent works have employed deep reinforcement learning (DRL) to learn priority dispatching rules (PDRs) for solving FJSP.
This paper presents a novel end-to-end learning framework that weds the merits of self-attention models for deep feature extraction and DRL for scalable decision-making.
arXiv Detail & Related papers (2023-05-09T01:35:48Z) - Pessimism meets VCG: Learning Dynamic Mechanism Design via Offline
Reinforcement Learning [114.36124979578896]
We design a dynamic mechanism using offline reinforcement learning algorithms.
Our algorithm is based on the pessimism principle and only requires a mild assumption on the coverage of the offline data set.
arXiv Detail & Related papers (2022-05-05T05:44:26Z) - Learning to schedule job-shop problems: Representation and policy
learning using graph neural network and reinforcement learning [9.379652654427959]
We propose a framework to learn to schedule a job-shop problem (JSSP) using a graph neural network (GNN) and reinforcement learning (RL)
We empirically demonstrate that the GNN scheduler, due to its superb generalization capability, outperforms practically favored rules and RL-based schedulers on various benchmark JSSP.
arXiv Detail & Related papers (2021-06-02T11:40:22Z) - Smart Scheduling based on Deep Reinforcement Learning for Cellular
Networks [18.04856086228028]
We propose a smart scheduling scheme based on deep reinforcement learning (DRL)
We provide implementation-friend designs, i.e., a scalable neural network design for the agent and a virtual environment training framework.
We show that the DRL-based smart scheduling outperforms the conventional scheduling method and can be adopted in practical systems.
arXiv Detail & Related papers (2021-03-22T02:09:16Z) - SUNRISE: A Simple Unified Framework for Ensemble Learning in Deep
Reinforcement Learning [102.78958681141577]
We present SUNRISE, a simple unified ensemble method, which is compatible with various off-policy deep reinforcement learning algorithms.
SUNRISE integrates two key ingredients: (a) ensemble-based weighted Bellman backups, which re-weight target Q-values based on uncertainty estimates from a Q-ensemble, and (b) an inference method that selects actions using the highest upper-confidence bounds for efficient exploration.
arXiv Detail & Related papers (2020-07-09T17:08:44Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.