Scheduling the NASA Deep Space Network with Deep Reinforcement Learning

• URL: http://arxiv.org/abs/2102.05167v1
• Date: Tue, 9 Feb 2021 22:48:05 GMT
• Title: Scheduling the NASA Deep Space Network with Deep Reinforcement Learning
• Authors: Edwin Goh, Hamsa Shwetha Venkataram, Mark Hoffmann, Mark Johnston, Brian Wilson
• Abstract summary: NASA's Deep Space Network (DSN) is the primary means of communications and a scientific instrument for dozens of active missions around the world. A rapidly rising number of spacecraft and increasingly complex scientific instruments with higher bandwidth requirements have resulted in demand that exceeds the network's capacity across its 12 antennae. This paper proposes a deep reinforcement learning approach to generate candidate DSN schedules from mission requests and spacecraft ephemeris data with demonstrated capability to address real-world operational constraints.
• Score: 0.4083182125683813
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: With three complexes spread evenly across the Earth, NASA's Deep Space Network (DSN) is the primary means of communications as well as a significant scientific instrument for dozens of active missions around the world. A rapidly rising number of spacecraft and increasingly complex scientific instruments with higher bandwidth requirements have resulted in demand that exceeds the network's capacity across its 12 antennae. The existing DSN scheduling process operates on a rolling weekly basis and is time-consuming; for a given week, generation of the final baseline schedule of spacecraft tracking passes takes roughly 5 months from the initial requirements submission deadline, with several weeks of peer-to-peer negotiations in between. This paper proposes a deep reinforcement learning (RL) approach to generate candidate DSN schedules from mission requests and spacecraft ephemeris data with demonstrated capability to address real-world operational constraints. A deep RL agent is developed that takes mission requests for a given week as input, and interacts with a DSN scheduling environment to allocate tracks such that its reward signal is maximized. A comparison is made between an agent trained using Proximal Policy Optimization and its random, untrained counterpart. The results represent a proof-of-concept that, given a well-shaped reward signal, a deep RL agent can learn the complex heuristics used by experts to schedule the DSN. A trained agent can potentially be used to generate candidate schedules to bootstrap the scheduling process and thus reduce the turnaround cycle for DSN scheduling.

Related papers

• Exploring RL-based LLM Training for Formal Language Tasks with Programmed Rewards [49.7719149179179]
  This paper investigates the feasibility of using PPO for reinforcement learning (RL) from explicitly programmed reward signals. We focus on tasks expressed through formal languages, such as programming, where explicit reward functions can be programmed to automatically assess quality of generated outputs. Our results show that pure RL-based training for the two formal language tasks is challenging, with success being limited even for the simple arithmetic task.
  arXiv  Detail & Related papers  (2024-10-22T15:59:58Z)
• Tera-SpaceCom: GNN-based Deep Reinforcement Learning for Joint Resource Allocation and Task Offloading in TeraHertz Band Space Networks [11.881917133887102]
  Tera-SpaceCom is envisioned as a promising technology to enable various space science and communication applications. A graph neural network (GNN)-deep reinforcement learning (DRL)-based joint resource allocation and task offloading algorithm is proposed.
  arXiv  Detail & Related papers  (2024-09-12T10:26:17Z)
• HARL: Hierarchical Adaptive Reinforcement Learning Based Auto Scheduler for Neural Networks [51.71682428015139]
  We propose HARL, a reinforcement learning-based auto-scheduler for efficient tensor program exploration. HarL improves the tensor operator performance by 22% and the search speed by 4.3x compared to the state-of-the-art auto-scheduler. Inference performance and search speed are also significantly improved on end-to-end neural networks.
  arXiv  Detail & Related papers  (2022-11-21T04:15:27Z)
• Training Spiking Neural Networks with Local Tandem Learning [96.32026780517097]
  Spiking neural networks (SNNs) are shown to be more biologically plausible and energy efficient than their predecessors. In this paper, we put forward a generalized learning rule, termed Local Tandem Learning (LTL) We demonstrate rapid network convergence within five training epochs on the CIFAR-10 dataset while having low computational complexity.
  arXiv  Detail & Related papers  (2022-10-10T10:05:00Z)
• Federated learning for LEO constellations via inter-HAP links [0.0]
  Low Earth Obit (LEO) satellite constellations have seen a sharp increase of deployment in recent years. To apply machine learning (ML) in such applications, the traditional way of downloading satellite data such as imagery to a ground station (GS) is not desirable. We show that existing FL solutions do not fit well in such LEO constellation scenarios because of significant challenges such as excessive convergence delay and unreliable wireless channels.
  arXiv  Detail & Related papers  (2022-05-15T08:22:52Z)
• Learning Emergent Random Access Protocol for LEO Satellite Networks [51.575090080749554]
  We propose a novel grant-free random access solution for LEO SAT networks, dubbed emergent random access channel protocol (eRACH) eRACH is a model-free approach that emerges through interaction with the non-stationary network environment. Compared to RACH, we show from various simulations that our proposed eRACH yields 54.6% higher average network throughput.
  arXiv  Detail & Related papers  (2021-12-03T07:44:45Z)
• Semantic-Aware Collaborative Deep Reinforcement Learning Over Wireless Cellular Networks [82.02891936174221]
  Collaborative deep reinforcement learning (CDRL) algorithms in which multiple agents can coordinate over a wireless network is a promising approach. In this paper, a novel semantic-aware CDRL method is proposed to enable a group of untrained agents with semantically-linked DRL tasks to collaborate efficiently across a resource-constrained wireless cellular network.
  arXiv  Detail & Related papers  (2021-11-23T18:24:47Z)
• Time Series Comparisons in Deep Space Network [0.0]
  The Deep Space Network is NASA's international array of antennas that support interplanetary spacecraft missions. Monitor data on each track reports on the performance of specific spacecraft operations and the DSN itself. This tool has three functions: (1) identification of the top 10 similar historical tracks, (2) detection of anomalies compared to the reference normal track, and (3) comparison of statistical differences between two given tracks.
  arXiv  Detail & Related papers  (2021-11-02T06:38:59Z)
• Smart Scheduling based on Deep Reinforcement Learning for Cellular Networks [18.04856086228028]
  We propose a smart scheduling scheme based on deep reinforcement learning (DRL) We provide implementation-friend designs, i.e., a scalable neural network design for the agent and a virtual environment training framework. We show that the DRL-based smart scheduling outperforms the conventional scheduling method and can be adopted in practical systems.
  arXiv  Detail & Related papers  (2021-03-22T02:09:16Z)
• Deep-Reinforcement-Learning-Based Scheduling with Contiguous Resource Allocation for Next-Generation Cellular Systems [4.227387975627387]
  We propose a novel scheduling algorithm with contiguous frequency-domain resource allocation (FDRA) based on deep reinforcement learning (DRL) The proposed DRL-based scheduling algorithm outperforms other representative baseline schemes while having lower online computational complexity.
  arXiv  Detail & Related papers  (2020-10-11T05:41:40Z)
• Integrating LEO Satellite and UAV Relaying via Reinforcement Learning for Non-Terrestrial Networks [51.05735925326235]
  A mega-constellation of low-earth orbit (LEO) satellites has the potential to enable long-range communication with low latency. We study the problem of forwarding packets between two faraway ground terminals, through an LEO satellite selected from an orbiting constellation. To maximize the end-to-end data rate, the satellite association and HAP location should be optimized. We tackle this problem using deep reinforcement learning (DRL) with a novel action dimension reduction technique.
  arXiv  Detail & Related papers  (2020-05-26T05:39:27Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.