Multi-Agent Reinforcement Learning for Heterogeneous Satellite Cluster Resources Optimization

• URL: http://arxiv.org/abs/2511.12792v1
• Date: Sun, 16 Nov 2025 21:47:04 GMT
• Title: Multi-Agent Reinforcement Learning for Heterogeneous Satellite Cluster Resources Optimization
• Authors: Mohamad A. Hady, Siyi Hu, Mahardhika Pratama, Zehong Cao, Ryszard Kowalczyk,
• Abstract summary: Two optical satellites and one SAR satellite operate cooperatively in low Earth orbit to capture ground targets and manage their limited onboard resources efficiently.<n>Traditional optimization methods struggle to handle the real-time, uncertain, and decentralized nature of Earth Observation (EO) operations.<n>This study systematically formulates the optimization problem from single-satellite to multi-satellite scenarios.<n>Using a near-realistic simulation environment built on the Basilisk and BSK-RL frameworks, we evaluate the performance and stability of state-of-the-art MARL algorithms.
• Score: 19.16014340215772
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: This work investigates resource optimization in heterogeneous satellite clusters performing autonomous Earth Observation (EO) missions using Reinforcement Learning (RL). In the proposed setting, two optical satellites and one Synthetic Aperture Radar (SAR) satellite operate cooperatively in low Earth orbit to capture ground targets and manage their limited onboard resources efficiently. Traditional optimization methods struggle to handle the real-time, uncertain, and decentralized nature of EO operations, motivating the use of RL and Multi-Agent Reinforcement Learning (MARL) for adaptive decision-making. This study systematically formulates the optimization problem from single-satellite to multi-satellite scenarios, addressing key challenges including energy and memory constraints, partial observability, and agent heterogeneity arising from diverse payload capabilities. Using a near-realistic simulation environment built on the Basilisk and BSK-RL frameworks, we evaluate the performance and stability of state-of-the-art MARL algorithms such as MAPPO, HAPPO, and HATRPO. Results show that MARL enables effective coordination across heterogeneous satellites, balancing imaging performance and resource utilization while mitigating non-stationarity and inter-agent reward coupling. The findings provide practical insights into scalable, autonomous satellite operations and contribute a foundation for future research on intelligent EO mission planning under heterogeneous and dynamic conditions.

Related papers

• Earth-Agent: Unlocking the Full Landscape of Earth Observation with Agents [49.3216026940601]
  Earth observation is essential for understanding the states of the Earth system.<n>Recent MLLMs have advanced EO research, but they still lack the capability to tackle complex tasks that require multi-step reasoning.<n>We introduce Earth-Agent, the first agentic framework that unifies RGB and spectral EO data within an MCP-based tool ecosystem.
  arXiv  Detail & Related papers  (2025-09-27T06:04:28Z)
• ASTREA: Introducing Agentic Intelligence for Orbital Thermal Autonomy [51.56484100374058]
  ASTREA is the first agentic system executed on flight-heritage hardware for autonomous spacecraft operations.<n>We integrate a resource-constrained Large Language Model (LLM) agent with a reinforcement learning controller in an asynchronous architecture tailored for space-qualified platforms.
  arXiv  Detail & Related papers  (2025-09-16T08:52:13Z)
• Joint AoI and Handover Optimization in Space-Air-Ground Integrated Network [48.485907216785904]
  Low Earth orbit (LEO) satellite constellations offer promising solutions with global coverage and reduced latency.<n>Yet struggle with intermittent coverage and intermittent communication windows due to orbital dynamics.<n>Our three-layer design employs hybrid free-space optical (FSO) links for high-capacity satellite-to-ground communication and reliable radio frequency (RF) links for HAP-to-ground transmission.
  arXiv  Detail & Related papers  (2025-09-16T06:16:56Z)
• TLE-Based A2C Agent for Terrestrial Coverage Orbital Path Planning [0.0]
  The congestion of Low Earth Orbit (LEO) poses persistent challenges to the efficient deployment and safe operation of Earth observation satellites.<n>This work presents a reinforcement learning framework using the Advantage Actor-Critic (A2C) algorithm to optimize satellite orbital parameters for precise terrestrial coverage.
  arXiv  Detail & Related papers  (2025-08-14T17:44:51Z)
• AI-Driven Collaborative Satellite Object Detection for Space Sustainability [29.817805350971366]
  The growing density of satellites in low-Earth orbit (LEO) presents serious challenges to space sustainability.<n>Traditional ground-based tracking systems are constrained by latency and coverage limitations.<n>We propose a novel satellite clustering framework that enables the collaborative execution of deep learning (DL)-based space object detection tasks across multiple satellites.
  arXiv  Detail & Related papers  (2025-08-01T16:31:55Z)
• Agentic Reinforced Policy Optimization [66.96989268893932]
  Large-scale reinforcement learning with verifiable rewards (RLVR) has demonstrated its effectiveness in harnessing the potential of large language models (LLMs) for single-turn reasoning tasks.<n>Current RL algorithms inadequately balance the models' intrinsic long-horizon reasoning capabilities and their proficiency in multi-turn tool interactions.<n>We propose Agentic Reinforced Policy Optimization (ARPO), a novel agentic RL algorithm tailored for training multi-turn LLM-based agents.
  arXiv  Detail & Related papers  (2025-07-26T07:53:11Z)
• LLM Meets the Sky: Heuristic Multi-Agent Reinforcement Learning for Secure Heterogeneous UAV Networks [57.27815890269697]
  This work focuses on maximizing the secrecy rate in heterogeneous UAV networks (HetUAVNs) under energy constraints.<n>We introduce a Large Language Model (LLM)-guided multi-agent learning approach.<n>Results show that our method outperforms existing baselines in secrecy and energy efficiency.
  arXiv  Detail & Related papers  (2025-07-23T04:22:57Z)
• On the Role of AI in Managing Satellite Constellations: Insights from the ConstellAI Project [1.706656684496508]
  This paper explores the role of Artificial Intelligence (AI) in optimizing the operation of satellite mega-constellations.<n>It draws from the ConstellAI project funded by the European Space Agency (ESA)<n>A consortium comprising GMV GmbH, Saarland University, and Thales Alenia Space collaborates to develop AI-driven algorithms.
  arXiv  Detail & Related papers  (2025-07-21T12:56:16Z)
• Multi-Agent Reinforcement Learning for Autonomous Multi-Satellite Earth Observation: A Realistic Case Study [10.393102715510937]
  The exponential growth of Low Earth Orbit (LEO) satellites has revolutionised Earth Observation (EO) missions.<n>Traditional optimisation approaches struggle to handle the real-time decision-making demands of dynamic EO missions.<n>We investigate RL-based autonomous EO mission planning by modelling single-satellite operations and extending to multi-satellite constellations.
  arXiv  Detail & Related papers  (2025-06-18T07:42:11Z)
• Low-altitude UAV Friendly-Jamming for Satellite-Maritime Communications via Generative AI-enabled Deep Reinforcement Learning [72.23178920029957]
  This paper presents a satellite-maritime communication system assisted by low-altitude unmanned aerial vehicle (UAV) friendly-jamming.<n>We formulate a secure satellite-maritime communication multi-objective optimization problem (SSMCMOP)<n>In order to solve the dynamic and long-term optimization problem, we reformulate it into a Markov decision process.<n>We then propose a transformer-enhanced soft actor-critic (TransSAC) algorithm, which is a generative artificial intelligence-enabled deep reinforcement learning approach.
  arXiv  Detail & Related papers  (2025-01-26T10:13:51Z)
• A Distance Similarity-based Genetic Optimization Algorithm for Satellite Ground Network Planning Considering Feeding Mode [53.71516191515285]
  The low transmission efficiency of the satellite data relay back mission has become a problem that is currently constraining the construction of the system. We propose a distance similarity-based genetic optimization algorithm (DSGA), which considers the state characteristics between the tasks and introduces a weighted Euclidean distance method to determine the similarity between the tasks.
  arXiv  Detail & Related papers  (2024-08-29T06:57:45Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.