Multi-Agent Reinforcement Learning for Autonomous Multi-Satellite Earth Observation: A Realistic Case Study

• URL: http://arxiv.org/abs/2506.15207v2
• Date: Wed, 05 Nov 2025 03:26:44 GMT
• Title: Multi-Agent Reinforcement Learning for Autonomous Multi-Satellite Earth Observation: A Realistic Case Study
• Authors: Mohamad A. Hady, Siyi Hu, Mahardhika Pratama, Jimmy Cao, Ryszard Kowalczyk,
• Abstract summary: The exponential growth of Low Earth Orbit (LEO) satellites has revolutionised Earth Observation (EO) missions.<n>Traditional optimisation approaches struggle to handle the real-time decision-making demands of dynamic EO missions.<n>We investigate RL-based autonomous EO mission planning by modelling single-satellite operations and extending to multi-satellite constellations.
• Score: 10.393102715510937
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: The exponential growth of Low Earth Orbit (LEO) satellites has revolutionised Earth Observation (EO) missions, addressing challenges in climate monitoring, disaster management, and more. However, autonomous coordination in multi-satellite systems remains a fundamental challenge. Traditional optimisation approaches struggle to handle the real-time decision-making demands of dynamic EO missions, necessitating the use of Reinforcement Learning (RL) and Multi-Agent Reinforcement Learning (MARL). In this paper, we investigate RL-based autonomous EO mission planning by modelling single-satellite operations and extending to multi-satellite constellations using MARL frameworks. We address key challenges, including energy and data storage limitations, uncertainties in satellite observations, and the complexities of decentralised coordination under partial observability. By leveraging a near-realistic satellite simulation environment, we evaluate the training stability and performance of state-of-the-art MARL algorithms, including PPO, IPPO, MAPPO, and HAPPO. Our results demonstrate that MARL can effectively balance imaging and resource management while addressing non-stationarity and reward interdependency in multi-satellite coordination. The insights gained from this study provide a foundation for autonomous satellite operations, offering practical guidelines for improving policy learning in decentralised EO missions.

Related papers

• IMAGINE: Intelligent Multi-Agent Godot-based Indoor Networked Exploration [0.0]
  This paper implements Multi-Agent Reinforcement Learning (MARL) to address challenges in a 2D indoor environment.<n>Policy training aims to achieve emergent collaborative behaviours and decision-making under uncertainty.
  arXiv  Detail & Related papers  (2026-02-02T22:08:41Z)
• Multi-Agent Reinforcement Learning for Heterogeneous Satellite Cluster Resources Optimization [19.16014340215772]
  Two optical satellites and one SAR satellite operate cooperatively in low Earth orbit to capture ground targets and manage their limited onboard resources efficiently.<n>Traditional optimization methods struggle to handle the real-time, uncertain, and decentralized nature of Earth Observation (EO) operations.<n>This study systematically formulates the optimization problem from single-satellite to multi-satellite scenarios.<n>Using a near-realistic simulation environment built on the Basilisk and BSK-RL frameworks, we evaluate the performance and stability of state-of-the-art MARL algorithms.
  arXiv  Detail & Related papers  (2025-11-16T21:47:04Z)
• Earth-Agent: Unlocking the Full Landscape of Earth Observation with Agents [49.3216026940601]
  Earth observation is essential for understanding the states of the Earth system.<n>Recent MLLMs have advanced EO research, but they still lack the capability to tackle complex tasks that require multi-step reasoning.<n>We introduce Earth-Agent, the first agentic framework that unifies RGB and spectral EO data within an MCP-based tool ecosystem.
  arXiv  Detail & Related papers  (2025-09-27T06:04:28Z)
• Joint AoI and Handover Optimization in Space-Air-Ground Integrated Network [48.485907216785904]
  Low Earth orbit (LEO) satellite constellations offer promising solutions with global coverage and reduced latency.<n>Yet struggle with intermittent coverage and intermittent communication windows due to orbital dynamics.<n>Our three-layer design employs hybrid free-space optical (FSO) links for high-capacity satellite-to-ground communication and reliable radio frequency (RF) links for HAP-to-ground transmission.
  arXiv  Detail & Related papers  (2025-09-16T06:16:56Z)
• Accelerating Privacy-Preserving Federated Learning in Large-Scale LEO Satellite Systems [57.692181589325116]
  Large-scale low-Earth-orbit (LEO) satellite systems are increasingly valued for their ability to enable rapid and wide-area data exchange.<n>Due to privacy concerns and regulatory constraints, raw data collected at remote clients cannot be centrally aggregated.<n>Federated learning offers a privacy-preserving alternative by training local models on distributed devices and exchanging only model parameters.<n>We propose a discrete temporal graph-based on-demand scheduling framework that dynamically allocates communication resources to accelerate federated learning.
  arXiv  Detail & Related papers  (2025-09-05T03:33:42Z)
• AI-Driven Collaborative Satellite Object Detection for Space Sustainability [29.817805350971366]
  The growing density of satellites in low-Earth orbit (LEO) presents serious challenges to space sustainability.<n>Traditional ground-based tracking systems are constrained by latency and coverage limitations.<n>We propose a novel satellite clustering framework that enables the collaborative execution of deep learning (DL)-based space object detection tasks across multiple satellites.
  arXiv  Detail & Related papers  (2025-08-01T16:31:55Z)
• On the Role of AI in Managing Satellite Constellations: Insights from the ConstellAI Project [1.706656684496508]
  This paper explores the role of Artificial Intelligence (AI) in optimizing the operation of satellite mega-constellations.<n>It draws from the ConstellAI project funded by the European Space Agency (ESA)<n>A consortium comprising GMV GmbH, Saarland University, and Thales Alenia Space collaborates to develop AI-driven algorithms.
  arXiv  Detail & Related papers  (2025-07-21T12:56:16Z)
• Benchmarking LLMs' Swarm intelligence [50.544186914115045]
  Large Language Models (LLMs) show potential for complex reasoning, yet their capacity for emergent coordination in Multi-Agent Systems (MAS) remains largely unexplored.<n>We introduce SwarmBench, a novel benchmark designed to systematically evaluate tasks of LLMs acting as decentralized agents.<n>We propose metrics for coordination effectiveness and analyze emergent group dynamics.
  arXiv  Detail & Related papers  (2025-05-07T12:32:01Z)
• Low-altitude Friendly-Jamming for Satellite-Maritime Communications via Generative AI-enabled Deep Reinforcement Learning [72.72954660774002]
  Low Earth Orbit (LEO) satellites can be used to assist maritime wireless communications for data transmission across wide-ranging areas.<n>Extensive coverage of LEO satellites, combined with openness of channels, can cause the communication process to suffer from security risks.<n>This paper presents a low-altitude friendly-jamming LEO satellite-maritime communication system enabled by a unmanned aerial vehicle.
  arXiv  Detail & Related papers  (2025-01-26T10:13:51Z)
• On-Air Deep Learning Integrated Semantic Inference Models for Enhanced Earth Observation Satellite Networks [28.69148416385582]
  Domain-adapted Large Language Models (LLMs) provide a solution by enabling the integration of raw and processed EO data. This study provides a thorough examination of using semantic inference and deep learning for sophisticated EO systems. It presents an innovative architecture for semantic communication in EO satellite networks, designed to improve data transmission efficiency.
  arXiv  Detail & Related papers  (2024-09-23T17:42:05Z)
• Reinforcement Learning-enabled Satellite Constellation Reconfiguration and Retasking for Mission-Critical Applications [10.652828373995519]
  We critically assess the impact of satellite failures on constellation performance and the associated task requirements. We introduce reinforcement learning (RL) techniques, specifically Q-learning, Policy Gradient, Deep Q-Network (DQN), and Proximal Policy Optimization (PPO) Our results demonstrate that DQN and PPO achieve effective outcomes in terms of average rewards, task completion rates, and response times.
  arXiv  Detail & Related papers  (2024-09-03T20:01:56Z)
• A Distance Similarity-based Genetic Optimization Algorithm for Satellite Ground Network Planning Considering Feeding Mode [53.71516191515285]
  The low transmission efficiency of the satellite data relay back mission has become a problem that is currently constraining the construction of the system. We propose a distance similarity-based genetic optimization algorithm (DSGA), which considers the state characteristics between the tasks and introduces a weighted Euclidean distance method to determine the similarity between the tasks.
  arXiv  Detail & Related papers  (2024-08-29T06:57:45Z)
• Collaborative Ground-Space Communications via Evolutionary Multi-objective Deep Reinforcement Learning [113.48727062141764]
  We propose a distributed collaborative beamforming (DCB)-based uplink communication paradigm for enabling ground-space direct communications. DCB treats the terminals that are unable to establish efficient direct connections with the low Earth orbit (LEO) satellites as distributed antennas. We propose an evolutionary multi-objective deep reinforcement learning algorithm to obtain the desirable policies.
  arXiv  Detail & Related papers  (2024-04-11T03:13:02Z)
• FedSN: A Federated Learning Framework over Heterogeneous LEO Satellite Networks [18.213174641216884]
  A large number of Low Earth Orbit (LEO) satellites have been launched and deployed successfully in space by commercial companies, such as SpaceX. Due to multimodal sensors equipped by the LEO satellites, they serve not only for communication but also for various machine learning applications, such as space modulation recognition, remote sensing image classification, etc. We propose FedSN as a general FL framework to tackle the above challenges, and fully explore data diversity on LEO satellites.
  arXiv  Detail & Related papers  (2023-11-02T14:47:06Z)
• Artificial Intelligence for Satellite Communication: A Review [91.3755431537592]
  This work provides a general overview of AI, its diverse sub-fields, and its state-of-the-art algorithms. The application of AI to a wide variety of satellite communication aspects have demonstrated excellent potential, including beam-hopping, anti-jamming, network traffic forecasting, channel modeling, telemetry mining, ionospheric scintillation detecting, interference managing, remote sensing, behavior modeling, space-air-ground integrating, and energy managing.
  arXiv  Detail & Related papers  (2021-01-25T13:01:16Z)
• Bottom-up mechanism and improved contract net protocol for the dynamic task planning of heterogeneous Earth observation resources [61.75759893720484]
  Earth observation resources are becoming increasingly indispensable in disaster relief, damage assessment and related domains. Many unpredicted factors, such as the change of observation task requirements, to the occurring of bad weather and resource failures, may cause the scheduled observation scheme to become infeasible. A bottom-up distributed coordinated framework together with an improved contract net are proposed to facilitate the dynamic task replanning for heterogeneous Earth observation resources.
  arXiv  Detail & Related papers  (2020-07-13T03:51:08Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.