LunarFM: A Shared Multimodal Representation of the Moon's Surface

• URL: http://arxiv.org/abs/2607.22408v1
• Date: Fri, 24 Jul 2026 15:25:20 GMT
• Title: LunarFM: A Shared Multimodal Representation of the Moon's Surface
• Abstract summary: We introduce LunarFM, a model that learns a general representation of the lunar surface from diverse orbital measurements.<n>LunarFM assimilates observations from six instruments across three lunar missions, mapping 18 input channels to a shared embedding space.<n>We demonstrate that this embedding space supports a diverse range of downstream applications, including similarity search, few-shot resource mapping, mineral abundance regression, and geological unit classification.
• Score: 3.029669084751708
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The renewed global focus on lunar exploration, driven by the prospect of in-situ resource utilization and a sustained human presence on the Moon, has created growing demand for accurate, large-scale characterization of the lunar surface. Although vast quantities of orbital remote-sensing data have been collected, scientific analysis and resource mapping remain fragmented by heterogeneous multiinstrument observations, sparse labels, and bespoke task-specific modelling workflows. Here we introduce LunarFM, a multimodal foundation model that learns a general representation of the lunar surface from diverse orbital measurements. LunarFM assimilates observations from six instruments across three lunar missions, mapping 18 input channels to a shared embedding space. We demonstrate that this embedding space supports a diverse range of downstream applications, including similarity search, few-shot resource mapping, mineral abundance regression, and geological unit classification, enabling efficient scientific investigation and resource-oriented analysis. We provide a machine-learning-ready dataset of co-registered multimodal observations spanning latitudes from 70°S to 70°N, a pretrained multimodal masked autoencoder, and a companion embedding dataset providing a joint 768-dimensional representation of lunar surface properties. All code and data are available at https://lunarfm.trillium.tech/

Related papers

• SomBench: Benchmark Dataset for Advancing Machine Learning in Lunar Science [4.888910677829709]
  We introduce SomBench, a unified, spatially-aligned, ML-ready lunar dataset aggregating 30+ co-registered layers from ten instruments across four missions.<n>An application benchmark suite spans impact processes, volcanic history, and polar volatiles.<n> Baseline experiments with ResNet-50 and SwinV2-B models confirm that each benchmark task is learnable from the released inputs.
  arXiv  Detail & Related papers  (2026-09-08T02:58:32Z)
• Moonstone: A Multimodal Foundation Model and Benchmark for Lunar Remote Sensing [0.0]
  We introduce Moonstone, the first multi-modal foundation model benchmark for lunar remote sensing.<n>Our contributions are: (1) a 28-channel, 128 pixels-per-degree (237 m) global lunar pretraining dataset from seven instrument families across five missions, (2) MG-MAE, a modality-grouped masked autoencoder with per-group convolutional tokenizers, and (3) a benchmark of six downstream tasks covering classification, regression, and segmentation.
  arXiv  Detail & Related papers  (2026-07-03T23:54:52Z)
• LLaVA-LE: Large Language-and-Vision Assistant for Lunar Exploration [5.671312847528642]
  We introduce LLaVA-LE, a vision-language model specialized for lunar surface and subsurface characterization.<n> LUCID dataset consists of 96k high-resolution panchromatic images paired with detailed captions describing lunar terrain.<n>LLaVA-LE achieves a 3.3x overall performance gain over Base LLaVA and 2.1x over our Stage 1 model, with a reasoning score of 1.070.
  arXiv  Detail & Related papers  (2026-03-25T18:17:53Z)
• LunarLoc: Segment-Based Global Localization on the Moon [52.418394025548295]
  Global localization is necessary for autonomous operations on the lunar surface where traditional Earth-based navigation infrastructure, such as GPS, is unavailable.<n>We propose LunarLoc, an approach to global localization that leverages instance segmentation for zero-shot extraction of boulder landmarks from onboard stereo imagery.<n>LunarLoc achieves sub-cm level accuracy in multi-session global localization experiments, significantly outperforming the state of the art in lunar global localization.
  arXiv  Detail & Related papers  (2025-06-20T12:06:47Z)
• TerraFM: A Scalable Foundation Model for Unified Multisensor Earth Observation [65.74990259650984]
  We introduce TerraFM, a scalable self-supervised learning model that leverages globally distributed Sentinel-1 and Sentinel-2 imagery.<n>Our training strategy integrates local-global contrastive learning and introduces a dual-centering mechanism.<n>TerraFM achieves strong generalization on both classification and segmentation tasks, outperforming prior models on GEO-Bench and Copernicus-Bench.
  arXiv  Detail & Related papers  (2025-06-06T17:59:50Z)
• EarthMind: Leveraging Cross-Sensor Data for Advanced Earth Observation Interpretation with a Unified Multimodal LLM [103.7537991413311]
  Earth Observation (EO) data analysis is vital for monitoring environmental and human dynamics.<n>Recent Multimodal Large Language Models (MLLMs) show potential in EO understanding but remain restricted to single-sensor inputs.<n>We propose EarthMind, a unified vision-language framework that handles both single- and cross-sensor inputs.
  arXiv  Detail & Related papers  (2025-06-02T13:36:05Z)
• OmniGeo: Towards a Multimodal Large Language Models for Geospatial Artificial Intelligence [51.0456395687016]
  multimodal large language models (LLMs) have opened new frontiers in artificial intelligence.<n>We propose a MLLM (OmniGeo) tailored to geospatial applications.<n>By combining the strengths of natural language understanding and spatial reasoning, our model enhances the ability of instruction following and the accuracy of GeoAI systems.
  arXiv  Detail & Related papers  (2025-03-20T16:45:48Z)
• EarthView: A Large Scale Remote Sensing Dataset for Self-Supervision [72.84868704100595]
  This paper presents a dataset specifically designed for self-supervision on remote sensing data, intended to enhance deep learning applications on Earth monitoring tasks.<n>The dataset spans 15 tera pixels of global remote-sensing data, combining imagery from a diverse range of sources, including NEON, Sentinel, and a novel release of 1m spatial resolution data from Satellogic.<n>Accompanying the dataset is EarthMAE, a tailored Masked Autoencoder developed to tackle the distinct challenges of remote sensing data.
  arXiv  Detail & Related papers  (2025-01-14T13:42:22Z)
• PolSAM: Polarimetric Scattering Mechanism Informed Segment Anything Model [83.35198885088093]
  PolSAR data presents unique challenges due to its rich and complex characteristics.<n>Existing data representations, such as complex-valued data, polarimetric features, and amplitude images, are widely used.<n>Most feature extraction networks for PolSAR are small, limiting their ability to capture features effectively.<n>We propose the Polarimetric Scattering Mechanism-Informed SAM (PolSAM), an enhanced Segment Anything Model (SAM) that integrates domain-specific scattering characteristics and a novel prompt generation strategy.
  arXiv  Detail & Related papers  (2024-12-17T09:59:53Z)
• LuSNAR:A Lunar Segmentation, Navigation and Reconstruction Dataset based on Muti-sensor for Autonomous Exploration [2.3011380360879237]
  Environmental perception and navigation algorithms are the foundation for lunar rovers. Most of the existing lunar datasets are targeted at a single task. We propose a multi-task, multi-scene, and multi-label lunar benchmark dataset LuSNAR.
  arXiv  Detail & Related papers  (2024-07-09T02:47:58Z)
• Unsupervised Distribution Learning for Lunar Surface Anomaly Detection [0.0]
  We show that modern data-driven machine learning techniques can be successfully applied on lunar surface remote sensing data. In particular we train an unsupervised distribution learning neural network model to find the Apollo 15 landing module.
  arXiv  Detail & Related papers  (2020-01-14T05:38:37Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.