LLaVA-LE: Large Language-and-Vision Assistant for Lunar Exploration

• URL: http://arxiv.org/abs/2603.24696v1
• Date: Wed, 25 Mar 2026 18:17:53 GMT
• Title: LLaVA-LE: Large Language-and-Vision Assistant for Lunar Exploration
• Abstract summary: We introduce LLaVA-LE, a vision-language model specialized for lunar surface and subsurface characterization.<n> LUCID dataset consists of 96k high-resolution panchromatic images paired with detailed captions describing lunar terrain.<n>LLaVA-LE achieves a 3.3x overall performance gain over Base LLaVA and 2.1x over our Stage 1 model, with a reasoning score of 1.070.
• Score: 5.671312847528642
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Recent advances in multimodal vision-language models (VLMs) have enabled joint reasoning over visual and textual information, yet their application to planetary science remains largely unexplored. A key hindrance is the absence of large-scale datasets that pair real planetary imagery with detailed scientific descriptions. In this work, we introduce LLaVA-LE (Large Language-and-Vision Assistant for Lunar Exploration), a vision-language model specialized for lunar surface and subsurface characterization. To enable this capability, we curate a new large-scale multimodal lunar dataset, LUCID (LUnar Caption Image Dataset) consisting of 96k high-resolution panchromatic images paired with detailed captions describing lunar terrain characteristics, and 81k question-answer (QA) pairs derived from approximately 20k images in the LUCID dataset. Leveraging this dataset, we fine-tune LLaVA using a two-stage training curriculum: (1) concept alignment for domain-specific terrain description, and (2) instruction-tuned visual question answering. We further design evaluation benchmarks spanning multiple levels of reasoning complexity relevant to lunar terrain analysis. Evaluated against GPT and Gemini judges, LLaVA-LE achieves a 3.3x overall performance gain over Base LLaVA and 2.1x over our Stage 1 model, with a reasoning score of 1.070, exceeding the judge's own reference score, highlighting the effectiveness of domain-specific multimodal data and instruction tuning to advance VLMs in planetary exploration. Code is available at https://github.com/OSUPCVLab/LLaVA-LE.

Related papers

• SomBench: Benchmark Dataset for Advancing Machine Learning in Lunar Science [4.888910677829709]
  We introduce SomBench, a unified, spatially-aligned, ML-ready lunar dataset aggregating 30+ co-registered layers from ten instruments across four missions.<n>An application benchmark suite spans impact processes, volcanic history, and polar volatiles.<n> Baseline experiments with ResNet-50 and SwinV2-B models confirm that each benchmark task is learnable from the released inputs.
  arXiv  Detail & Related papers  (2026-09-08T02:58:32Z)
• LunarFM: A Shared Multimodal Representation of the Moon's Surface [3.029669084751708]
  We introduce LunarFM, a model that learns a general representation of the lunar surface from diverse orbital measurements.<n>LunarFM assimilates observations from six instruments across three lunar missions, mapping 18 input channels to a shared embedding space.<n>We demonstrate that this embedding space supports a diverse range of downstream applications, including similarity search, few-shot resource mapping, mineral abundance regression, and geological unit classification.
  arXiv  Detail & Related papers  (2026-07-24T15:25:20Z)
• TerraScope: Pixel-Grounded Visual Reasoning for Earth Observation [71.39112735716172]
  We introduce TerraScope, a unified vision-language model that delivers pixel-grounded geospatial reasoning.<n>We also propose TerraScope-Bench, the first benchmark for pixel-grounded geospatial reasoning.
  arXiv  Detail & Related papers  (2026-03-19T15:38:02Z)
• UAVBench and UAVIT-1M: Benchmarking and Enhancing MLLMs for Low-Altitude UAV Vision-Language Understanding [4.817647738745087]
  UAVBench and UAVIT-1M are designed to evaluate and improve MLLMs' abilities in low-altitude vision-language tasks.<n> UAVBench comprises 43 test units and 966k high-quality data samples across 10 tasks at the image-level and region-level.<n> UAVIT-1M consists of approximately 1.24 million diverse instructions, covering 789k multi-scene images and about 2,000 types of spatial resolutions with 11 distinct tasks.
  arXiv  Detail & Related papers  (2026-03-15T12:04:23Z)
• SD-VLM: Spatial Measuring and Understanding with Depth-Encoded Vision-Language Models [75.64836077468722]
  Vision language models (VLMs) excel in 2D semantic visual understanding, but their ability to quantitatively reason about 3D spatial relationships remains under-explored.<n>We propose SD-VLM, a novel framework that significantly enhances fundamental spatial perception abilities of VLMs.<n>We have trained SD-VLM, a strong generalist VLM which shows superior quantitative spatial measuring and understanding capability.
  arXiv  Detail & Related papers  (2025-09-22T12:08:12Z)
• InternSpatial: A Comprehensive Dataset for Spatial Reasoning in Vision-Language Models [59.7084864920244]
  InternSpatial is the largest open-source dataset for spatial reasoning in vision-language models (VLMs)<n> InternSpatial comprises 12 million QA pairs spanning both single-view and multi-view settings.<n> InternSpatial-Bench is a corresponding evaluation benchmark designed to assess spatial understanding under diverse instruction formats.
  arXiv  Detail & Related papers  (2025-06-23T08:17:22Z)
• Benchmarking Large Vision-Language Models on Fine-Grained Image Tasks: A Comprehensive Evaluation [53.84282335629258]
  We introduce a comprehensive fine-grained evaluation benchmark, i.e., FG-BMK, comprising 1.01 million questions and 0.33 million images.<n>Our evaluation systematically examines LVLMs from both human-oriented and machine-oriented perspectives.<n>We uncover key findings regarding the influence of training paradigms, modality alignment, perturbation susceptibility, and fine-grained category reasoning on task performance.
  arXiv  Detail & Related papers  (2025-04-21T09:30:41Z)
• Open3D-VQA: A Benchmark for Comprehensive Spatial Reasoning with Multimodal Large Language Model in Open Space [38.482463743451625]
  We present Open3D-VQA, a novel benchmark for evaluating MLLMs' ability to reason about complex spatial relationships from an aerial perspective.<n>The benchmark comprises 73k QA pairs spanning 7 general spatial reasoning tasks, including multiple-choice, true/false, and short-answer formats.
  arXiv  Detail & Related papers  (2025-03-14T05:35:38Z)
• Exploring Spatial Representation to Enhance LLM Reasoning in Aerial Vision-Language Navigation [11.267956604072845]
  Aerial Vision-and-Language Navigation (VLN) is a novel task enabling Unmanned Aerial Vehicles (UAVs) to navigate in outdoor environments through natural language instructions and visual cues.<n>We propose a training-free, zero-shot framework for aerial VLN tasks, where the large language model (LLM) is leveraged as the agent for action prediction.
  arXiv  Detail & Related papers  (2024-10-11T03:54:48Z)
• LuSNAR:A Lunar Segmentation, Navigation and Reconstruction Dataset based on Muti-sensor for Autonomous Exploration [2.3011380360879237]
  Environmental perception and navigation algorithms are the foundation for lunar rovers. Most of the existing lunar datasets are targeted at a single task. We propose a multi-task, multi-scene, and multi-label lunar benchmark dataset LuSNAR.
  arXiv  Detail & Related papers  (2024-07-09T02:47:58Z)
• Towards Vision-Language Geo-Foundation Model: A Survey [65.70547895998541]
  Vision-Language Foundation Models (VLFMs) have made remarkable progress on various multimodal tasks. This paper thoroughly reviews VLGFMs, summarizing and analyzing recent developments in the field.
  arXiv  Detail & Related papers  (2024-06-13T17:57:30Z)
• Recognize Any Regions [55.76437190434433]
  RegionSpot integrates position-aware localization knowledge from a localization foundation model with semantic information from a ViL model.<n>Experiments in open-world object recognition show that our RegionSpot achieves significant performance gain over prior alternatives.
  arXiv  Detail & Related papers  (2023-11-02T16:31:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.