Inpainting the Red Planet: Diffusion Models for the Reconstruction of Martian Environments in Virtual Reality

• URL: http://arxiv.org/abs/2510.14765v1
• Date: Thu, 16 Oct 2025 15:02:05 GMT
• Title: Inpainting the Red Planet: Diffusion Models for the Reconstruction of Martian Environments in Virtual Reality
• Authors: Giuseppe Lorenzo Catalano, Agata Marta Soccini,
• Abstract summary: Training was conducted on an augmented dataset of 12000 Martian heightmaps derived from NASA's HiRISE survey.<n>A non-homogeneous rescaling strategy captures terrain features across multiple scales before resizing to a fixed 128x128 model resolution.<n>Results show that our approach consistently outperforms these methods in terms of reconstruction accuracy (4-15% on RMSE) and perceptual similarity (29-81% on LPIPS) with the original data.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Space exploration increasingly relies on Virtual Reality for several tasks, such as mission planning, multidisciplinary scientific analysis, and astronaut training. A key factor for the reliability of the simulations is having accurate 3D representations of planetary terrains. Extraterrestrial heightmaps derived from satellite imagery often contain missing values due to acquisition and transmission constraints. Mars is among the most studied planets beyond Earth, and its extensive terrain datasets make the Martian surface reconstruction a valuable task, although many areas remain unmapped. Deep learning algorithms can support void-filling tasks; however, whereas Earth's comprehensive datasets enables the use of conditional methods, such approaches cannot be applied to Mars. Current approaches rely on simpler interpolation techniques which, however, often fail to preserve geometric coherence. In this work, we propose a method for reconstructing the surface of Mars based on an unconditional diffusion model. Training was conducted on an augmented dataset of 12000 Martian heightmaps derived from NASA's HiRISE survey. A non-homogeneous rescaling strategy captures terrain features across multiple scales before resizing to a fixed 128x128 model resolution. We compared our method against established void-filling and inpainting techniques, including Inverse Distance Weighting, kriging, and Navier-Stokes algorithm, on an evaluation set of 1000 samples. Results show that our approach consistently outperforms these methods in terms of reconstruction accuracy (4-15% on RMSE) and perceptual similarity (29-81% on LPIPS) with the original data.

Related papers

• Martian World Models: Controllable Video Synthesis with Physically Accurate 3D Reconstructions [116.56517155163716]
  We propose a data curation pipeline that reconstructs 3D Martian environments from real stereo navigation images.<n>A Martian terrain video generator, MarsGen, synthesizes novel videos visually realistic and geometrically consistent with the 3D structure encoded in the data.<n>Our approach outperforms video synthesis models trained on terrestrial datasets, achieving superior visual fidelity and 3D structural consistency.
  arXiv  Detail & Related papers  (2025-07-10T17:54:27Z)
• M3Depth: Wavelet-Enhanced Depth Estimation on Mars via Mutual Boosting of Dual-Modal Data [16.951488779261343]
  We propose M3Depth, a depth estimation model tailored for Mars rovers.<n>Considering the sparse and smooth texture of Martian terrain, our model incorporates a convolutional kernel based on wavelet transform.<n>M3Depth achieves a 16% improvement in depth estimation accuracy compared to other state-of-the-art methods in depth estimation.
  arXiv  Detail & Related papers  (2025-05-20T10:13:00Z)
• AerialMegaDepth: Learning Aerial-Ground Reconstruction and View Synthesis [57.249817395828174]
  We propose a scalable framework combining pseudo-synthetic renderings from 3D city-wide meshes with real, ground-level crowd-sourced images.<n>The pseudo-synthetic data simulates a wide range of aerial viewpoints, while the real, crowd-sourced images help improve visual fidelity for ground-level images.<n>Using this hybrid dataset, we fine-tune several state-of-the-art algorithms and achieve significant improvements on real-world, zero-shot aerial-ground tasks.
  arXiv  Detail & Related papers  (2025-04-17T17:57:05Z)
• Vision-based Geo-Localization of Future Mars Rotorcraft in Challenging Illumination Conditions [4.6901215692204286]
  Geo-LoFTR is a geometry-aided deep learning model for image registration that is more robust under large illumination differences than prior models.<n>We show that our proposed system outperforms prior MbL efforts in terms of localization accuracy under significant lighting and scale variations.
  arXiv  Detail & Related papers  (2025-02-13T22:10:21Z)
• Structure-Invariant Range-Visual-Inertial Odometry [17.47284320862407]
  This work introduces a novel range-visual-inertial odometry system tailored for the Mars Science Helicopter mission. Our system extends the state-of-the-art xVIO framework by fusing consistent range information with visual and inertial measurements. We demonstrate that our range-VIO approach estimates terrain-relative velocity meeting the stringent mission requirements.
  arXiv  Detail & Related papers  (2024-09-06T21:49:10Z)
• MaRF: Representing Mars as Neural Radiance Fields [1.4680035572775534]
  MaRF is a framework able to synthesize the Martian environment using several collections of images from rover cameras. It addresses key challenges in planetary surface exploration such as: planetary geology, simulated navigation and shape analysis. In the experimental section, we demonstrate the environments created from actual Mars datasets captured by Curiosity rover, Perseverance rover and Ingenuity helicopter.
  arXiv  Detail & Related papers  (2022-12-03T18:58:00Z)
• 6D Camera Relocalization in Visually Ambiguous Extreme Environments [79.68352435957266]
  We propose a novel method to reliably estimate the pose of a camera given a sequence of images acquired in extreme environments such as deep seas or extraterrestrial terrains. Our method achieves comparable performance with state-of-the-art methods on the indoor benchmark (7-Scenes dataset) using only 20% training data.
  arXiv  Detail & Related papers  (2022-07-13T16:40:02Z)
• Embedding Earth: Self-supervised contrastive pre-training for dense land cover classification [61.44538721707377]
  We present Embedding Earth a self-supervised contrastive pre-training method for leveraging the large availability of satellite imagery. We observe significant improvements up to 25% absolute mIoU when pre-trained with our proposed method. We find that learnt features can generalize between disparate regions opening up the possibility of using the proposed pre-training scheme.
  arXiv  Detail & Related papers  (2022-03-11T16:14:14Z)
• Towards Robust Monocular Visual Odometry for Flying Robots on Planetary Missions [49.79068659889639]
  Ingenuity, that just landed on Mars, will mark the beginning of a new era of exploration unhindered by traversability. We present an advanced robust monocular odometry algorithm that uses efficient optical flow tracking. We also present a novel approach to estimate the current risk of scale drift based on a principal component analysis of the relative translation information matrix.
  arXiv  Detail & Related papers  (2021-09-12T12:52:20Z)
• Latent World Models For Intrinsically Motivated Exploration [140.21871701134626]
  We present a self-supervised representation learning method for image-based observations. We consider episodic and life-long uncertainties to guide the exploration of partially observable environments.
  arXiv  Detail & Related papers  (2020-10-05T19:47:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.