S$^{5}$Mars: Semi-Supervised Learning for Mars Semantic Segmentation

• URL: http://arxiv.org/abs/2207.01200v4
• Date: Mon, 8 Apr 2024 01:11:22 GMT
• Title: S$^{5}$Mars: Semi-Supervised Learning for Mars Semantic Segmentation
• Authors: Jiahang Zhang, Lilang Lin, Zejia Fan, Wenjing Wang, Jiaying Liu,
• Abstract summary: Mars semantic segmentation is an important Martian vision task, which is the base of rover autonomous planning and safe driving. There is a lack of sufficient detailed and high-confidence data annotations, which are exactly required by most deep learning methods to obtain a good model. We propose our solution from the perspective of joint data and method design. Experimental results show that our method can outperform state-of-the-art SSL approaches remarkably.
• Score: 18.92602724896845
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Deep learning has become a powerful tool for Mars exploration. Mars terrain semantic segmentation is an important Martian vision task, which is the base of rover autonomous planning and safe driving. However, there is a lack of sufficient detailed and high-confidence data annotations, which are exactly required by most deep learning methods to obtain a good model. To address this problem, we propose our solution from the perspective of joint data and method design. We first present a newdataset S5Mars for Semi-SuperviSed learning on Mars Semantic Segmentation, which contains 6K high-resolution images and is sparsely annotated based on confidence, ensuring the high quality of labels. Then to learn from this sparse data, we propose a semi-supervised learning (SSL) framework for Mars image semantic segmentation, to learn representations from limited labeled data. Different from the existing SSL methods which are mostly targeted at the Earth image data, our method takes into account Mars data characteristics. Specifically, we first investigate the impact of current widely used natural image augmentations on Mars images. Based on the analysis, we then proposed two novel and effective augmentations for SSL of Mars segmentation, AugIN and SAM-Mix, which serve as strong augmentations to boost the model performance. Meanwhile, to fully leverage the unlabeled data, we introduce a soft-to-hard consistency learning strategy, learning from different targets based on prediction confidence. Experimental results show that our method can outperform state-of-the-art SSL approaches remarkably. Our proposed dataset is available at https://jhang2020.github.io/S5Mars.github.io/.

Related papers

• MarsSeg: Mars Surface Semantic Segmentation with Multi-level Extractor and Connector [19.053126804261034]
  We propose a novel encoder-decoder based Mars segmentation network, termed MarsSeg. The Mini-ASPP and PSA are specifically designed for shadow feature enhancement. The SPPM is employed for deep feature enhancement, facilitating the extraction of high-level semantic category-related information.
  arXiv  Detail & Related papers  (2024-04-05T15:04:57Z)
• Noise2Noise Denoising of CRISM Hyperspectral Data [6.502987568800912]
  Noise2Noise4Mars (N2N4M) is introduced to remove noise from CRISM images. Our model is self-supervised and does not require zero-noise target data. This allows for detailed analysis for critical sites of interest on the Martian surface, including proposed lander sites.
  arXiv  Detail & Related papers  (2024-03-26T14:49:22Z)
• SatSynth: Augmenting Image-Mask Pairs through Diffusion Models for Aerial Semantic Segmentation [69.42764583465508]
  We explore the potential of generative image diffusion to address the scarcity of annotated data in earth observation tasks. To the best of our knowledge, we are the first to generate both images and corresponding masks for satellite segmentation.
  arXiv  Detail & Related papers  (2024-03-25T10:30:22Z)
• ConeQuest: A Benchmark for Cone Segmentation on Mars [9.036303895516745]
  ConeQuest is the first expert-annotated public dataset to identify cones on Mars. We propose two benchmark tasks using ConeQuest: (i) Spatial Generalization and (ii) Cone-size Generalization.
  arXiv  Detail & Related papers  (2023-11-15T02:33:08Z)
• Delving Deeper into Data Scaling in Masked Image Modeling [145.36501330782357]
  We conduct an empirical study on the scaling capability of masked image modeling (MIM) methods for visual recognition. Specifically, we utilize the web-collected Coyo-700M dataset. Our goal is to investigate how the performance changes on downstream tasks when scaling with different sizes of data and models.
  arXiv  Detail & Related papers  (2023-05-24T15:33:46Z)
• Improving Contrastive Learning on Visually Homogeneous Mars Rover Images [3.206547922373737]
  We show how contrastive learning can be applied to hundreds of thousands of unlabeled Mars terrain images. Contrastive learning assumes that any given pair of distinct images contain distinct semantic content. We propose two approaches to resolve this: 1) an unsupervised deep clustering step on the Mars datasets, which identifies clusters of images containing similar semantic content and corrects false negative errors during training, and 2) a simple approach which mixes data from different domains to increase visual diversity of the total training dataset.
  arXiv  Detail & Related papers  (2022-10-17T16:26:56Z)
• Towards Realistic Semi-Supervised Learning [73.59557447798134]
  We propose a novel approach to tackle SSL in open-world setting, where we simultaneously learn to classify known and unknown classes. Our approach substantially outperforms the existing state-of-the-art on seven diverse datasets.
  arXiv  Detail & Related papers  (2022-07-05T19:04:43Z)
• Semi-Supervised Learning for Mars Imagery Classification and Segmentation [35.103989798891476]
  We introduce a semi-supervised framework for machine vision on Mars. We try to resolve two specific tasks: classification and segmentation. Our learning strategies can improve the classification and segmentation models by a large margin and outperform state-of-the-art approaches.
  arXiv  Detail & Related papers  (2022-06-05T13:55:10Z)
• Embedding Earth: Self-supervised contrastive pre-training for dense land cover classification [61.44538721707377]
  We present Embedding Earth a self-supervised contrastive pre-training method for leveraging the large availability of satellite imagery. We observe significant improvements up to 25% absolute mIoU when pre-trained with our proposed method. We find that learnt features can generalize between disparate regions opening up the possibility of using the proposed pre-training scheme.
  arXiv  Detail & Related papers  (2022-03-11T16:14:14Z)
• Towards Robust Monocular Visual Odometry for Flying Robots on Planetary Missions [49.79068659889639]
  Ingenuity, that just landed on Mars, will mark the beginning of a new era of exploration unhindered by traversability. We present an advanced robust monocular odometry algorithm that uses efficient optical flow tracking. We also present a novel approach to estimate the current risk of scale drift based on a principal component analysis of the relative translation information matrix.
  arXiv  Detail & Related papers  (2021-09-12T12:52:20Z)
• Large-scale Unsupervised Semantic Segmentation [163.3568726730319]
  We propose a new problem of large-scale unsupervised semantic segmentation (LUSS) with a newly created benchmark dataset to track the research progress. Based on the ImageNet dataset, we propose the ImageNet-S dataset with 1.2 million training images and 40k high-quality semantic segmentation annotations for evaluation.
  arXiv  Detail & Related papers  (2021-06-06T15:02:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.