Masked Angle-Aware Autoencoder for Remote Sensing Images

• URL: http://arxiv.org/abs/2408.01946v1
• Date: Sun, 4 Aug 2024 07:12:59 GMT
• Title: Masked Angle-Aware Autoencoder for Remote Sensing Images
• Abstract summary: This paper proposes the Masked Angle-Aware Autoencoder (MA3E) to perceive and learn angles during pre-training. We show that MA3E demonstrates more competitive performance than existing pre-training methods on seven different RS image datasets in three downstream tasks.
• Score: 43.65954772044253
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: To overcome the inherent domain gap between remote sensing (RS) images and natural images, some self-supervised representation learning methods have made promising progress. However, they have overlooked the diverse angles present in RS objects. This paper proposes the Masked Angle-Aware Autoencoder (MA3E) to perceive and learn angles during pre-training. We design a \textit{scaling center crop} operation to create the rotated crop with random orientation on each original image, introducing the explicit angle variation. MA3E inputs this composite image while reconstruct the original image, aiming to effectively learn rotation-invariant representations by restoring the angle variation introduced on the rotated crop. To avoid biases caused by directly reconstructing the rotated crop, we propose an Optimal Transport (OT) loss that automatically assigns similar original image patches to each rotated crop patch for reconstruction. MA3E demonstrates more competitive performance than existing pre-training methods on seven different RS image datasets in three downstream tasks.

Related papers

• OmniPoint: Universal Monocular Metric Pointcloud from Any Camera [79.24417597712802]
  We present OmniPoint, a unified framework designed to generalize metric reconstruction across diverse imaging sensors.<n>To overcome projection rigidity, OmniPoint abandons conventional planar depth regression.<n>It instead adopts a decoupled ray and distance representation alongside a decoupled training objective.
  arXiv  Detail & Related papers  (2026-09-08T19:45:58Z)
• Filling the Unseen: Scene Extrapolation via 3D Gaussian Splatting [10.582831709592789]
  3D Gaussian Splatting reconstruction within training view distribution degrades on out-of-distribution novel views, exhibiting holes in unobserved regions and artifacts in observable areas.<n>Recent works formulate this task as extrapolation and conditioning and try to address it with generative models, but remain limited in extrapolation scale and quality.<n>We propose a holistic framework for extrapolation and conditioning, with an independent camera view detection mechanism to enable parallel conflict-free extrapolation.
  arXiv  Detail & Related papers  (2026-09-06T19:02:27Z)
• Visual Geometry Transformer in the Wild: Distractor-Free 3D Reconstruction [77.75911117889915]
  Current end-to-end multi-view 3D reconstruction methods achieve impressive results, but rely on a restrictive static assumption.<n>This reliance on idealized inputs causes even the most advanced methods to fail in real-world settings.<n>We propose Visual Geometry Transformer in the Wild, an end-to-end framework for robust reconstruction from inconsistent views.
  arXiv  Detail & Related papers  (2026-06-22T02:52:22Z)
• Seg-VAR: Image Segmentation with Visual Autoregressive Modeling [60.79579744943664]
  We propose a novel framework that rethinks segmentation as a conditional autoregressive mask generation problem.<n>This is achieved by replacing the discriminative learning with the latent learning process.<n>Our method incorporates three core components: (1) an image encoder generating latent priors from input images, (2) a spatial-aware seglat (a latent expression of segmentation mask) encoder that maps segmentation masks into discrete latent tokens, and (3) a decoder reconstructing masks from these latents.
  arXiv  Detail & Related papers  (2025-11-16T13:36:19Z)
• Towards Rotation-only Imaging Geometry: Rotation Estimation [11.806182001858454]
  Structure from Motion (SfM) is a critical task in computer vision, aiming to recover the 3D scene structure and camera motion from a sequence of 2D images.<n>Recent pose-only imaging geometry decouples 3D coordinates from camera poses and demonstrates significantly better SfM performance through pose adjustment.
  arXiv  Detail & Related papers  (2025-11-16T02:04:32Z)
• Discovering an Image-Adaptive Coordinate System for Photography Processing [51.164345878060956]
  We propose a novel algorithm, IAC, to learn an image-adaptive coordinate system in the RGB color space before performing curve operations.<n>This end-to-end trainable approach enables us to efficiently adjust images with a jointly learned image-adaptive coordinate system and curves.
  arXiv  Detail & Related papers  (2025-01-11T06:20:07Z)
• RI-MAE: Rotation-Invariant Masked AutoEncoders for Self-Supervised Point Cloud Representation Learning [24.532566288697684]
  Masked point modeling methods have recently achieved great success in self-supervised learning for point cloud data. These methods are sensitive to rotations and often exhibit sharp performance drops when encountering rotational variations. We propose a novel Rotation-Invariant Masked AutoEncoders (RI-MAE) to address two major challenges.
  arXiv  Detail & Related papers  (2024-08-31T05:17:55Z)
• 2L3: Lifting Imperfect Generated 2D Images into Accurate 3D [16.66666619143761]
  Multi-view (MV) 3D reconstruction is a promising solution to fuse generated MV images into consistent 3D objects. However, the generated images usually suffer from inconsistent lighting, misaligned geometry, and sparse views, leading to poor reconstruction quality. We present a novel 3D reconstruction framework that leverages intrinsic decomposition guidance, transient-mono prior guidance, and view augmentation to cope with the three issues.
  arXiv  Detail & Related papers  (2024-01-29T02:30:31Z)
• Feature Guided Masked Autoencoder for Self-supervised Learning in Remote Sensing [16.683132793313693]
  Masked AutoEncoder (MAE) has attracted wide attention for pretraining vision transformers in remote sensing. We propose Feature Guided Masked Autoencoder (FG-MAE): reconstructing a combination of Histograms of Oriented Graidents (HOG) and Normalized Difference Indices (NDI) for multispectral images, and reconstructing HOG for SAR images.
  arXiv  Detail & Related papers  (2023-10-28T09:43:13Z)
• RelPose++: Recovering 6D Poses from Sparse-view Observations [66.6922660401558]
  We address the task of estimating 6D camera poses from sparse-view image sets (2-8 images) We build on the recent RelPose framework which learns a network that infers distributions over relative rotations over image pairs. Our final system results in large improvements in 6D pose prediction over prior art on both seen and unseen object categories.
  arXiv  Detail & Related papers  (2023-05-08T17:59:58Z)
• An End-to-End Network for Upright Adjustment of Panoramic Images [4.8866383302403955]
  We propose an online end-to-end network for upright adjustment. Our network consists of three modules: orientation estimation, LUT online generation, and upright reconstruction. In terms of image reconstruction, we have achieved the first real-time online upright reconstruction of panoramic images using deep learning networks.
  arXiv  Detail & Related papers  (2023-04-12T01:31:46Z)
• Unsupervised Multi-View Object Segmentation Using Radiance Field Propagation [55.9577535403381]
  We present a novel approach to segmenting objects in 3D during reconstruction given only unlabeled multi-view images of a scene. The core of our method is a novel propagation strategy for individual objects' radiance fields with a bidirectional photometric loss. To the best of our knowledge, RFP is the first unsupervised approach for tackling 3D scene object segmentation for neural radiance field (NeRF)
  arXiv  Detail & Related papers  (2022-10-02T11:14:23Z)
• Attentive Symmetric Autoencoder for Brain MRI Segmentation [56.02577247523737]
  We propose a novel Attentive Symmetric Auto-encoder based on Vision Transformer (ViT) for 3D brain MRI segmentation tasks. In the pre-training stage, the proposed auto-encoder pays more attention to reconstruct the informative patches according to the gradient metrics. Experimental results show that our proposed attentive symmetric auto-encoder outperforms the state-of-the-art self-supervised learning methods and medical image segmentation models.
  arXiv  Detail & Related papers  (2022-09-19T09:43:19Z)
• Deep Rotation Correction without Angle Prior [57.76737888499145]
  We propose a new and practical task, named Rotation Correction, to automatically correct the tilt with high content fidelity. This task can be easily integrated into image editing applications, allowing users to correct the rotated images without any manual operations. We leverage a neural network to predict the optical flows that can warp the tilted images to be perceptually horizontal.
  arXiv  Detail & Related papers  (2022-07-07T02:46:27Z)
• Recursive Self-Improvement for Camera Image and Signal Processing Pipeline [6.318974730864278]
  Current camera image and signal processing pipelines (ISPs) tend to apply a single filter that is uniformly applied to the entire image. This despite the fact that most acquired camera images have spatially heterogeneous artifacts. We present a deep reinforcement learning model that works in learned latent subspaces.
  arXiv  Detail & Related papers  (2021-11-15T02:23:40Z)
• In-Domain GAN Inversion for Real Image Editing [56.924323432048304]
  A common practice of feeding a real image to a trained GAN generator is to invert it back to a latent code. Existing inversion methods typically focus on reconstructing the target image by pixel values yet fail to land the inverted code in the semantic domain of the original latent space. We propose an in-domain GAN inversion approach, which faithfully reconstructs the input image and ensures the inverted code to be semantically meaningful for editing.
  arXiv  Detail & Related papers  (2020-03-31T18:20:18Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.