Sphere2Vec: A General-Purpose Location Representation Learning over a Spherical Surface for Large-Scale Geospatial Predictions

• URL: http://arxiv.org/abs/2306.17624v2
• Date: Mon, 3 Jul 2023 01:26:30 GMT
• Title: Sphere2Vec: A General-Purpose Location Representation Learning over a Spherical Surface for Large-Scale Geospatial Predictions
• Authors: Gengchen Mai, Yao Xuan, Wenyun Zuo, Yutong He, Jiaming Song, Stefano Ermon, Krzysztof Janowicz, and Ni Lao
• Abstract summary: Current 2D and 3D location encoders are designed to model point distances in Euclidean space. We propose a multi-scale location encoder called Sphere2Vec which can preserve spherical distances when encoding point coordinates on a spherical surface.
• Score: 73.60788465154572
• License: http://creativecommons.org/publicdomain/zero/1.0/
• Abstract: Generating learning-friendly representations for points in space is a fundamental and long-standing problem in ML. Recently, multi-scale encoding schemes (such as Space2Vec and NeRF) were proposed to directly encode any point in 2D/3D Euclidean space as a high-dimensional vector, and has been successfully applied to various geospatial prediction and generative tasks. However, all current 2D and 3D location encoders are designed to model point distances in Euclidean space. So when applied to large-scale real-world GPS coordinate datasets, which require distance metric learning on the spherical surface, both types of models can fail due to the map projection distortion problem (2D) and the spherical-to-Euclidean distance approximation error (3D). To solve these problems, we propose a multi-scale location encoder called Sphere2Vec which can preserve spherical distances when encoding point coordinates on a spherical surface. We developed a unified view of distance-reserving encoding on spheres based on the DFS. We also provide theoretical proof that the Sphere2Vec preserves the spherical surface distance between any two points, while existing encoding schemes do not. Experiments on 20 synthetic datasets show that Sphere2Vec can outperform all baseline models on all these datasets with up to 30.8% error rate reduction. We then apply Sphere2Vec to three geo-aware image classification tasks - fine-grained species recognition, Flickr image recognition, and remote sensing image classification. Results on 7 real-world datasets show the superiority of Sphere2Vec over multiple location encoders on all three tasks. Further analysis shows that Sphere2Vec outperforms other location encoder models, especially in the polar regions and data-sparse areas because of its nature for spherical surface distance preservation. Code and data are available at https://gengchenmai.github.io/sphere2vec-website/.

Related papers

• Twin Deformable Point Convolutions for Point Cloud Semantic Segmentation in Remote Sensing Scenes [12.506628755166814]
  We propose novel convolution operators, termed Twin Deformable point Convolutions (TDConvs) These operators aim to achieve adaptive feature learning by learning deformable sampling points in the latitude-longitude plane and altitude direction. Experiments on existing popular benchmarks conclude that our TDConvs achieve the best segmentation performance.
  arXiv  Detail & Related papers  (2024-05-30T06:31:03Z)
• NDC-Scene: Boost Monocular 3D Semantic Scene Completion in Normalized Device Coordinates Space [77.6067460464962]
  Monocular 3D Semantic Scene Completion (SSC) has garnered significant attention in recent years due to its potential to predict complex semantics and geometry shapes from a single image, requiring no 3D inputs. We identify several critical issues in current state-of-the-art methods, including the Feature Ambiguity of projected 2D features in the ray to the 3D space, the Pose Ambiguity of the 3D convolution, and the Imbalance in the 3D convolution across different depth levels. We devise a novel Normalized Device Coordinates scene completion network (NDC-Scene) that directly extends the 2
  arXiv  Detail & Related papers  (2023-09-26T02:09:52Z)
• NeuroGF: A Neural Representation for Fast Geodesic Distance and Path Queries [77.04220651098723]
  This paper presents the first attempt to represent geodesics on 3D mesh models using neural implicit functions. Specifically, we introduce neural geodesic fields (NeuroGFs), which are learned to represent the all-pairs geodesics of a given mesh. NeuroGFs exhibit exceptional performance in solving the single-source all-destination (SSAD) and point-to-point geodesics.
  arXiv  Detail & Related papers  (2023-06-01T13:32:21Z)
• SketchSampler: Sketch-based 3D Reconstruction via View-dependent Depth Sampling [75.957103837167]
  Reconstructing a 3D shape based on a single sketch image is challenging due to the large domain gap between a sparse, irregular sketch and a regular, dense 3D shape. Existing works try to employ the global feature extracted from sketch to directly predict the 3D coordinates, but they usually suffer from losing fine details that are not faithful to the input sketch.
  arXiv  Detail & Related papers  (2022-08-14T16:37:51Z)
• Optimal and Robust Category-level Perception: Object Pose and Shape Estimation from 2D and 3D Semantic Keypoints [24.232254155643574]
  We consider a problem where one is given 2D or 3D sensor data picturing an object of a given category (e.g., a car) and has to reconstruct the 3D pose and shape of the object. Our first contribution is to develop PACE3D* and PACE2D*, the first certifiably optimal solvers for pose and shape estimation. Our second contribution is to developrobust versions of both solvers, named PACE3D# and PACE2D#.
  arXiv  Detail & Related papers  (2022-06-24T21:58:00Z)
• Sphere2Vec: Multi-Scale Representation Learning over a Spherical Surface for Geospatial Predictions [4.754823920235069]
  We propose a multi-scale location encoding model called Sphere2Vec. It directly encodes point coordinates on a spherical surface while avoiding the mapprojection distortion problem. We provide theoretical proof that the Sphere2Vec encoding preserves the spherical surface distance between any two points.
  arXiv  Detail & Related papers  (2022-01-25T17:34:29Z)
• Concentric Spherical GNN for 3D Representation Learning [53.45704095146161]
  We propose a novel multi-resolution convolutional architecture for learning over concentric spherical feature maps. Our hierarchical architecture is based on alternatively learning to incorporate both intra-sphere and inter-sphere information. We demonstrate the effectiveness of our approach in improving state-of-the-art performance on 3D classification tasks with rotated data.
  arXiv  Detail & Related papers  (2021-03-18T19:05:04Z)
• DualConvMesh-Net: Joint Geodesic and Euclidean Convolutions on 3D Meshes [28.571946680616765]
  We propose a family of deep hierarchical convolutional networks over 3D geometric data. The first type, geodesic convolutions, defines the kernel weights over mesh surfaces or graphs. The second type, Euclidean convolutions, is independent of any underlying mesh structure.
  arXiv  Detail & Related papers  (2020-04-02T13:52:00Z)
• PUGeo-Net: A Geometry-centric Network for 3D Point Cloud Upsampling [103.09504572409449]
  We propose a novel deep neural network based method, called PUGeo-Net, to generate uniform dense point clouds. Thanks to its geometry-centric nature, PUGeo-Net works well for both CAD models with sharp features and scanned models with rich geometric details.
  arXiv  Detail & Related papers  (2020-02-24T14:13:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.