Learning Multi-resolution Functional Maps with Spectral Attention for Robust Shape Matching

• URL: http://arxiv.org/abs/2210.06373v1
• Date: Wed, 12 Oct 2022 16:24:53 GMT
• Title: Learning Multi-resolution Functional Maps with Spectral Attention for Robust Shape Matching
• Authors: Lei Li, Nicolas Donati, Maks Ovsjanikov
• Abstract summary: We present a novel non-rigid shape matching framework based on multi-resolution functional maps with spectral attention. Our framework is applicable in both supervised and unsupervised settings. We show that it is possible to train the network so that it can adapt the spectral resolution, depending on the given shape input.
• Score: 38.160024675855496
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In this work, we present a novel non-rigid shape matching framework based on multi-resolution functional maps with spectral attention. Existing functional map learning methods all rely on the critical choice of the spectral resolution hyperparameter, which can severely affect the overall accuracy or lead to overfitting, if not chosen carefully. In this paper, we show that spectral resolution tuning can be alleviated by introducing spectral attention. Our framework is applicable in both supervised and unsupervised settings, and we show that it is possible to train the network so that it can adapt the spectral resolution, depending on the given shape input. More specifically, we propose to compute multi-resolution functional maps that characterize correspondence across a range of spectral resolutions, and introduce a spectral attention network that helps to combine this representation into a single coherent final correspondence. Our approach is not only accurate with near-isometric input, for which a high spectral resolution is typically preferred, but also robust and able to produce reasonable matching even in the presence of significant non-isometric distortion, which poses great challenges to existing methods. We demonstrate the superior performance of our approach through experiments on a suite of challenging near-isometric and non-isometric shape matching benchmarks.

Related papers

• Spectrum-oriented Point-supervised Saliency Detector for Hyperspectral Images [13.79887292039637]
  We introduce point supervision into Hyperspectral salient object detection (HSOD) We incorporate Spectral Saliency, derived from conventional HSOD methods, as a pivotal spectral representation within the framework. We propose a novel pipeline, specifically designed for HSIs, to generate pseudo-labels, effectively mitigating the performance decline associated with point supervision strategy.
  arXiv  Detail & Related papers  (2024-12-24T02:52:43Z)
• Point-Calibrated Spectral Neural Operators [54.13671100638092]
  We introduce Point-Calibrated Spectral Transform, which learns operator mappings by approximating functions with the point-level adaptive spectral basis. Point-Calibrated Spectral Neural Operators learn operator mappings by approximating functions with the point-level adaptive spectral basis.
  arXiv  Detail & Related papers  (2024-10-15T08:19:39Z)
• Hyperspectral Image Reconstruction via Combinatorial Embedding of Cross-Channel Spatio-Spectral Clues [6.580484964018551]
  Existing learning-based hyperspectral reconstruction methods show limitations in fully exploiting the information among the hyperspectral bands. We propose to investigate the inter-dependencies in their respective hyperspectral space. These embedded features can be fully exploited by querying the inter-channel correlations.
  arXiv  Detail & Related papers  (2023-12-18T11:37:19Z)
• SSIF: Learning Continuous Image Representation for Spatial-Spectral Super-Resolution [73.46167948298041]
  We propose a neural implicit model that represents an image as a function of both continuous pixel coordinates in the spatial domain and continuous wavelengths in the spectral domain. We show that SSIF generalizes well to both unseen spatial resolutions and spectral resolutions. It can generate high-resolution images that improve the performance of downstream tasks by 1.7%-7%.
  arXiv  Detail & Related papers  (2023-09-30T15:23:30Z)
• ESSAformer: Efficient Transformer for Hyperspectral Image Super-resolution [76.7408734079706]
  Single hyperspectral image super-resolution (single-HSI-SR) aims to restore a high-resolution hyperspectral image from a low-resolution observation. We propose ESSAformer, an ESSA attention-embedded Transformer network for single-HSI-SR with an iterative refining structure.
  arXiv  Detail & Related papers  (2023-07-26T07:45:14Z)
• Degradation-agnostic Correspondence from Resolution-asymmetric Stereo [96.03964515969652]
  We study the problem of stereo matching from a pair of images with different resolutions, e.g., those acquired with a tele-wide camera system. We propose to impose the consistency between two views in a feature space instead of the image space, named feature-metric consistency. We find that, although a stereo matching network trained with the photometric loss is not optimal, its feature extractor can produce degradation-agnostic and matching-specific features.
  arXiv  Detail & Related papers  (2022-04-04T12:24:34Z)
• Cross-Modality Attentive Feature Fusion for Object Detection in Multispectral Remote Sensing Imagery [0.6853165736531939]
  Cross-modality fusing complementary information of multispectral remote sensing image pairs can improve the perception ability of detection algorithms. We propose a novel and lightweight multispectral feature fusion approach with joint common-modality and differential-modality attentions. Our proposed approach can achieve the state-of-the-art performance at a low cost.
  arXiv  Detail & Related papers  (2021-12-06T13:12:36Z)
• Spectral Splitting and Aggregation Network for Hyperspectral Face Super-Resolution [82.59267937569213]
  High-resolution (HR) hyperspectral face image plays an important role in face related computer vision tasks under uncontrolled conditions. In this paper, we investigate how to adapt the deep learning techniques to hyperspectral face image super-resolution. We present a spectral splitting and aggregation network (SSANet) for HFSR with limited training samples.
  arXiv  Detail & Related papers  (2021-08-31T02:13:00Z)
• There and Back Again: Self-supervised Multispectral Correspondence Estimation [13.56924750612194]
  We introduce a novel cycle-consistency metric that allows us to self-supervise. This, combined with our spectra-agnostic loss functions, allows us to train the same network across multiple spectra. We demonstrate our approach on the challenging task of dense RGB-FIR correspondence estimation.
  arXiv  Detail & Related papers  (2021-03-19T12:33:56Z)
• Instant recovery of shape from spectrum via latent space connections [33.83258865005668]
  We introduce the first learning-based method for recovering shapes from Laplacian spectra. Given an auto-encoder, our model takes the form of a cycle-consistent module to map latent vectors to sequences of eigenvalues. Our data-driven approach replaces the need for ad-hoc regularizers required by prior methods, while providing more accurate results at a fraction of the computational cost.
  arXiv  Detail & Related papers  (2020-03-14T00:48:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.