Semantic-Aware Implicit Template Learning via Part Deformation Consistency

• URL: http://arxiv.org/abs/2308.11916v1
• Date: Wed, 23 Aug 2023 05:02:17 GMT
• Title: Semantic-Aware Implicit Template Learning via Part Deformation Consistency
• Authors: Sihyeon Kim, Minseok Joo, Jaewon Lee, Juyeon Ko, Juhan Cha, Hyunwoo J. Kim
• Abstract summary: We propose a semantic-aware implicit template learning framework to enable semantically plausible deformation. By leveraging semantic prior from a self-supervised feature extractor, we suggest local conditioning with novel semantic-aware deformation code. Our experiments demonstrate the superiority of the proposed method over baselines in various tasks.
• Score: 18.63665468429503
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Learning implicit templates as neural fields has recently shown impressive performance in unsupervised shape correspondence. Despite the success, we observe current approaches, which solely rely on geometric information, often learn suboptimal deformation across generic object shapes, which have high structural variability. In this paper, we highlight the importance of part deformation consistency and propose a semantic-aware implicit template learning framework to enable semantically plausible deformation. By leveraging semantic prior from a self-supervised feature extractor, we suggest local conditioning with novel semantic-aware deformation code and deformation consistency regularizations regarding part deformation, global deformation, and global scaling. Our extensive experiments demonstrate the superiority of the proposed method over baselines in various tasks: keypoint transfer, part label transfer, and texture transfer. More interestingly, our framework shows a larger performance gain under more challenging settings. We also provide qualitative analyses to validate the effectiveness of semantic-aware deformation. The code is available at https://github.com/mlvlab/PDC.

Related papers

• Flow Factorized Representation Learning [109.51947536586677]
  We introduce a generative model which specifies a distinct set of latent probability paths that define different input transformations. We show that our model achieves higher likelihoods on standard representation learning benchmarks while simultaneously being closer to approximately equivariant models.
  arXiv  Detail & Related papers  (2023-09-22T20:15:37Z)
• Delving into Shape-aware Zero-shot Semantic Segmentation [18.51025849474123]
  We present textbfshape-aware zero-shot semantic segmentation. Inspired by classical spectral methods, we propose to leverage the eigen vectors of Laplacian matrices constructed with self-supervised pixel-wise features. Our method sets new state-of-the-art performance for zero-shot semantic segmentation on both Pascal and COCO.
  arXiv  Detail & Related papers  (2023-04-17T17:59:46Z)
• Learning Context-aware Classifier for Semantic Segmentation [88.88198210948426]
  In this paper, contextual hints are exploited via learning a context-aware classifier. Our method is model-agnostic and can be easily applied to generic segmentation models. With only negligible additional parameters and +2% inference time, decent performance gain has been achieved on both small and large models.
  arXiv  Detail & Related papers  (2023-03-21T07:00:35Z)
• Implicit field supervision for robust non-rigid shape matching [29.7672368261038]
  Establishing a correspondence between two non-rigidly deforming shapes is one of the most fundamental problems in visual computing. We introduce an approach based on auto-decoder framework, that learns a continuous shape-wise deformation field over a fixed template. Our method is remarkably robust in the presence of strong artefacts and can be generalised to arbitrary shape categories.
  arXiv  Detail & Related papers  (2022-03-15T07:22:52Z)
• Bending Graphs: Hierarchical Shape Matching using Gated Optimal Transport [80.64516377977183]
  Shape matching has been a long-studied problem for the computer graphics and vision community. We investigate a hierarchical learning design, to which we incorporate local patch-level information and global shape-level structures. We propose a novel optimal transport solver by recurrently updating features on non-confident nodes to learn globally consistent correspondences between the shapes.
  arXiv  Detail & Related papers  (2022-02-03T11:41:46Z)
• Stochastic Planner-Actor-Critic for Unsupervised Deformable Image Registration [33.72954116727303]
  We present a novel reinforcement learning-based framework that performs step-wise registration of medical images with large deformations. We evaluate our method on several 2D and 3D medical image datasets, some of which contain large deformations.
  arXiv  Detail & Related papers  (2021-12-14T14:08:56Z)
• Unsupervised Dense Deformation Embedding Network for Template-Free Shape Correspondence [18.48814403488283]
  Current deep learning based methods require the supervision of dense annotations to learn per-point translations. We develop a new Unsupervised Deformation Embedding Network (i.e., UD2E-Net), which learns to predict deformations between non-rigid shapes from dense local features. Our UD2E-Net outperforms state-of-the-art unsupervised methods by 24% on Faust Inter challenge and even supervised methods by 13% on Faust Intra challenge.
  arXiv  Detail & Related papers  (2021-08-26T07:07:19Z)
• Augmenting Implicit Neural Shape Representations with Explicit Deformation Fields [95.39603371087921]
  Implicit neural representation is a recent approach to learn shape collections as zero level-sets of neural networks. We advocate deformation-aware regularization for implicit neural representations, aiming at producing plausible deformations as latent code changes.
  arXiv  Detail & Related papers  (2021-08-19T22:07:08Z)
• Context Decoupling Augmentation for Weakly Supervised Semantic Segmentation [53.49821324597837]
  Weakly supervised semantic segmentation is a challenging problem that has been deeply studied in recent years. We present a Context Decoupling Augmentation ( CDA) method to change the inherent context in which the objects appear. To validate the effectiveness of the proposed method, extensive experiments on PASCAL VOC 2012 dataset with several alternative network architectures demonstrate that CDA can boost various popular WSSS methods to the new state-of-the-art by a large margin.
  arXiv  Detail & Related papers  (2021-03-02T15:05:09Z)
• Multiscale Mesh Deformation Component Analysis with Attention-based Autoencoders [49.62443496989065]
  We propose a novel method to exact multiscale deformation components automatically with a stacked attention-based autoencoder. The attention mechanism is designed to learn to softly weight multi-scale deformation components in active deformation regions. With our method, the user can edit shapes in a coarse-to-fine fashion which facilitates effective modeling of new shapes.
  arXiv  Detail & Related papers  (2020-12-04T08:30:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.