SemAffiNet: Semantic-Affine Transformation for Point Cloud Segmentation
- URL: http://arxiv.org/abs/2205.13490v1
- Date: Thu, 26 May 2022 17:00:23 GMT
- Title: SemAffiNet: Semantic-Affine Transformation for Point Cloud Segmentation
- Authors: Ziyi Wang, Yongming Rao, Xumin Yu, Jie Zhou, Jiwen Lu
- Abstract summary: We propose SemAffiNet for point cloud semantic segmentation.
We conduct extensive experiments on the ScanNetV2 and NYUv2 datasets.
- Score: 94.11915008006483
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Conventional point cloud semantic segmentation methods usually employ an
encoder-decoder architecture, where mid-level features are locally aggregated
to extract geometric information. However, the over-reliance on these
class-agnostic local geometric representations may raise confusion between
local parts from different categories that are similar in appearance or
spatially adjacent. To address this issue, we argue that mid-level features can
be further enhanced with semantic information, and propose semantic-affine
transformation that transforms features of mid-level points belonging to
different categories with class-specific affine parameters. Based on this
technique, we propose SemAffiNet for point cloud semantic segmentation, which
utilizes the attention mechanism in the Transformer module to implicitly and
explicitly capture global structural knowledge within local parts for overall
comprehension of each category. We conduct extensive experiments on the
ScanNetV2 and NYUv2 datasets, and evaluate semantic-affine transformation on
various 3D point cloud and 2D image segmentation baselines, where both
qualitative and quantitative results demonstrate the superiority and
generalization ability of our proposed approach. Code is available at
https://github.com/wangzy22/SemAffiNet.
Related papers
- GSTran: Joint Geometric and Semantic Coherence for Point Cloud Segmentation [33.72549134362884]
We propose GSTran, a novel transformer network tailored for the segmentation task.
The proposed network mainly consists of two principal components: a local geometric transformer and a global semantic transformer.
Experiments on ShapeNetPart and S3DIS benchmarks demonstrate the effectiveness of the proposed method.
arXiv Detail & Related papers (2024-08-21T12:12:37Z) - Exploiting Object-based and Segmentation-based Semantic Features for Deep Learning-based Indoor Scene Classification [0.5572976467442564]
The work described in this paper uses both semantic information, obtained from object detection, and semantic segmentation techniques.
A novel approach that uses a semantic segmentation mask to provide Hu-moments-based segmentation categories' shape characterization, designated by Hu-Moments Features (SHMFs) is proposed.
A three-main-branch network, designated by GOS$2$F$2$App, that exploits deep-learning-based global features, object-based features, and semantic segmentation-based features is also proposed.
arXiv Detail & Related papers (2024-04-11T13:37:51Z) - Geometrically-driven Aggregation for Zero-shot 3D Point Cloud Understanding [11.416392706435415]
Zero-shot 3D point cloud understanding can be achieved via 2D Vision-Language Models (VLMs)
Existing strategies directly map Vision-Language Models from 2D pixels of rendered or captured views to 3D points, overlooking the inherent and expressible point cloud geometric structure.
We introduce the first training-free aggregation technique that leverages the point cloud's 3D geometric structure to improve the quality of the transferred Vision-Language Models.
arXiv Detail & Related papers (2023-12-04T12:30:07Z) - Deep Semantic Graph Matching for Large-scale Outdoor Point Clouds
Registration [22.308070598885532]
We treat the point cloud registration problem as a semantic instance matching and registration task.
We propose a deep semantic graph matching method (DeepSGM) for large-scale outdoor point cloud registration.
Experimental results conducted on the KITTI Odometry dataset demonstrate that the proposed method improves the registration performance.
arXiv Detail & Related papers (2023-08-10T03:07:28Z) - Part-guided Relational Transformers for Fine-grained Visual Recognition [59.20531172172135]
We propose a framework to learn the discriminative part features and explore correlations with a feature transformation module.
Our proposed approach does not rely on additional part branches and reaches state-the-of-art performance on 3-of-the-level object recognition.
arXiv Detail & Related papers (2022-12-28T03:45:56Z) - Framework-agnostic Semantically-aware Global Reasoning for Segmentation [29.69187816377079]
We propose a component that learns to project image features into latent representations and reason between them.
Our design encourages the latent regions to represent semantic concepts by ensuring that the activated regions are spatially disjoint.
Our latent tokens are semantically interpretable and diverse and provide a rich set of features that can be transferred to downstream tasks.
arXiv Detail & Related papers (2022-12-06T21:42:05Z) - GFNet: Geometric Flow Network for 3D Point Cloud Semantic Segmentation [91.15865862160088]
We introduce a geometric flow network (GFNet) to explore the geometric correspondence between different views in an align-before-fuse manner.
Specifically, we devise a novel geometric flow module (GFM) to bidirectionally align and propagate the complementary information across different views.
arXiv Detail & Related papers (2022-07-06T11:48:08Z) - Learning Implicit Feature Alignment Function for Semantic Segmentation [51.36809814890326]
Implicit Feature Alignment function (IFA) is inspired by the rapidly expanding topic of implicit neural representations.
We show that IFA implicitly aligns the feature maps at different levels and is capable of producing segmentation maps in arbitrary resolutions.
Our method can be combined with improvement on various architectures, and it achieves state-of-the-art accuracy trade-off on common benchmarks.
arXiv Detail & Related papers (2022-06-17T09:40:14Z) - Learning to Predict Context-adaptive Convolution for Semantic
Segmentation [66.27139797427147]
Long-range contextual information is essential for achieving high-performance semantic segmentation.
We propose a Context-adaptive Convolution Network (CaC-Net) to predict a spatially-varying feature weighting vector.
Our CaC-Net achieves superior segmentation performance on three public datasets.
arXiv Detail & Related papers (2020-04-17T13:09:17Z) - A Rotation-Invariant Framework for Deep Point Cloud Analysis [132.91915346157018]
We introduce a new low-level purely rotation-invariant representation to replace common 3D Cartesian coordinates as the network inputs.
Also, we present a network architecture to embed these representations into features, encoding local relations between points and their neighbors, and the global shape structure.
We evaluate our method on multiple point cloud analysis tasks, including shape classification, part segmentation, and shape retrieval.
arXiv Detail & Related papers (2020-03-16T14:04:45Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.