Learning and Memorizing Representative Prototypes for 3D Point Cloud
Semantic and Instance Segmentation
- URL: http://arxiv.org/abs/2001.01349v1
- Date: Mon, 6 Jan 2020 01:07:46 GMT
- Title: Learning and Memorizing Representative Prototypes for 3D Point Cloud
Semantic and Instance Segmentation
- Authors: Tong He and Dong Gong and Zhi Tian and Chunhua Shen
- Abstract summary: 3D point cloud semantic and instance segmentation is crucial and fundamental for 3D scene understanding.
Deep networks can easily forget the non-dominant cases during the learning process, resulting in unsatisfactory performance.
We propose a memory-augmented network to learn and memorize the representative prototypes that cover diverse samples universally.
- Score: 117.29799759864127
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: 3D point cloud semantic and instance segmentation is crucial and fundamental
for 3D scene understanding. Due to the complex structure, point sets are
distributed off balance and diversely, which appears as both category imbalance
and pattern imbalance. As a result, deep networks can easily forget the
non-dominant cases during the learning process, resulting in unsatisfactory
performance. Although re-weighting can reduce the influence of the
well-classified examples, they cannot handle the non-dominant patterns during
the dynamic training. In this paper, we propose a memory-augmented network to
learn and memorize the representative prototypes that cover diverse samples
universally. Specifically, a memory module is introduced to alleviate the
forgetting issue by recording the patterns seen in mini-batch training. The
learned memory items consistently reflect the interpretable and meaningful
information for both dominant and non-dominant categories and cases. The
distorted observations and rare cases can thus be augmented by retrieving the
stored prototypes, leading to better performances and generalization.
Exhaustive experiments on the benchmarks, i.e. S3DIS and ScanNetV2, reflect the
superiority of our method on both effectiveness and efficiency. Not only the
overall accuracy but also nondominant classes have improved substantially.
Related papers
- Benchmarking Hebbian learning rules for associative memory [0.0]
Associative memory is a key concept in cognitive and computational brain science.
We benchmark six different learning rules on storage capacity and prototype extraction.
arXiv Detail & Related papers (2023-12-30T21:49:47Z) - Gaussian Switch Sampling: A Second Order Approach to Active Learning [11.775252660867285]
In active learning, acquisition functions define informativeness directly on the representation position within the model manifold.
We propose a grounded second-order definition of information content and sample importance within the context of active learning.
We show that our definition produces highly accurate importance scores even when the model representations are constrained by the lack of training data.
arXiv Detail & Related papers (2023-02-16T15:24:56Z) - Learning sparse features can lead to overfitting in neural networks [9.2104922520782]
We show that feature learning can perform worse than lazy training.
Although sparsity is known to be essential for learning anisotropic data, it is detrimental when the target function is constant or smooth.
arXiv Detail & Related papers (2022-06-24T14:26:33Z) - Memory-Guided Semantic Learning Network for Temporal Sentence Grounding [55.31041933103645]
We propose a memory-augmented network that learns and memorizes the rarely appeared content in TSG tasks.
MGSL-Net consists of three main parts: a cross-modal inter-action module, a memory augmentation module, and a heterogeneous attention module.
arXiv Detail & Related papers (2022-01-03T02:32:06Z) - Unsupervised Representation Learning for 3D Point Cloud Data [66.92077180228634]
We propose a simple yet effective approach for unsupervised point cloud learning.
In particular, we identify a very useful transformation which generates a good contrastive version of an original point cloud.
We conduct experiments on three downstream tasks which are 3D object classification, shape part segmentation and scene segmentation.
arXiv Detail & Related papers (2021-10-13T10:52:45Z) - Point Discriminative Learning for Unsupervised Representation Learning
on 3D Point Clouds [54.31515001741987]
We propose a point discriminative learning method for unsupervised representation learning on 3D point clouds.
We achieve this by imposing a novel point discrimination loss on the middle level and global level point features.
Our method learns powerful representations and achieves new state-of-the-art performance.
arXiv Detail & Related papers (2021-08-04T15:11:48Z) - Rethinking Deep Contrastive Learning with Embedding Memory [58.66613563148031]
Pair-wise loss functions have been extensively studied and shown to continuously improve the performance of deep metric learning (DML)
We provide a new methodology for systematically studying weighting strategies of various pair-wise loss functions, and rethink pair weighting with an embedding memory.
arXiv Detail & Related papers (2021-03-25T17:39:34Z) - Learning What Makes a Difference from Counterfactual Examples and
Gradient Supervision [57.14468881854616]
We propose an auxiliary training objective that improves the generalization capabilities of neural networks.
We use pairs of minimally-different examples with different labels, a.k.a counterfactual or contrasting examples, which provide a signal indicative of the underlying causal structure of the task.
Models trained with this technique demonstrate improved performance on out-of-distribution test sets.
arXiv Detail & Related papers (2020-04-20T02:47:49Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.