ATRM: Attention-based Task-level Relation Module for GNN-based Few-shot Learning

• URL: http://arxiv.org/abs/2101.09840v1
• Date: Mon, 25 Jan 2021 00:53:04 GMT
• Title: ATRM: Attention-based Task-level Relation Module for GNN-based Few-shot Learning
• Authors: Yurong Guo, Zhanyu Ma, Xiaoxu Li, and Yuan Dong
• Abstract summary: We propose a new relation measure method, namely the attention-based task-level relation module (ATRM) The proposed module captures the relation representations between nodes by considering the sample-to-task instead of sample-to-sample embedding features. Experimental results demonstrate that the proposed module is effective for GNN-based few-shot learning.
• Score: 14.464964336101028
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Recently, graph neural networks (GNNs) have shown powerful ability to handle few-shot classification problem, which aims at classifying unseen samples when trained with limited labeled samples per class. GNN-based few-shot learning architectures mostly replace traditional metric with a learnable GNN. In the GNN, the nodes are set as the samples embedding, and the relationship between two connected nodes can be obtained by a network, the input of which is the difference of their embedding features. We consider this method of measuring relation of samples only models the sample-to-sample relation, while neglects the specificity of different tasks. That is, this method of measuring relation does not take the task-level information into account. To this end, we propose a new relation measure method, namely the attention-based task-level relation module (ATRM), to explicitly model the task-level relation of one sample to all the others. The proposed module captures the relation representations between nodes by considering the sample-to-task instead of sample-to-sample embedding features. We conducted extensive experiments on four benchmark datasets: mini-ImageNet, tiered-ImageNet, CUB-200-2011, and CIFAR-FS. Experimental results demonstrate that the proposed module is effective for GNN-based few-shot learning.

Related papers

• Supervised Gradual Machine Learning for Aspect Category Detection [0.9857683394266679]
  Aspect Category Detection (ACD) aims to identify implicit and explicit aspects in a given review sentence. We propose a novel approach to tackle the ACD task by combining Deep Neural Networks (DNNs) with Gradual Machine Learning (GML) in a supervised setting.
  arXiv  Detail & Related papers  (2024-04-08T07:21:46Z)
• Two-level Graph Network for Few-Shot Class-Incremental Learning [7.815043173207539]
  Few-shot class-incremental learning (FSCIL) aims to design machine learning algorithms that can continually learn new concepts from a few data points. Existing FSCIL methods ignore the semantic relationships between sample-level and class-level. In this paper, we designed a two-level graph network for FSCIL named Sample-level and Class-level Graph Neural Network (SCGN)
  arXiv  Detail & Related papers  (2023-03-24T08:58:08Z)
• Compare learning: bi-attention network for few-shot learning [6.559037166322981]
  One of the Few-shot learning methods called metric learning addresses this challenge by first learning a deep distance metric to determine whether a pair of images belong to the same category. In this paper, we propose a novel approach named Bi-attention network to compare the instances, which can measure the similarity between embeddings of instances precisely, globally and efficiently.
  arXiv  Detail & Related papers  (2022-03-25T07:39:10Z)
• BatchFormer: Learning to Explore Sample Relationships for Robust Representation Learning [93.38239238988719]
  We propose to enable deep neural networks with the ability to learn the sample relationships from each mini-batch. BatchFormer is applied into the batch dimension of each mini-batch to implicitly explore sample relationships during training. We perform extensive experiments on over ten datasets and the proposed method achieves significant improvements on different data scarcity applications.
  arXiv  Detail & Related papers  (2022-03-03T05:31:33Z)
• ECKPN: Explicit Class Knowledge Propagation Network for Transductive Few-shot Learning [53.09923823663554]
  Class-level knowledge can be easily learned by humans from just a handful of samples. We propose an Explicit Class Knowledge Propagation Network (ECKPN) to address this problem. We conduct extensive experiments on four few-shot classification benchmarks, and the experimental results show that the proposed ECKPN significantly outperforms the state-of-the-art methods.
  arXiv  Detail & Related papers  (2021-06-16T02:29:43Z)
• Policy-GNN: Aggregation Optimization for Graph Neural Networks [60.50932472042379]
  Graph neural networks (GNNs) aim to model the local graph structures and capture the hierarchical patterns by aggregating the information from neighbors. It is a challenging task to develop an effective aggregation strategy for each node, given complex graphs and sparse features. We propose Policy-GNN, a meta-policy framework that models the sampling procedure and message passing of GNNs into a combined learning process.
  arXiv  Detail & Related papers  (2020-06-26T17:03:06Z)
• High-order structure preserving graph neural network for few-shot learning [10.296473510866228]
  Few-shot learning can find the latent structure information between the prior knowledge and the queried data by the similarity metric of meta-learning. Most existing methods try to model the similarity relationship of the samples in the intra tasks, and generalize the model to identify the new categories. The proposed high-order structure preserving graph neural network(HOSP-GNN) can explore the rich structure of the samples to predict the label of the queried data on graph.
  arXiv  Detail & Related papers  (2020-05-29T06:38:51Z)
• Memory-Augmented Relation Network for Few-Shot Learning [114.47866281436829]
  In this work, we investigate a new metric-learning method, Memory-Augmented Relation Network (MRN) In MRN, we choose the samples that are visually similar from the working context, and perform weighted information propagation to attentively aggregate helpful information from chosen ones to enhance its representation. We empirically demonstrate that MRN yields significant improvement over its ancestor and achieves competitive or even better performance when compared with other few-shot learning approaches.
  arXiv  Detail & Related papers  (2020-05-09T10:09:13Z)
• One-Shot Object Detection without Fine-Tuning [62.39210447209698]
  We introduce a two-stage model consisting of a first stage Matching-FCOS network and a second stage Structure-Aware Relation Module. We also propose novel training strategies that effectively improve detection performance. Our method exceeds the state-of-the-art one-shot performance consistently on multiple datasets.
  arXiv  Detail & Related papers  (2020-05-08T01:59:23Z)
• Learning What Makes a Difference from Counterfactual Examples and Gradient Supervision [57.14468881854616]
  We propose an auxiliary training objective that improves the generalization capabilities of neural networks. We use pairs of minimally-different examples with different labels, a.k.a counterfactual or contrasting examples, which provide a signal indicative of the underlying causal structure of the task. Models trained with this technique demonstrate improved performance on out-of-distribution test sets.
  arXiv  Detail & Related papers  (2020-04-20T02:47:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.