PCPL: Predicate-Correlation Perception Learning for Unbiased Scene Graph Generation

• URL: http://arxiv.org/abs/2009.00893v1
• Date: Wed, 2 Sep 2020 08:30:09 GMT
• Title: PCPL: Predicate-Correlation Perception Learning for Unbiased Scene Graph Generation
• Authors: Shaotian Yan, Chen Shen, Zhongming Jin, Jianqiang Huang, Rongxin Jiang, Yaowu Chen, Xian-Sheng Hua
• Abstract summary: We propose a novel Predicate-Correlation Perception Learning scheme to adaptively seek out appropriate loss weights. Our PCPL framework is further equipped with a graph encoder module to better extract context features.
• Score: 58.98802062945709
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Today, scene graph generation(SGG) task is largely limited in realistic scenarios, mainly due to the extremely long-tailed bias of predicate annotation distribution. Thus, tackling the class imbalance trouble of SGG is critical and challenging. In this paper, we first discover that when predicate labels have strong correlation with each other, prevalent re-balancing strategies(e.g., re-sampling and re-weighting) will give rise to either over-fitting the tail data(e.g., bench sitting on sidewalk rather than on), or still suffering the adverse effect from the original uneven distribution(e.g., aggregating varied parked on/standing on/sitting on into on). We argue the principal reason is that re-balancing strategies are sensitive to the frequencies of predicates yet blind to their relatedness, which may play a more important role to promote the learning of predicate features. Therefore, we propose a novel Predicate-Correlation Perception Learning(PCPL for short) scheme to adaptively seek out appropriate loss weights by directly perceiving and utilizing the correlation among predicate classes. Moreover, our PCPL framework is further equipped with a graph encoder module to better extract context features. Extensive experiments on the benchmark VG150 dataset show that the proposed PCPL performs markedly better on tail classes while well-preserving the performance on head ones, which significantly outperforms previous state-of-the-art methods.

Related papers

• Ensemble Predicate Decoding for Unbiased Scene Graph Generation [40.01591739856469]
  Scene Graph Generation (SGG) aims to generate a comprehensive graphical representation that captures semantic information of a given scenario. The model's performance in predicting more fine-grained predicates is hindered by a significant predicate bias. This paper proposes Ensemble Predicate Decoding (EPD), which employs multiple decoders to attain unbiased scene graph generation.
  arXiv  Detail & Related papers  (2024-08-26T11:24:13Z)
• Unbiased Scene Graph Generation via Two-stage Causal Modeling [11.390135312161044]
  We propose Two-stage Causal Modeling (TsCM) for the Scene Graph Generation (SGG) task. TsCM takes the long-tailed distribution and semantic confusion as confounders to the Structural Causal Model (SCM) and then decouples the causal intervention into two stages. Our method can achieve state-of-the-art performance in terms of mean recall rate.
  arXiv  Detail & Related papers  (2023-07-11T14:11:24Z)
• Skew Class-balanced Re-weighting for Unbiased Scene Graph Generation [23.237308265907373]
  The Skew Class-balanced Re-weighting (SCR) loss function is considered for the unbiased SGG models. The SCR estimates the target predicate weight coefficient and then re-weights more to the biased predicates for better trading-off between the majority predicates and the minority ones.
  arXiv  Detail & Related papers  (2023-01-01T05:26:33Z)
• Dual-branch Hybrid Learning Network for Unbiased Scene Graph Generation [87.13847750383778]
  We propose a Dual-branch Hybrid Learning network (DHL) to take care of both head predicates and tail ones for Scene Graph Generation (SGG) We show that our approach achieves a new state-of-the-art performance on VG and GQA datasets.
  arXiv  Detail & Related papers  (2022-07-16T11:53:50Z)
• Adaptive Fine-Grained Predicates Learning for Scene Graph Generation [122.4588401267544]
  General Scene Graph Generation (SGG) models tend to predict head predicates and re-balancing strategies prefer tail categories. We propose an Adaptive Fine-Grained Predicates Learning (FGPL-A) which aims at differentiating hard-to-distinguish predicates for SGG. Our proposed model-agnostic strategy significantly boosts performance of benchmark models on VG-SGG and GQA-SGG datasets by up to 175% and 76% on Mean Recall@100, achieving new state-of-the-art performance.
  arXiv  Detail & Related papers  (2022-07-11T03:37:57Z)
• Relieving Long-tailed Instance Segmentation via Pairwise Class Balance [85.53585498649252]
  Long-tailed instance segmentation is a challenging task due to the extreme imbalance of training samples among classes. It causes severe biases of the head classes (with majority samples) against the tailed ones. We propose a novel Pairwise Class Balance (PCB) method, built upon a confusion matrix which is updated during training to accumulate the ongoing prediction preferences.
  arXiv  Detail & Related papers  (2022-01-08T07:48:36Z)
• Generalizing Graph Neural Networks on Out-Of-Distribution Graphs [51.33152272781324]
  Graph Neural Networks (GNNs) are proposed without considering the distribution shifts between training and testing graphs. In such a setting, GNNs tend to exploit subtle statistical correlations existing in the training set for predictions, even though it is a spurious correlation. We propose a general causal representation framework, called StableGNN, to eliminate the impact of spurious correlations.
  arXiv  Detail & Related papers  (2021-11-20T18:57:18Z)
• Predicate correlation learning for scene graph generation [46.097869554455336]
  For a typical Scene Graph Generation (SGG) method, there is often a large gap in the performance of the predicates' head classes and tail classes. A Predicate Correlation Matrix (PCM) is defined to quantify the relationship between predicate pairs. PCM is integrated into a Predicate Correlation Loss function ($L_PC$) to reduce discouraging gradients of unannotated classes.
  arXiv  Detail & Related papers  (2021-07-06T16:24:33Z)
• Dual ResGCN for Balanced Scene GraphGeneration [106.7828712878278]
  We propose a novel model, dubbed textitdual ResGCN, which consists of an object residual graph convolutional network and a relation residual graph convolutional network. The two networks are complementary to each other. The former captures object-level context information, textiti.e., the connections among objects. The latter is carefully designed to explicitly capture relation-level context information textiti.e., the connections among relations.
  arXiv  Detail & Related papers  (2020-11-09T07:44:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.