Recovering the Unbiased Scene Graphs from the Biased Ones

• URL: http://arxiv.org/abs/2107.02112v1
• Date: Mon, 5 Jul 2021 16:10:41 GMT
• Title: Recovering the Unbiased Scene Graphs from the Biased Ones
• Authors: Meng-Jiun Chiou, Henghui Ding, Hanshu Yan, Changhu Wang, Roger Zimmermann, Jiashi Feng
• Abstract summary: We show that due to the missing labels, scene graph generation (SGG) can be viewed as a "Learning from Positive and Unlabeled data" (PU learning) problem. We propose Dynamic Label Frequency Estimation (DLFE) to take advantage of training-time data augmentation and average over multiple training iterations to introduce more valid examples. Extensive experiments show that DLFE is more effective in estimating label frequencies than a naive variant of the traditional estimate, and DLFE significantly alleviates the long tail.
• Score: 99.24441932582195
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Given input images, scene graph generation (SGG) aims to produce comprehensive, graphical representations describing visual relationships among salient objects. Recently, more efforts have been paid to the long tail problem in SGG; however, the imbalance in the fraction of missing labels of different classes, or reporting bias, exacerbating the long tail is rarely considered and cannot be solved by the existing debiasing methods. In this paper we show that, due to the missing labels, SGG can be viewed as a "Learning from Positive and Unlabeled data" (PU learning) problem, where the reporting bias can be removed by recovering the unbiased probabilities from the biased ones by utilizing label frequencies, i.e., the per-class fraction of labeled, positive examples in all the positive examples. To obtain accurate label frequency estimates, we propose Dynamic Label Frequency Estimation (DLFE) to take advantage of training-time data augmentation and average over multiple training iterations to introduce more valid examples. Extensive experiments show that DLFE is more effective in estimating label frequencies than a naive variant of the traditional estimate, and DLFE significantly alleviates the long tail and achieves state-of-the-art debiasing performance on the VG dataset. We also show qualitatively that SGG models with DLFE produce prominently more balanced and unbiased scene graphs.

Related papers

• Continuous Contrastive Learning for Long-Tailed Semi-Supervised Recognition [50.61991746981703]
  Current state-of-the-art LTSSL approaches rely on high-quality pseudo-labels for large-scale unlabeled data. This paper introduces a novel probabilistic framework that unifies various recent proposals in long-tail learning. We introduce a continuous contrastive learning method, CCL, extending our framework to unlabeled data using reliable and smoothed pseudo-labels.
  arXiv  Detail & Related papers  (2024-10-08T15:06:10Z)
• FloCoDe: Unbiased Dynamic Scene Graph Generation with Temporal Consistency and Correlation Debiasing [14.50214193838818]
  FloCoDe: Flow-aware Temporal and Correlation Debiasing with uncertainty attenuation for unbiased dynamic scene graphs. We propose correlation debiasing and a correlation-based loss to learn unbiased relation representations for long-tailed classes.
  arXiv  Detail & Related papers  (2023-10-24T14:59:51Z)
• Beyond Myopia: Learning from Positive and Unlabeled Data through Holistic Predictive Trends [26.79150786180822]
  We unveil an intriguing yet long-overlooked observation in PUL. Predictive trends for positive and negative classes display distinctly different patterns. We propose a novel TPP-inspired measure for trend detection and prove its unbiasedness in predicting changes.
  arXiv  Detail & Related papers  (2023-10-06T08:06:15Z)
• Graph Out-of-Distribution Generalization with Controllable Data Augmentation [51.17476258673232]
  Graph Neural Network (GNN) has demonstrated extraordinary performance in classifying graph properties. Due to the selection bias of training and testing data, distribution deviation is widespread. We propose OOD calibration to measure the distribution deviation of virtual samples.
  arXiv  Detail & Related papers  (2023-08-16T13:10:27Z)
• Semi-Supervised Graph Imbalanced Regression [17.733488328772943]
  We propose a semi-supervised framework to progressively balance training data and reduce model bias via self-training. Results demonstrate that the proposed framework significantly reduces the error of predicted graph properties.
  arXiv  Detail & Related papers  (2023-05-20T04:11:00Z)
• Incorporating Semi-Supervised and Positive-Unlabeled Learning for Boosting Full Reference Image Quality Assessment [73.61888777504377]
  Full-reference (FR) image quality assessment (IQA) evaluates the visual quality of a distorted image by measuring its perceptual difference with pristine-quality reference. Unlabeled data can be easily collected from an image degradation or restoration process, making it encouraging to exploit unlabeled training data to boost FR-IQA performance. In this paper, we suggest to incorporate semi-supervised and positive-unlabeled (PU) learning for exploiting unlabeled data while mitigating the adverse effect of outliers.
  arXiv  Detail & Related papers  (2022-04-19T09:10:06Z)
• Confidence May Cheat: Self-Training on Graph Neural Networks under Distribution Shift [39.73304203101909]
  Self-training methods have been widely adopted on graphs by labeling high-confidence unlabeled nodes and then adding them to the training step. We propose a novel Distribution Recovered Graph Self-Training framework (DR- GST), which could recover the distribution of the original labeled dataset. Both our theoretical analysis and extensive experiments on five benchmark datasets demonstrate the effectiveness of the proposed DR- GST.
  arXiv  Detail & Related papers  (2022-01-27T07:12:27Z)
• General Greedy De-bias Learning [163.65789778416172]
  We propose a General Greedy De-bias learning framework (GGD), which greedily trains the biased models and the base model like gradient descent in functional space. GGD can learn a more robust base model under the settings of both task-specific biased models with prior knowledge and self-ensemble biased model without prior knowledge.
  arXiv  Detail & Related papers  (2021-12-20T14:47:32Z)
• Adaptive Label Smoothing To Regularize Large-Scale Graph Training [46.00927775402987]
  We propose the adaptive label smoothing (ALS) method to replace the one-hot hard labels with smoothed ones. ALS propagates node labels to aggregate the neighborhood label distribution in a pre-processing step, and then updates the optimal smoothed labels online to adapt to specific graph structure.
  arXiv  Detail & Related papers  (2021-08-30T23:51:31Z)
• Disentangling Sampling and Labeling Bias for Learning in Large-Output Spaces [64.23172847182109]
  We show that different negative sampling schemes implicitly trade-off performance on dominant versus rare labels. We provide a unified means to explicitly tackle both sampling bias, arising from working with a subset of all labels, and labeling bias, which is inherent to the data due to label imbalance.
  arXiv  Detail & Related papers  (2021-05-12T15:40:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.