NICEST: Noisy Label Correction and Training for Robust Scene Graph Generation

• URL: http://arxiv.org/abs/2207.13316v2
• Date: Sat, 13 Apr 2024 07:39:29 GMT
• Title: NICEST: Noisy Label Correction and Training for Robust Scene Graph Generation
• Authors: Lin Li, Jun Xiao, Hanrong Shi, Hanwang Zhang, Yi Yang, Wei Liu, Long Chen,
• Abstract summary: We propose a novel NoIsy label CorrEction and Sample Training strategy for SGG: NICEST. NICE first detects noisy samples and then reassigns them more high-quality soft predicate labels. NICEST can be seamlessly incorporated into any SGG architecture to boost its performance on different predicate categories.
• Score: 65.78472854070316
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Nearly all existing scene graph generation (SGG) models have overlooked the ground-truth annotation qualities of mainstream SGG datasets, i.e., they assume: 1) all the manually annotated positive samples are equally correct; 2) all the un-annotated negative samples are absolutely background. In this paper, we argue that neither of the assumptions applies to SGG: there are numerous noisy ground-truth predicate labels that break these two assumptions and harm the training of unbiased SGG models. To this end, we propose a novel NoIsy label CorrEction and Sample Training strategy for SGG: NICEST. Specifically, it consists of two parts: NICE and NIST, which rule out these noisy label issues by generating high-quality samples and the effective training strategy, respectively. NICE first detects noisy samples and then reassigns them more high-quality soft predicate labels. NIST is a multi-teacher knowledge distillation based training strategy, which enables the model to learn unbiased fusion knowledge. And a dynamic trade-off weighting strategy in NIST is designed to penalize the bias of different teachers. Due to the model-agnostic nature of both NICE and NIST, our NICEST can be seamlessly incorporated into any SGG architecture to boost its performance on different predicate categories. In addition, to better evaluate the generalization of SGG models, we further propose a new benchmark VG-OOD, by re-organizing the prevalent VG dataset and deliberately making the predicate distributions of the training and test sets as different as possible for each subject-object category pair. This new benchmark helps disentangle the influence of subject-object category based frequency biases. Extensive ablations and results on different backbones and tasks have attested to the effectiveness and generalization ability of each component of NICEST.

Related papers

• TD^2-Net: Toward Denoising and Debiasing for Dynamic Scene Graph Generation [76.24766055944554]
  We introduce a network named TD$2$-Net that aims at denoising and debiasing for dynamic SGG. TD$2$-Net outperforms the second-best competitors by 12.7 % on mean-Recall@10 for predicate classification.
  arXiv  Detail & Related papers  (2024-01-23T04:17:42Z)
• Decomposed Prototype Learning for Few-Shot Scene Graph Generation [28.796734816086065]
  We focus on a new promising task of scene graph generation (SGG): few-shot SGG (FSSGG) FSSGG encourages models to be able to quickly transfer previous knowledge and recognize novel predicates with only a few examples. We propose a novel Decomposed Prototype Learning (DPL)
  arXiv  Detail & Related papers  (2023-03-20T04:54:26Z)
• Towards Open-vocabulary Scene Graph Generation with Prompt-based Finetuning [84.39787427288525]
  Scene graph generation (SGG) is a fundamental task aimed at detecting visual relations between objects in an image. We introduce open-vocabulary scene graph generation, a novel, realistic and challenging setting in which a model is trained on a set of base object classes. Our method can support inference over completely unseen object classes, which existing methods are incapable of handling.
  arXiv  Detail & Related papers  (2022-08-17T09:05:38Z)
• Label Semantic Knowledge Distillation for Unbiased Scene Graph Generation [34.20922091969159]
  We propose a novel model-agnostic Label Semantic Knowledge Distillation (LS-KD) for unbiased Scene Graph Generation (SGG) LS-KD dynamically generates a soft label for each subject-object instance by fusing a predicted Label Semantic Distribution (LSD) with its original one-hot target label.
  arXiv  Detail & Related papers  (2022-08-07T16:19:19Z)
• Adaptive Fine-Grained Predicates Learning for Scene Graph Generation [122.4588401267544]
  General Scene Graph Generation (SGG) models tend to predict head predicates and re-balancing strategies prefer tail categories. We propose an Adaptive Fine-Grained Predicates Learning (FGPL-A) which aims at differentiating hard-to-distinguish predicates for SGG. Our proposed model-agnostic strategy significantly boosts performance of benchmark models on VG-SGG and GQA-SGG datasets by up to 175% and 76% on Mean Recall@100, achieving new state-of-the-art performance.
  arXiv  Detail & Related papers  (2022-07-11T03:37:57Z)
• The Devil is in the Labels: Noisy Label Correction for Robust Scene Graph Generation [33.45310571580091]
  We propose a novel model-agnostic NoIsy label CorrEction strategy for unbiased SGG models. NICE can not only detect noisy samples but also reassign more high-quality predicate labels to them. NICE consists of three components: negative Noisy Sample Detection (Neg-NSD), positive NSD (Pos-NSD), and Noisy Sample Correction (NSC)
  arXiv  Detail & Related papers  (2022-06-07T05:03:57Z)
• Fine-Grained Predicates Learning for Scene Graph Generation [155.48614435437355]
  Fine-Grained Predicates Learning aims at differentiating among hard-to-distinguish predicates for Scene Graph Generation task. We introduce a Predicate Lattice that helps SGG models to figure out fine-grained predicate pairs. We then propose a Category Discriminating Loss and an Entity Discriminating Loss, which both contribute to distinguishing fine-grained predicates.
  arXiv  Detail & Related papers  (2022-04-06T06:20:09Z)
• Not All Relations are Equal: Mining Informative Labels for Scene Graph Generation [48.21846438269506]
  Scene graph generation (SGG) aims to capture a wide variety of interactions between pairs of objects. Existing SGG methods fail to acquire complex reasoning about visual and textual correlations due to various biases in training data. We propose a novel framework for SGG training that exploits relation labels based on their informativeness.
  arXiv  Detail & Related papers  (2021-11-26T14:34:12Z)
• Unified Robust Training for Graph NeuralNetworks against Label Noise [12.014301020294154]
  We propose a new framework, UnionNET, for learning with noisy labels on graphs under a semi-supervised setting. Our approach provides a unified solution for robustly training GNNs and performing label correction simultaneously.
  arXiv  Detail & Related papers  (2021-03-05T01:17:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.