CogTree: Cognition Tree Loss for Unbiased Scene Graph Generation

• URL: http://arxiv.org/abs/2009.07526v2
• Date: Tue, 8 Jun 2021 06:27:33 GMT
• Title: CogTree: Cognition Tree Loss for Unbiased Scene Graph Generation
• Authors: Jing Yu, Yuan Chai, Yujing Wang, Yue Hu, Qi Wu
• Abstract summary: Scene Graph Generation (SGG) is unsatisfactory when faced with biased data in real-world scenarios. We propose a novel debiasing Cognition Tree (CogTree) loss for unbiased SGG. The loss is model-agnostic and consistently boosting the performance of several state-of-the-art models.
• Score: 23.55530043171931
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Scene graphs are semantic abstraction of images that encourage visual understanding and reasoning. However, the performance of Scene Graph Generation (SGG) is unsatisfactory when faced with biased data in real-world scenarios. Conventional debiasing research mainly studies from the view of balancing data distribution or learning unbiased models and representations, ignoring the correlations among the biased classes. In this work, we analyze this problem from a novel cognition perspective: automatically building a hierarchical cognitive structure from the biased predictions and navigating that hierarchy to locate the relationships, making the tail relationships receive more attention in a coarse-to-fine mode. To this end, we propose a novel debiasing Cognition Tree (CogTree) loss for unbiased SGG. We first build a cognitive structure CogTree to organize the relationships based on the prediction of a biased SGG model. The CogTree distinguishes remarkably different relationships at first and then focuses on a small portion of easily confused ones. Then, we propose a debiasing loss specially for this cognitive structure, which supports coarse-to-fine distinction for the correct relationships. The loss is model-agnostic and consistently boosting the performance of several state-of-the-art models. The code is available at: https://github.com/CYVincent/Scene-Graph-Transformer-CogTree.

Related papers

• Tensor-based Graph Learning with Consistency and Specificity for Multi-view Clustering [20.925436328405574]
  We propose a novel multi-view graph learning framework that simultaneously considers consistency and specificity. We formulate a new tensor-based target graph learning paradigm for noise-free graph fusion. Experiments on six datasets have demonstrated the superiority of our method.
  arXiv  Detail & Related papers  (2024-03-27T09:30:50Z)
• Graph Out-of-Distribution Generalization via Causal Intervention [69.70137479660113]
  We introduce a conceptually simple yet principled approach for training robust graph neural networks (GNNs) under node-level distribution shifts. Our method resorts to a new learning objective derived from causal inference that coordinates an environment estimator and a mixture-of-expert GNN predictor. Our model can effectively enhance generalization with various types of distribution shifts and yield up to 27.4% accuracy improvement over state-of-the-arts on graph OOD generalization benchmarks.
  arXiv  Detail & Related papers  (2024-02-18T07:49:22Z)
• Vision Relation Transformer for Unbiased Scene Graph Generation [31.29954125135073]
  Current Scene Graph Generation (SGG) methods suffer from an information loss regarding the entities local-level cues during the relation encoding process. We introduce the Vision rElation TransfOrmer (VETO), consisting of a novel local-level entity relation encoder. We show that VETO + MEET boosts the predictive performance by up to 47 percentage over the state of the art while being 10 times smaller.
  arXiv  Detail & Related papers  (2023-08-18T11:15:31Z)
• Rethinking Explaining Graph Neural Networks via Non-parametric Subgraph Matching [68.35685422301613]
  We propose a novel non-parametric subgraph matching framework, dubbed MatchExplainer, to explore explanatory subgraphs. It couples the target graph with other counterpart instances and identifies the most crucial joint substructure by minimizing the node corresponding-based distance. Experiments on synthetic and real-world datasets show the effectiveness of our MatchExplainer by outperforming all state-of-the-art parametric baselines with significant margins.
  arXiv  Detail & Related papers  (2023-01-07T05:14:45Z)
• Debiasing Graph Neural Networks via Learning Disentangled Causal Substructure [46.86463923605841]
  We present a graph classification investigation on the training graphs with severe bias. We discover that GNNs always tend to explore the spurious correlations to make decision. We propose a general disentangled GNN framework to learn the causal substructure and bias substructure.
  arXiv  Detail & Related papers  (2022-09-28T13:55:52Z)
• TREE-G: Decision Trees Contesting Graph Neural Networks [33.364191419692105]
  TREE-G modifies standard decision trees, by introducing a novel split function that is specialized for graph data. We show that TREE-G consistently outperforms other tree-based models and often outperforms other graph-learning algorithms such as Graph Neural Networks (GNNs) and Graph Kernels.
  arXiv  Detail & Related papers  (2022-07-06T15:53:17Z)
• Biasing Like Human: A Cognitive Bias Framework for Scene Graph Generation [20.435023745201878]
  We propose a novel 3-paradigms framework that simulates how humans incorporate the label linguistic features as guidance of vision-based representations. Our framework is model-agnostic to any scene graph model.
  arXiv  Detail & Related papers  (2022-03-17T08:29:52Z)
• Deconfounded Training for Graph Neural Networks [98.06386851685645]
  We present a new paradigm of decon training (DTP) that better mitigates the confounding effect and latches on the critical information. Specifically, we adopt the attention modules to disentangle the critical subgraph and trivial subgraph. It allows GNNs to capture a more reliable subgraph whose relation with the label is robust across different distributions.
  arXiv  Detail & Related papers  (2021-12-30T15:22:35Z)
• ExplaGraphs: An Explanation Graph Generation Task for Structured Commonsense Reasoning [65.15423587105472]
  We present a new generative and structured commonsense-reasoning task (and an associated dataset) of explanation graph generation for stance prediction. Specifically, given a belief and an argument, a model has to predict whether the argument supports or counters the belief and also generate a commonsense-augmented graph that serves as non-trivial, complete, and unambiguous explanation for the predicted stance. A significant 83% of our graphs contain external commonsense nodes with diverse structures and reasoning depths.
  arXiv  Detail & Related papers  (2021-04-15T17:51:36Z)
• High-Order Information Matters: Learning Relation and Topology for Occluded Person Re-Identification [84.43394420267794]
  We propose a novel framework by learning high-order relation and topology information for discriminative features and robust alignment. Our framework significantly outperforms state-of-the-art by6.5%mAP scores on Occluded-Duke dataset.
  arXiv  Detail & Related papers  (2020-03-18T12:18:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.