Graphonomy: Universal Image Parsing via Graph Reasoning and Transfer

• URL: http://arxiv.org/abs/2101.10620v1
• Date: Tue, 26 Jan 2021 08:19:03 GMT
• Title: Graphonomy: Universal Image Parsing via Graph Reasoning and Transfer
• Authors: Liang Lin and Yiming Gao and Ke Gong and Meng Wang and Xiaodan Liang
• Abstract summary: We propose a graph reasoning and transfer learning framework named "Graphonomy" It incorporates human knowledge and label taxonomy into the intermediate graph representation learning beyond local convolutions. It learns the global and structured semantic coherency in multiple domains via semantic-aware graph reasoning and transfer.
• Score: 140.72439827136085
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Prior highly-tuned image parsing models are usually studied in a certain domain with a specific set of semantic labels and can hardly be adapted into other scenarios (e.g., sharing discrepant label granularity) without extensive re-training. Learning a single universal parsing model by unifying label annotations from different domains or at various levels of granularity is a crucial but rarely addressed topic. This poses many fundamental learning challenges, e.g., discovering underlying semantic structures among different label granularity or mining label correlation across relevant tasks. To address these challenges, we propose a graph reasoning and transfer learning framework, named "Graphonomy", which incorporates human knowledge and label taxonomy into the intermediate graph representation learning beyond local convolutions. In particular, Graphonomy learns the global and structured semantic coherency in multiple domains via semantic-aware graph reasoning and transfer, enforcing the mutual benefits of the parsing across domains (e.g., different datasets or co-related tasks). The Graphonomy includes two iterated modules: Intra-Graph Reasoning and Inter-Graph Transfer modules. The former extracts the semantic graph in each domain to improve the feature representation learning by propagating information with the graph; the latter exploits the dependencies among the graphs from different domains for bidirectional knowledge transfer. We apply Graphonomy to two relevant but different image understanding research topics: human parsing and panoptic segmentation, and show Graphonomy can handle both of them well via a standard pipeline against current state-of-the-art approaches. Moreover, some extra benefit of our framework is demonstrated, e.g., generating the human parsing at various levels of granularity by unifying annotations across different datasets.

Related papers

• SA-GDA: Spectral Augmentation for Graph Domain Adaptation [38.71041292000361]
  Graph neural networks (GNNs) have achieved impressive impressions for graph-related tasks. This paper presents the textitSpectral Augmentation for Graph Domain Adaptation (method) for graph node classification. We develop a dual graph convolutional network to jointly exploits local and global consistency for feature aggregation.
  arXiv  Detail & Related papers  (2024-08-17T13:01:45Z)
• Text-Free Multi-domain Graph Pre-training: Toward Graph Foundation Models [33.2696184519275]
  We propose MDGPT, a text free Multi-Domain Graph Pre-Training and adaptation framework. First, we propose a set of domain tokens to align features across source domains for synergistic pre-training. Second, we propose a dual prompts, consisting of a unifying prompt and a mixing prompt, to further adapt the target domain with unified multi-domain knowledge.
  arXiv  Detail & Related papers  (2024-05-22T19:06:39Z)
• UniGraph: Learning a Unified Cross-Domain Foundation Model for Text-Attributed Graphs [30.635472655668078]
  Text-Attributed Graphs (TAGs) can generalize to unseen graphs and tasks across diverse domains. We propose a novel cascaded architecture of Language Models (LMs) and Graph Neural Networks (GNNs) as backbone networks. We demonstrate the model's effectiveness in self-supervised representation learning on unseen graphs, few-shot in-context transfer, and zero-shot transfer.
  arXiv  Detail & Related papers  (2024-02-21T09:06:31Z)
• Multiview Graph Learning with Consensus Graph [24.983233822595274]
  Graph topology inference is a significant task in many application domains. Many modern datasets are heterogeneous or mixed and involve multiple related graphs, i.e., multiview graphs. We propose an alternative method based on consensus regularization, where views are ensured to be similar. It is also employed to infer the functional brain connectivity networks of multiple subjects from their electroencephalogram (EEG) recordings.
  arXiv  Detail & Related papers  (2024-01-24T19:35:54Z)
• One for All: Towards Training One Graph Model for All Classification Tasks [61.656962278497225]
  A unified model for various graph tasks remains underexplored, primarily due to the challenges unique to the graph learning domain. We propose textbfOne for All (OFA), the first general framework that can use a single graph model to address the above challenges. OFA performs well across different tasks, making it the first general-purpose across-domains classification model on graphs.
  arXiv  Detail & Related papers  (2023-09-29T21:15:26Z)
• KMF: Knowledge-Aware Multi-Faceted Representation Learning for Zero-Shot Node Classification [75.95647590619929]
  Zero-Shot Node Classification (ZNC) has been an emerging and crucial task in graph data analysis. We propose a Knowledge-Aware Multi-Faceted framework (KMF) that enhances the richness of label semantics. A novel geometric constraint is developed to alleviate the problem of prototype drift caused by node information aggregation.
  arXiv  Detail & Related papers  (2023-08-15T02:38:08Z)
• Graph Adaptive Semantic Transfer for Cross-domain Sentiment Classification [68.06496970320595]
  Cross-domain sentiment classification (CDSC) aims to use the transferable semantics learned from the source domain to predict the sentiment of reviews in the unlabeled target domain. We present Graph Adaptive Semantic Transfer (GAST) model, an adaptive syntactic graph embedding method that is able to learn domain-invariant semantics from both word sequences and syntactic graphs.
  arXiv  Detail & Related papers  (2022-05-18T07:47:01Z)
• Group Contrastive Self-Supervised Learning on Graphs [101.45974132613293]
  We study self-supervised learning on graphs using contrastive methods. We argue that contrasting graphs in multiple subspaces enables graph encoders to capture more abundant characteristics.
  arXiv  Detail & Related papers  (2021-07-20T22:09:21Z)
• Multi-Level Graph Contrastive Learning [38.022118893733804]
  We propose a Multi-Level Graph Contrastive Learning (MLGCL) framework for learning robust representation of graph data by contrasting space views of graphs. The original graph is first-order approximation structure and contains uncertainty or error, while the $k$NN graph generated by encoding features preserves high-order proximity. Extensive experiments indicate MLGCL achieves promising results compared with the existing state-of-the-art graph representation learning methods on seven datasets.
  arXiv  Detail & Related papers  (2021-07-06T14:24:43Z)
• Knowledge-Guided Multi-Label Few-Shot Learning for General Image Recognition [75.44233392355711]
  KGGR framework exploits prior knowledge of statistical label correlations with deep neural networks. It first builds a structured knowledge graph to correlate different labels based on statistical label co-occurrence. Then, it introduces the label semantics to guide learning semantic-specific features. It exploits a graph propagation network to explore graph node interactions.
  arXiv  Detail & Related papers  (2020-09-20T15:05:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.