Self-supervised Hypergraphs for Learning Multiple World Interpretations

• URL: http://arxiv.org/abs/2308.07615v2
• Date: Mon, 21 Aug 2023 14:48:09 GMT
• Title: Self-supervised Hypergraphs for Learning Multiple World Interpretations
• Authors: Alina Marcu, Mihai Pirvu, Dragos Costea, Emanuela Haller, Emil Slusanschi, Ahmed Nabil Belbachir, Rahul Sukthankar, Marius Leordeanu
• Abstract summary: We present a method for learning multiple scene representations given a small labeled set, by exploiting the relationships between such representations in the form of a multi-task hypergraph. We show how we can use the hypergraph to improve a powerful pretrained VisTransformer model without any additional labeled data. We also introduce Dronescapes, a large video dataset captured with UAVs in different complex real-world scenes, with multiple representations, suitable for multi-task learning.
• Score: 16.83248115598725
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We present a method for learning multiple scene representations given a small labeled set, by exploiting the relationships between such representations in the form of a multi-task hypergraph. We also show how we can use the hypergraph to improve a powerful pretrained VisTransformer model without any additional labeled data. In our hypergraph, each node is an interpretation layer (e.g., depth or segmentation) of the scene. Within each hyperedge, one or several input nodes predict the layer at the output node. Thus, each node could be an input node in some hyperedges and an output node in others. In this way, multiple paths can reach the same node, to form ensembles from which we obtain robust pseudolabels, which allow self-supervised learning in the hypergraph. We test different ensemble models and different types of hyperedges and show superior performance to other multi-task graph models in the field. We also introduce Dronescapes, a large video dataset captured with UAVs in different complex real-world scenes, with multiple representations, suitable for multi-task learning.

Related papers

• Representation learning in multiplex graphs: Where and how to fuse information? [5.0235828656754915]
  Multiplex graphs possess richer information, provide better modeling capabilities and integrate more detailed data from potentially different sources. In this paper, we tackle the problem of learning representations for nodes in multiplex networks in an unsupervised or self-supervised manner. We propose improvements in how to construct GNN architectures that deal with multiplex graphs.
  arXiv  Detail & Related papers  (2024-02-27T21:47:06Z)
• Hypergraph Transformer for Semi-Supervised Classification [50.92027313775934]
  We propose a novel hypergraph learning framework, HyperGraph Transformer (HyperGT) HyperGT uses a Transformer-based neural network architecture to effectively consider global correlations among all nodes and hyperedges. It achieves comprehensive hypergraph representation learning by effectively incorporating global interactions while preserving local connectivity patterns.
  arXiv  Detail & Related papers  (2023-12-18T17:50:52Z)
• Classification of Edge-dependent Labels of Nodes in Hypergraphs [17.454063924648896]
  We introduce a classification of edge-dependent node labels as a new problem. This problem can be used as a benchmark task for hypergraph neural networks. We propose WHATsNet, a novel hypergraph neural network that represents the same node differently depending on the hyperedges it participates in.
  arXiv  Detail & Related papers  (2023-06-05T16:50:34Z)
• All the World's a (Hyper)Graph: A Data Drama [55.144729234861316]
  Hyperbard is a dataset of diverse data representations from Shakespeare's plays. Our representations range from simple graphs capturing character co-occurrence in single scenes to hypergraphs encoding complex communication settings. As an homage to our data source, and asserting that science can also be art, we present all our points in the form of a play.
  arXiv  Detail & Related papers  (2022-06-16T14:51:28Z)
• Hypergraph Convolutional Networks via Equivalency between Hypergraphs and Undirected Graphs [59.71134113268709]
  We present General Hypergraph Spectral Convolution(GHSC), a general learning framework that can handle EDVW and EIVW hypergraphs. In this paper, we show that the proposed framework can achieve state-of-the-art performance. Experiments from various domains including social network analysis, visual objective classification, protein learning demonstrate that the proposed framework can achieve state-of-the-art performance.
  arXiv  Detail & Related papers  (2022-03-31T10:46:47Z)
• Representing Videos as Discriminative Sub-graphs for Action Recognition [165.54738402505194]
  We introduce a new design of sub-graphs to represent and encode theriminative patterns of each action in the videos. We present MUlti-scale Sub-Earn Ling (MUSLE) framework that novelly builds space-time graphs and clusters into compact sub-graphs on each scale.
  arXiv  Detail & Related papers  (2022-01-11T16:15:25Z)
• Adaptive Neural Message Passing for Inductive Learning on Hypergraphs [21.606287447052757]
  We present HyperMSG, a novel hypergraph learning framework. It adapts to the data and task by learning an attention weight associated with each node's degree centrality. It is robust and outperforms state-of-the-art hypergraph learning methods on a wide range of tasks and datasets.
  arXiv  Detail & Related papers  (2021-09-22T12:24:02Z)
• Learning Multi-Granular Hypergraphs for Video-Based Person Re-Identification [110.52328716130022]
  Video-based person re-identification (re-ID) is an important research topic in computer vision. We propose a novel graph-based framework, namely Multi-Granular Hypergraph (MGH) to better representational capabilities. 90.0% top-1 accuracy on MARS is achieved using MGH, outperforming the state-of-the-arts schemes.
  arXiv  Detail & Related papers  (2021-04-30T11:20:02Z)
• Reducing the Annotation Effort for Video Object Segmentation Datasets [50.893073670389164]
  densely labeling every frame with pixel masks does not scale to large datasets. We use a deep convolutional network to automatically create pseudo-labels on a pixel level from much cheaper bounding box annotations. We obtain the new TAO-VOS benchmark, which we make publicly available at www.vision.rwth-aachen.de/page/taovos.
  arXiv  Detail & Related papers  (2020-11-02T17:34:45Z)
• HyperSAGE: Generalizing Inductive Representation Learning on Hypergraphs [24.737560790401314]
  We present HyperSAGE, a novel hypergraph learning framework that uses a two-level neural message passing strategy to accurately and efficiently propagate information through hypergraphs. We show that HyperSAGE outperforms state-of-the-art hypergraph learning methods on representative benchmark datasets.
  arXiv  Detail & Related papers  (2020-10-09T13:28:06Z)
• Semi-Supervised Learning for Multi-Task Scene Understanding by Neural Graph Consensus [23.528834793031894]
  We address the problem of semi-supervised learning in the context of multiple visual interpretations of the world. We show how prediction of different representations could be effectively learned through self-supervised consensus in our graph. We also compare to state-of-the-art methods for multi-task and semi-supervised learning and show superior performance.
  arXiv  Detail & Related papers  (2020-10-02T16:30:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.