HGFormer: Topology-Aware Vision Transformer with HyperGraph Learning

• URL: http://arxiv.org/abs/2504.02440v2
• Date: Wed, 09 Apr 2025 02:45:11 GMT
• Title: HGFormer: Topology-Aware Vision Transformer with HyperGraph Learning
• Authors: Hao Wang, Shuo Zhang, Biao Leng,
• Abstract summary: We introduce the concept of hypergraph for perceptual exploration.<n>Specifically, we propose a topology-aware vision transformer called HyperGraph Transformer (HGFormer)<n>We develop an effective and unitive representation, achieving distinct and detailed scene depictions.
• Score: 14.344989900296968
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The computer vision community has witnessed an extensive exploration of vision transformers in the past two years. Drawing inspiration from traditional schemes, numerous works focus on introducing vision-specific inductive biases. However, the implicit modeling of permutation invariance and fully-connected interaction with individual tokens disrupts the regional context and spatial topology, further hindering higher-order modeling. This deviates from the principle of perceptual organization that emphasizes the local groups and overall topology of visual elements. Thus, we introduce the concept of hypergraph for perceptual exploration. Specifically, we propose a topology-aware vision transformer called HyperGraph Transformer (HGFormer). Firstly, we present a Center Sampling K-Nearest Neighbors (CS-KNN) algorithm for semantic guidance during hypergraph construction. Secondly, we present a topology-aware HyperGraph Attention (HGA) mechanism that integrates hypergraph topology as perceptual indications to guide the aggregation of global and unbiased information during hypergraph messaging. Using HGFormer as visual backbone, we develop an effective and unitive representation, achieving distinct and detailed scene depictions. Empirical experiments show that the proposed HGFormer achieves competitive performance compared to the recent SoTA counterparts on various visual benchmarks. Extensive ablation and visualization studies provide comprehensive explanations of our ideas and contributions.

Related papers

• Cog-RAG: Cognitive-Inspired Dual-Hypergraph with Theme Alignment Retrieval-Augmented Generation [43.737415090081555]
  We propose a theme-aligned dual-hypergraph RAG framework (Cog-RAG) that uses a theme hypergraph to capture inter-chunk thematic structure and an entity hypergraph to model high-order semantic relations.<n>Our experiments demonstrate that Cog-RAG significantly outperforms existing state-of-the-art baseline approaches.
  arXiv  Detail & Related papers  (2025-11-17T10:10:33Z)
• What Improves the Generalization of Graph Transformers? A Theoretical Dive into the Self-attention and Positional Encoding [67.59552859593985]
  Graph Transformers, which incorporate self-attention and positional encoding, have emerged as a powerful architecture for various graph learning tasks. This paper introduces first theoretical investigation of a shallow Graph Transformer for semi-supervised classification.
  arXiv  Detail & Related papers  (2024-06-04T05:30:16Z)
• Hypergraph Transformer for Semi-Supervised Classification [50.92027313775934]
  We propose a novel hypergraph learning framework, HyperGraph Transformer (HyperGT) HyperGT uses a Transformer-based neural network architecture to effectively consider global correlations among all nodes and hyperedges. It achieves comprehensive hypergraph representation learning by effectively incorporating global interactions while preserving local connectivity patterns.
  arXiv  Detail & Related papers  (2023-12-18T17:50:52Z)
• Hypergraph Node Representation Learning with One-Stage Message Passing [28.311325846217574]
  We propose a novel one-stage message passing paradigm to model both global and local information propagation for hypergraphs. We integrate this paradigm into HGraphormer, a Transformer-based framework for hypergraph node representation learning.
  arXiv  Detail & Related papers  (2023-12-01T04:10:00Z)
• Topology-guided Hypergraph Transformer Network: Unveiling Structural Insights for Improved Representation [1.1606619391009658]
  We propose a Topology-guided Hypergraph Transformer Network (THTN) In this model, we first formulate a hypergraph from a graph while retaining its structural essence to learn higher-order relations within the graph. We present a structure-aware self-attention mechanism that discovers the important nodes and hyperedges from both semantic and structural viewpoints.
  arXiv  Detail & Related papers  (2023-10-14T20:08:54Z)
• A Unified View Between Tensor Hypergraph Neural Networks And Signal Denoising [7.083679120873857]
  We show that the tensor-hypergraph convolutional network (T-HGCN) has emerged as a powerful architecture for preserving higher-order interactions on hypergraphs. We further design a tensor-hypergraph iterative network (T-HGIN) based on the HyperGSD problem, which takes advantage of a multi-step updating scheme in every single layer.
  arXiv  Detail & Related papers  (2023-09-15T13:19:31Z)
• From Hypergraph Energy Functions to Hypergraph Neural Networks [94.88564151540459]
  We present an expressive family of parameterized, hypergraph-regularized energy functions. We then demonstrate how minimizers of these energies effectively serve as node embeddings. We draw parallels between the proposed bilevel hypergraph optimization, and existing GNN architectures in common use.
  arXiv  Detail & Related papers  (2023-06-16T04:40:59Z)
• Vision Transformers: From Semantic Segmentation to Dense Prediction [139.15562023284187]
  We explore the global context learning potentials of vision transformers (ViTs) for dense visual prediction. Our motivation is that through learning global context at full receptive field layer by layer, ViTs may capture stronger long-range dependency information. We formulate a family of Hierarchical Local-Global (HLG) Transformers, characterized by local attention within windows and global-attention across windows in a pyramidal architecture.
  arXiv  Detail & Related papers  (2022-07-19T15:49:35Z)
• Hypergraph Convolutional Networks via Equivalency between Hypergraphs and Undirected Graphs [59.71134113268709]
  We present General Hypergraph Spectral Convolution(GHSC), a general learning framework that can handle EDVW and EIVW hypergraphs. In this paper, we show that the proposed framework can achieve state-of-the-art performance. Experiments from various domains including social network analysis, visual objective classification, protein learning demonstrate that the proposed framework can achieve state-of-the-art performance.
  arXiv  Detail & Related papers  (2022-03-31T10:46:47Z)
• Learning Multi-Granular Hypergraphs for Video-Based Person Re-Identification [110.52328716130022]
  Video-based person re-identification (re-ID) is an important research topic in computer vision. We propose a novel graph-based framework, namely Multi-Granular Hypergraph (MGH) to better representational capabilities. 90.0% top-1 accuracy on MARS is achieved using MGH, outperforming the state-of-the-arts schemes.
  arXiv  Detail & Related papers  (2021-04-30T11:20:02Z)
• Interferometric Graph Transform: a Deep Unsupervised Graph Representation [0.0]
  Interferometric Graph Transform (IGT) is a new class of deep unsupervised graph convolutional neural network for building graph representations. We show that our learned representation consists of both discriminative and invariant features, thanks to a novel greedy concave objective.
  arXiv  Detail & Related papers  (2020-06-10T08:27:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.