S2Aligner: Pair-Efficient and Transferable Pre-Training for Sparse Text-Attributed Graphs

• URL: http://arxiv.org/abs/2605.18579v3
• Date: Wed, 20 May 2026 03:15:37 GMT
• Title: S2Aligner: Pair-Efficient and Transferable Pre-Training for Sparse Text-Attributed Graphs
• Abstract summary: Pre-training on text-attributed graphs (TAGs) is central to building transferable graph foundation models.<n>This paper presents S2Aligner, a sparsity-aware and structure-enhanced LLM-as-Aligner framework for graph-text pre-training on sparse TAGs.
• Score: 23.83658846856642
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Pre-training on text-attributed graphs (TAGs) is central to building transferable graph foundation models, where LLM-as-Aligner methods align graph and text representations through the semantic knowledge of large language models. However, these methods usually assume that node texts provide sufficient and reliable supervision, an assumption often violated in real-world sparse TAGs. When textual anchors are missing, noisy, or uneven across domains, graph structures must be aligned with weak semantic evidence, leading to unreliable structure-semantics correspondence and sparsity-induced transfer bias. This paper presents S2Aligner, a sparsity-aware and structure-enhanced LLM-as-Aligner framework for graph-text pre-training on sparse TAGs. The key idea is to decouple semantic alignment from structural modeling, allowing topology-aware signals to enhance alignment without contaminating the shared semantic space. Specifically, S2Aligner decomposes graph-text representations into semantic and structural components, uses structure-oriented reconstruction with consistency control to inject reliable topology cues into text representations, and suppresses inconsistent structural signals under textual sparsity. Moreover, S2Aligner introduces sparsity-aware cross-domain risk balancing, which calibrates domain risks through a global-domain density ratio and downweights unreliable sparse samples via graph reliability estimation. Theoretical analysis shows that this objective reduces cross-domain generalization gaps by controlling domain risk discrepancy. Extensive experiments across diverse graph domains, sparsity levels, and downstream tasks demonstrate that S2Aligner consistently outperforms existing baselines.

Related papers

• TERGAD: Structure-Aware Text-Enhanced Representations for Graph Anomaly Detection [15.051477921035264]
  Graph Anomaly Detection (GAD) aims to identify atypical graph entities, such as nodes, edges, or substructures, that deviate significantly from the majority.<n>Existing text-rich approaches typically integrate structural context into the data representation pipeline using raw textual features.<n>We propose TERGAD, a novel data augmentation framework that enriches structural semantics for GAD via the semantic reasoning capabilities of Large Language Models.
  arXiv  Detail & Related papers  (2026-05-19T12:09:36Z)
• PRISM: Prior Rectification and Uncertainty-Aware Structure Modeling for Diffusion-Based Text Image Super-Resolution [51.96078493242164]
  PRISM is a single-step diffusion-based Text-SR framework.<n> PRISM achieves state-of-the-art performance with millisecond-level inference.
  arXiv  Detail & Related papers  (2026-05-13T05:31:06Z)
• Improving LLM Reasoning with Homophily-aware Structural and Semantic Text-Attributed Graph Compression [55.51959317490934]
  Large language models (LLMs) have demonstrated promising capabilities in Text-Attributed Graph (TAG) understanding.<n>We argue that graphs inherently contain rich structural and semantic information, and that their effective exploitation can unlock potential gains in LLMs reasoning performance.<n>We propose Homophily-aware Structural and Semantic Compression for LLMs (HS2C), a framework centered on exploiting graph homophily.
  arXiv  Detail & Related papers  (2026-01-13T03:35:18Z)
• Text2Graph VPR: A Text-to-Graph Expert System for Explainable Place Recognition in Changing Environments [0.0]
  Text2Graph VPR converts image sequences into textual scene descriptions.<n> Scene graphs capture objects, attributes and pairwise relations.<n>We demonstrate robust retrieval under severe appearance shifts.
  arXiv  Detail & Related papers  (2025-12-21T06:16:20Z)
• Integrating Structural and Semantic Signals in Text-Attributed Graphs with BiGTex [0.0]
  BiGTex is a novel architecture that tightly integrates GNNs and LLMs through stacked Graph-Text Fusion Units.<n>BiGTex achieves state-of-the-art performance in node classification and generalizes effectively to link prediction.
  arXiv  Detail & Related papers  (2025-04-16T20:25:11Z)
• SaVe-TAG: Semantic-aware Vicinal Risk Minimization for Long-Tailed Text-Attributed Graphs [16.24571541782205]
  Real-world graph data often follows long-tailed distributions, making it difficult for Graph Neural Networks (GNNs) to generalize well across both head and tail classes.<n>Recent advances in Vicinal Risk Minimization (VRM) have shown promise in mitigating class imbalance with numeric semantics.
  arXiv  Detail & Related papers  (2024-10-22T10:36:15Z)
• A Pure Transformer Pretraining Framework on Text-attributed Graphs [50.833130854272774]
  We introduce a feature-centric pretraining perspective by treating graph structure as a prior. Our framework, Graph Sequence Pretraining with Transformer (GSPT), samples node contexts through random walks. GSPT can be easily adapted to both node classification and link prediction, demonstrating promising empirical success on various datasets.
  arXiv  Detail & Related papers  (2024-06-19T22:30:08Z)
• Counterfactual Intervention Feature Transfer for Visible-Infrared Person Re-identification [69.45543438974963]
  We find graph-based methods in the visible-infrared person re-identification task (VI-ReID) suffer from bad generalization because of two issues. The well-trained input features weaken the learning of graph topology, making it not generalized enough during the inference process. We propose a Counterfactual Intervention Feature Transfer (CIFT) method to tackle these problems.
  arXiv  Detail & Related papers  (2022-08-01T16:15:31Z)
• Graph Spectral Embedding using the Geodesic Betweeness Centrality [76.27138343125985]
  We introduce the Graph Sylvester Embedding (GSE), an unsupervised graph representation of local similarity, connectivity, and global structure. GSE uses the solution of the Sylvester equation to capture both network structure and neighborhood proximity in a single representation.
  arXiv  Detail & Related papers  (2022-05-07T04:11:23Z)
• Structure-Augmented Text Representation Learning for Efficient Knowledge Graph Completion [53.31911669146451]
  Human-curated knowledge graphs provide critical supportive information to various natural language processing tasks. These graphs are usually incomplete, urging auto-completion of them. graph embedding approaches, e.g., TransE, learn structured knowledge via representing graph elements into dense embeddings. textual encoding approaches, e.g., KG-BERT, resort to graph triple's text and triple-level contextualized representations.
  arXiv  Detail & Related papers  (2020-04-30T13:50:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.