HIGHT: Hierarchical Graph Tokenization for Graph-Language Alignment

• URL: http://arxiv.org/abs/2406.14021v1
• Date: Thu, 20 Jun 2024 06:37:35 GMT
• Title: HIGHT: Hierarchical Graph Tokenization for Graph-Language Alignment
• Authors: Yongqiang Chen, Quanming Yao, Juzheng Zhang, James Cheng, Yatao Bian,
• Abstract summary: We propose a novel strategy called HIerarchical GrapH Tokenization (HIGHT) to improve the graph perception of large language models (LLMs) HIGHT employs a hierarchical graph tokenizer that extracts and encodes the hierarchy of node, motif, and graph levels of informative tokens to improve the graph perception of LLMs. Experiments on 7 molecule-centric benchmarks confirm the effectiveness of HIGHT in reducing hallucination by 40%, as well as significant improvements in various molecule-language downstream tasks.
• Score: 41.75926736949724
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Recently there has been a surge of interest in extending the success of large language models (LLMs) to graph modality, such as social networks and molecules. As LLMs are predominantly trained with 1D text data, most existing approaches adopt a graph neural network to represent a graph as a series of node tokens and feed these tokens to LLMs for graph-language alignment. Despite achieving some successes, existing approaches have overlooked the hierarchical structures that are inherent in graph data. Especially, in molecular graphs, the high-order structural information contains rich semantics of molecular functional groups, which encode crucial biochemical functionalities of the molecules. We establish a simple benchmark showing that neglecting the hierarchical information in graph tokenization will lead to subpar graph-language alignment and severe hallucination in generated outputs. To address this problem, we propose a novel strategy called HIerarchical GrapH Tokenization (HIGHT). HIGHT employs a hierarchical graph tokenizer that extracts and encodes the hierarchy of node, motif, and graph levels of informative tokens to improve the graph perception of LLMs. HIGHT also adopts an augmented graph-language supervised fine-tuning dataset, enriched with the hierarchical graph information, to further enhance the graph-language alignment. Extensive experiments on 7 molecule-centric benchmarks confirm the effectiveness of HIGHT in reducing hallucination by 40%, as well as significant improvements in various molecule-language downstream tasks.

Related papers

• Bridging Molecular Graphs and Large Language Models [10.647911401603801]
  Large Language Models (LLMs) have shown exceptional generalization capabilities, but their ability to process graph data, such as molecular structures, remains limited.<n>This paper proposes Graph2Token, an efficient solution that aligns graph tokens to LLM tokens.<n>Extensive experiments on molecular classification and regression tasks demonstrate the effectiveness of our proposed Graph2Token.
  arXiv  Detail & Related papers  (2025-03-05T03:15:38Z)
• Knowledge-aware contrastive heterogeneous molecular graph learning [77.94721384862699]
  We propose a paradigm shift by encoding molecular graphs into Heterogeneous Molecular Graph Learning (KCHML)<n>KCHML conceptualizes molecules through three distinct graph views-molecular, elemental, and pharmacological-enhanced by heterogeneous molecular graphs and a dual message-passing mechanism.<n>This design offers a comprehensive representation for property prediction, as well as for downstream tasks such as drug-drug interaction (DDI) prediction.
  arXiv  Detail & Related papers  (2025-02-17T11:53:58Z)
• Automatic Annotation Augmentation Boosts Translation between Molecules and Natural Language [7.458295743918249]
  This paper introduces LA$3$, a Language-based Automatic Augmentation framework that leverages large language models to augment existing datasets.<n>We demonstrate the effectiveness of LA$3$ by creating an enhanced dataset, LaChEBI-20, where we rewrite the annotations of molecules from an established dataset.<n>We train LaMolT5 based on a benchmark architecture to learn the mapping between molecular representations and augmented annotations.
  arXiv  Detail & Related papers  (2025-02-10T16:29:21Z)
• Neural Graph Pattern Machine [50.78679002846741]
  We propose the Neural Graph Pattern Machine (GPM), a framework designed to learn directly from graph patterns. GPM efficiently extracts and encodes substructures while identifying the most relevant ones for downstream tasks.
  arXiv  Detail & Related papers  (2025-01-30T20:37:47Z)
• Query-Aware Learnable Graph Pooling Tokens as Prompt for Large Language Models [3.9489815622117566]
  Learnable Graph Pooling Token (LGPT) enables flexible and efficient graph representation. Our method achieves a 4.13% performance improvement on the GraphQA benchmark without training the large language model.
  arXiv  Detail & Related papers  (2025-01-29T10:35:41Z)
• A Hierarchical Language Model For Interpretable Graph Reasoning [47.460255447561906]
  We introduce Hierarchical Language Model for Graph (HLM-G), which employs a two-block architecture to capture node-centric local information and interaction-centric global structure. The proposed scheme allows LLMs to address various graph queries with high efficacy, efficiency, and robustness, while reducing computational costs on large-scale graph tasks. Comprehensive evaluations across diverse graph reasoning and real-world tasks of node, link, and graph-levels highlight the superiority of our method.
  arXiv  Detail & Related papers  (2024-10-29T00:28:02Z)
• Graph Linearization Methods for Reasoning on Graphs with Large Language Models [25.3545522174459]
  Graphs should be linearized to reflect certain properties of natural language text, such as local dependency and global alignment. We develop several graph linearization methods based on graph centrality, degeneracy, and node relabeling schemes. Our work introduces novel graph representations suitable for LLMs, contributing to the potential integration of graph machine learning with the trend of multi-modal processing.
  arXiv  Detail & Related papers  (2024-10-25T11:51:37Z)
• What Do LLMs Need to Understand Graphs: A Survey of Parametric Representation of Graphs [69.48708136448694]
  Large language models (LLMs) are reorganizing in the AI community for their expected reasoning and inference abilities. We believe this kind of parametric representation of graphs, graph laws, can be a solution for making LLMs understand graph data as the input.
  arXiv  Detail & Related papers  (2024-10-16T00:01:31Z)
• Dynamic and Textual Graph Generation Via Large-Scale LLM-based Agent Simulation [70.60461609393779]
  GraphAgent-Generator (GAG) is a novel simulation-based framework for dynamic graph generation. Our framework effectively replicates seven macro-level structural characteristics in established network science theories. It supports generating graphs with up to nearly 100,000 nodes or 10 million edges, with a minimum speed-up of 90.4%.
  arXiv  Detail & Related papers  (2024-10-13T12:57:08Z)
• FARM: Functional Group-Aware Representations for Small Molecules [55.281754551202326]
  We introduce Functional Group-Aware Representations for Small Molecules (FARM)<n>FARM is a novel model designed to bridge the gap between SMILES, natural language, and molecular graphs.<n>We evaluate FARM on the MoleculeNet dataset, where it achieves state-of-the-art performance on 11 out of 13 tasks.
  arXiv  Detail & Related papers  (2024-10-02T23:04:58Z)
• Data-Efficient Molecular Generation with Hierarchical Textual Inversion [48.816943690420224]
  We introduce Hierarchical textual Inversion for Molecular generation (HI-Mol), a novel data-efficient molecular generation method. HI-Mol is inspired by the importance of hierarchical information, e.g., both coarse- and fine-grained features, in understanding the molecule distribution. Compared to the conventional textual inversion method in the image domain using a single-level token embedding, our multi-level token embeddings allow the model to effectively learn the underlying low-shot molecule distribution.
  arXiv  Detail & Related papers  (2024-05-05T08:35:23Z)
• GraphEdit: Large Language Models for Graph Structure Learning [62.618818029177355]
  Graph Structure Learning (GSL) focuses on capturing intrinsic dependencies and interactions among nodes in graph-structured data. Existing GSL methods heavily depend on explicit graph structural information as supervision signals. We propose GraphEdit, an approach that leverages large language models (LLMs) to learn complex node relationships in graph-structured data.
  arXiv  Detail & Related papers  (2024-02-23T08:29:42Z)
• G-Retriever: Retrieval-Augmented Generation for Textual Graph Understanding and Question Answering [61.93058781222079]
  We develop a flexible question-answering framework targeting real-world textual graphs. We introduce the first retrieval-augmented generation (RAG) approach for general textual graphs. G-Retriever performs RAG over a graph by formulating this task as a Prize-Collecting Steiner Tree optimization problem.
  arXiv  Detail & Related papers  (2024-02-12T13:13:04Z)
• GraphGPT: Graph Instruction Tuning for Large Language Models [27.036935149004726]
  Graph Neural Networks (GNNs) have evolved to understand graph structures. To enhance robustness, self-supervised learning (SSL) has become a vital tool for data augmentation. Our research tackles this by advancing graph model generalization in zero-shot learning environments.
  arXiv  Detail & Related papers  (2023-10-19T06:17:46Z)
• MolGrapher: Graph-based Visual Recognition of Chemical Structures [50.13749978547401]
  We introduce MolGrapher to recognize chemical structures visually. We treat all candidate atoms and bonds as nodes and put them in a graph. We classify atom and bond nodes in the graph with a Graph Neural Network.
  arXiv  Detail & Related papers  (2023-08-23T16:16:11Z)
• Structure-free Graph Condensation: From Large-scale Graphs to Condensed Graph-free Data [91.27527985415007]
  Existing graph condensation methods rely on the joint optimization of nodes and structures in the condensed graph. We advocate a new Structure-Free Graph Condensation paradigm, named SFGC, to distill a large-scale graph into a small-scale graph node set.
  arXiv  Detail & Related papers  (2023-06-05T07:53:52Z)
• GraphMAE: Self-Supervised Masked Graph Autoencoders [52.06140191214428]
  We present a masked graph autoencoder GraphMAE that mitigates issues for generative self-supervised graph learning. We conduct extensive experiments on 21 public datasets for three different graph learning tasks. The results manifest that GraphMAE--a simple graph autoencoder with our careful designs--can consistently generate outperformance over both contrastive and generative state-of-the-art baselines.
  arXiv  Detail & Related papers  (2022-05-22T11:57:08Z)
• Molecular Contrastive Learning with Chemical Element Knowledge Graph [16.136921143416927]
  Molecular representation learning contributes to multiple downstream tasks such as molecular property prediction and drug design. We construct a Chemical Element Knowledge Graph (KG) to summarize microscopic associations between elements. The first module, knowledge-guided graph augmentation, augments the original molecular graph based on the Chemical Element KG. The second module, knowledge-aware graph representation, extracts molecular representations with a common graph encoder for the original molecular graph and a Knowledge-aware Message Passing Neural Network (KMPNN) to encode complex information in the augmented molecular graph.
  arXiv  Detail & Related papers  (2021-12-01T15:04:39Z)
• Multi-Level Graph Contrastive Learning [38.022118893733804]
  We propose a Multi-Level Graph Contrastive Learning (MLGCL) framework for learning robust representation of graph data by contrasting space views of graphs. The original graph is first-order approximation structure and contains uncertainty or error, while the $k$NN graph generated by encoding features preserves high-order proximity. Extensive experiments indicate MLGCL achieves promising results compared with the existing state-of-the-art graph representation learning methods on seven datasets.
  arXiv  Detail & Related papers  (2021-07-06T14:24:43Z)
• MolCLR: Molecular Contrastive Learning of Representations via Graph Neural Networks [11.994553575596228]
  MolCLR is a self-supervised learning framework for large unlabeled molecule datasets. We propose three novel molecule graph augmentations: atom masking, bond deletion, and subgraph removal. Our method achieves state-of-the-art performance on many challenging datasets.
  arXiv  Detail & Related papers  (2021-02-19T17:35:18Z)
• Reinforced Molecular Optimization with Neighborhood-Controlled Grammars [63.84003497770347]
  We propose MNCE-RL, a graph convolutional policy network for molecular optimization. We extend the original neighborhood-controlled embedding grammars to make them applicable to molecular graph generation. We show that our approach achieves state-of-the-art performance in a diverse range of molecular optimization tasks.
  arXiv  Detail & Related papers  (2020-11-14T05:42:15Z)
• Hierarchical Inter-Message Passing for Learning on Molecular Graphs [9.478108870211365]
  We present a hierarchical neural message passing architecture for learning on molecular graphs. Our method is able to overcome some of the restrictions known from classical GNNs, like detecting cycles, while still being very efficient to train.
  arXiv  Detail & Related papers  (2020-06-22T12:25:24Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.