A Hierarchical Quantized Tokenization Framework for Task-Adaptive Graph Representation Learning

• URL: http://arxiv.org/abs/2510.12369v1
• Date: Tue, 14 Oct 2025 10:36:43 GMT
• Title: A Hierarchical Quantized Tokenization Framework for Task-Adaptive Graph Representation Learning
• Authors: Yang Xiang, Li Fan, Chenke Yin, Chengtao Ji,
• Abstract summary: This work presents a hierarchical quantization framework that introduces a self-weighted mechanism for task-adaptive aggregation across multiple scales.<n> Experiments on benchmark datasets for node classification and link prediction demonstrate consistent improvements over strong baselines under comparable computational budgets.
• Score: 8.608851021844576
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Recent progress in language and vision foundation models demonstrates the importance of discrete token interfaces that transform complex inputs into compact sequences for large-scale modeling. Extending this paradigm to graphs requires a tokenization scheme that handles non-Euclidean structures and multi-scale dependencies efficiently. Existing approaches to graph tokenization, linearized, continuous, and quantized, remain limited in adaptability and efficiency. In particular, most current quantization-based tokenizers organize hierarchical information in fixed or task-agnostic ways, which may either over-represent or under-utilize structural cues, and lack the ability to dynamically reweight contributions from different levels without retraining the encoder. This work presents a hierarchical quantization framework that introduces a self-weighted mechanism for task-adaptive aggregation across multiple scales. The proposed method maintains a frozen encoder while modulating information flow through a lightweight gating process, enabling parameter-efficient adaptation to diverse downstream tasks. Experiments on benchmark datasets for node classification and link prediction demonstrate consistent improvements over strong baselines under comparable computational budgets.

Related papers

• Accelerate Speculative Decoding with Sparse Computation in Verification [49.74839681322316]
  Speculative decoding accelerates autoregressive language model inference by verifying multiple draft tokens in parallel.<n>Existing sparsification methods are designed primarily for standard token-by-token autoregressive decoding.<n>We propose a sparse verification framework that jointly sparsifies attention, FFN, and MoE components during the verification stage to reduce the dominant computation cost.
  arXiv  Detail & Related papers  (2025-12-26T07:53:41Z)
• Improving Pattern Recognition of Scheduling Anomalies through Structure-Aware and Semantically-Enhanced Graphs [9.545534210398964]
  This paper proposes a structure-aware driven scheduling graph modeling method.<n>It integrates task execution stages, resource node states, and scheduling path information to build dynamically evolving scheduling behavior graphs.<n>Under the combined effect of structure guidance and semantic aggregation, the scheduling behavior graph exhibits stronger anomaly separability and pattern representation.
  arXiv  Detail & Related papers  (2025-12-21T10:27:06Z)
• MergeDNA: Context-aware Genome Modeling with Dynamic Tokenization through Token Merging [65.07273789940116]
  This paper introduces a hierarchical architecture that jointly optimize a dynamic genomic tokenizer and latent Transformers with context-aware pre-training tasks.<n> MergeDNA achieves superior performance on three popular DNA benchmarks and several multi-omics tasks with fine-tuning or zero-shot evaluation.
  arXiv  Detail & Related papers  (2025-11-17T19:27:41Z)
• Structural Priors and Modular Adapters in the Composable Fine-Tuning Algorithm of Large-Scale Models [8.15127799301814]
  This paper proposes a composable fine-tuning method that integrates graph structural priors with modular adapters.<n>The proposed framework significantly enhances task prediction accuracy, adapter weight allocation precision, and overall computational efficiency.
  arXiv  Detail & Related papers  (2025-11-06T02:10:20Z)
• Neural Network Reprogrammability: A Unified Theme on Model Reprogramming, Prompt Tuning, and Prompt Instruction [55.914891182214475]
  We introduce neural network reprogrammability as a unifying framework for model adaptation.<n>We present a taxonomy that categorizes such information manipulation approaches across four key dimensions.<n>We also analyze remaining technical challenges and ethical considerations.
  arXiv  Detail & Related papers  (2025-06-05T05:42:27Z)
• Structured Convergence in Large Language Model Representations via Hierarchical Latent Space Folding [0.0]
  Token representations in high-dimensional latent spaces often exhibit redundancy, limiting computational efficiency and reducing structural coherence across model layers.<n>This paper introduces a structured transformation mechanism that enforces a multi-scale organization within learned embeddings.<n> Empirical evaluation demonstrates a reduction in representational variance across layers, contributing to more stable perplexity distributions and enhancing predictive confidence in text generation.
  arXiv  Detail & Related papers  (2025-02-13T04:01:54Z)
• Contextual Reinforcement in Multimodal Token Compression for Large Language Models [0.0]
  token compression remains a critical challenge for scaling models to handle increasingly complex and diverse datasets.<n>A novel mechanism based on contextual reinforcement is introduced, dynamically adjusting token importance through interdependencies and semantic relevance.<n>This approach enables substantial reductions in token usage while preserving the quality and coherence of information representation.
  arXiv  Detail & Related papers  (2025-01-28T02:44:31Z)
• Autonomous Structural Memory Manipulation for Large Language Models Using Hierarchical Embedding Augmentation [0.0]
  This study introduces hierarchical embedding augmentation as a means to redefine the representation of tokens through multi-level semantic structures.<n>Results reveal substantial improvements in computational efficiency, with marked reductions in processing overhead for longer input sequences.<n>The ability to dynamically adjust token representations and memory configurations contributed to the model's robustness under varied and unpredictable input conditions.
  arXiv  Detail & Related papers  (2025-01-23T22:20:36Z)
• Structural Entropy Guided Probabilistic Coding [52.01765333755793]
  We propose a novel structural entropy-guided probabilistic coding model, named SEPC.<n>We incorporate the relationship between latent variables into the optimization by proposing a structural entropy regularization loss.<n> Experimental results across 12 natural language understanding tasks, including both classification and regression tasks, demonstrate the superior performance of SEPC.
  arXiv  Detail & Related papers  (2024-12-12T00:37:53Z)
• T-GAE: Transferable Graph Autoencoder for Network Alignment [79.89704126746204]
  T-GAE is a graph autoencoder framework that leverages transferability and stability of GNNs to achieve efficient network alignment without retraining. Our experiments demonstrate that T-GAE outperforms the state-of-the-art optimization method and the best GNN approach by up to 38.7% and 50.8%, respectively.
  arXiv  Detail & Related papers  (2023-10-05T02:58:29Z)
• Disentanglement via Latent Quantization [60.37109712033694]
  In this work, we construct an inductive bias towards encoding to and decoding from an organized latent space. We demonstrate the broad applicability of this approach by adding it to both basic data-re (vanilla autoencoder) and latent-reconstructing (InfoGAN) generative models.
  arXiv  Detail & Related papers  (2023-05-28T06:30:29Z)
• Adaptive Discrete Communication Bottlenecks with Dynamic Vector Quantization [76.68866368409216]
  We propose learning to dynamically select discretization tightness conditioned on inputs. We show that dynamically varying tightness in communication bottlenecks can improve model performance on visual reasoning and reinforcement learning tasks.
  arXiv  Detail & Related papers  (2022-02-02T23:54:26Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.