Domain-Agnostic Neural Topic Modeling with Contextual Token-Level Semantic Graph Representation

• URL: http://arxiv.org/abs/2608.16269v1
• Date: Mon, 17 Aug 2026 08:43:32 GMT
• Title: Domain-Agnostic Neural Topic Modeling with Contextual Token-Level Semantic Graph Representation
• Abstract summary: DARTopic is a framework that constructs token-level semantic graphs from frozen PLM embeddings and jointly trains a GNN encoder with topic inference.<n>We show DARTopic consistently outperforms strong baselines in topic coherence and document clus- tering without any encoder fine-tuning.
• Score: 5.677982677132719
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Recent advances in neural topic models with pre-trained language models (PLMs) have achieved strong performance by leveraging general-domain pre-training, yet their topic interpretability often degrades on specialized corpora. This limitation primarily stems from the geometry of the embedding space, where domain-specific terms unseen during pre-training collapse into an indistinguishable region, and neither domain-specific re-training, word-level graph enrichment, nor parameter-efficient fine-tuning can restructure this space without inheriting the capacity ceiling of the underlying encoder. Our key insight is that a learnable graph layer operating on token-level PLM embeddings can acquire corpus-specific semantic structure that the frozen encoder lacks, because token-level graphs preserve document-local context that word-level representations discard and joint optimization with the topic objective reshapes embedding geometry directly from target-domain evidence. We instantiate this insight as DARTopic, a domain-agnostic framework that constructs token-level semantic graphs from frozen PLM embeddings and jointly trains a GNN encoder with topic inference. Across three benchmarks spanning general, biomedical, and legal domains, DARTopic consistently outperforms strong baselines in topic coherence and document clus- tering without any encoder fine-tuning, while demonstrating robustness to PLM choice and favorable runtime efficiency over fine-tuning based alternatives.

Related papers

• Modality-free Graph In-context Alignment [14.497854061852292]
  Modality-Free Graph In-context Alignment (MF-GIA) is a framework that makes a pretrained graph encoder promptable for few-shot prediction across heterogeneous domains.<n>MF-GIA captures domain characteristics through gradient fingerprints, which parameterize lightweight transformations.<n>Experiments demonstrate that MF-GIA achieves superior few-shot performance across diverse graph domains.
  arXiv  Detail & Related papers  (2026-03-13T07:45:34Z)
• Rethinking Federated Graph Foundation Models: A Graph-Language Alignment-based Approach [8.517604507672262]
  Recent studies of federated graph foundational models (FedGFMs) break the idealized and untenable assumption of having centralized data storage to train graph foundation models.<n>Existing studies that project aligned generalizable knowledge onto a discrete token space via vector-quantized backbones suffer from irreversible knowledge loss during the quantization process.
  arXiv  Detail & Related papers  (2026-01-29T07:50:00Z)
• Contrastive Graph Modeling for Cross-Domain Few-Shot Medical Image Segmentation [58.41482540044918]
  Cross-domain few-shot medical image segmentation (CD-FSMIS) offers a promising and data-efficient solution for medical applications.<n>We present Contrastive Graph Modeling (C-Graph), a framework that leverages the structural consistency of medical images as a reliable domain-transferable prior.
  arXiv  Detail & Related papers  (2025-12-25T14:00:17Z)
• Enhancing Graph Representations with Neighborhood-Contextualized Message-Passing [0.0]
  This paper formalizes the concept of neighborhood-contextualization, rooted in a key property of the attentional variant.<n>A simple, practical, and efficient method to parametrize and operationalize NCMP is presented.<n>A preliminary analysis on a synthetic binary node classification problem then underscores both the expressivity and efficiency of the proposed GNN architecture.
  arXiv  Detail & Related papers  (2025-11-14T08:00:19Z)
• GeoGNN: Quantifying and Mitigating Semantic Drift in Text-Attributed Graphs [59.61242815508687]
  Graph neural networks (GNNs) on text--attributed graphs (TAGs) encode node texts using pretrained language models (PLMs) and propagate these embeddings through linear neighborhood aggregation.<n>This work introduces a local PCA-based metric that measures the degree of semantic drift and provides the first quantitative framework to analyze how different aggregation mechanisms affect manifold structure.
  arXiv  Detail & Related papers  (2025-11-12T06:48:43Z)
• GILT: An LLM-Free, Tuning-Free Graph Foundational Model for In-Context Learning [50.40400074353263]
  Graph Neural Networks (GNNs) are powerful tools for precessing relational data but often struggle to generalize to unseen graphs.<n>We introduce textbfGraph textbfIn-context textbfL textbfTransformer (GILT), a framework built on an LLM-free and tuning-free architecture.
  arXiv  Detail & Related papers  (2025-10-06T08:09:15Z)
• Predict, Cluster, Refine: A Joint Embedding Predictive Self-Supervised Framework for Graph Representation Learning [0.0]
  Graph representation learning has emerged as a cornerstone for tasks like node classification and link prediction.<n>Current self-supervised learning (SSL) methods face challenges such as computational inefficiency, reliance on contrastive objectives, and representation collapse.<n>We propose a novel joint embedding predictive framework for graph SSL that eliminates contrastive objectives and negative sampling while preserving semantic and structural information.
  arXiv  Detail & Related papers  (2025-02-02T07:42:45Z)
• Degree-Conscious Spiking Graph for Cross-Domain Adaptation [51.58506501415558]
  Spiking Graph Networks (SGNs) have demonstrated significant potential in graph classification.<n>We introduce a novel framework named Degree-Consicious Spiking Graph for Cross-Domain Adaptation (DeSGraDA)<n>DeSGraDA enhances generalization across domains with three key components.
  arXiv  Detail & Related papers  (2024-10-09T13:45:54Z)
• A Pure Transformer Pretraining Framework on Text-attributed Graphs [50.833130854272774]
  We introduce a feature-centric pretraining perspective by treating graph structure as a prior. Our framework, Graph Sequence Pretraining with Transformer (GSPT), samples node contexts through random walks. GSPT can be easily adapted to both node classification and link prediction, demonstrating promising empirical success on various datasets.
  arXiv  Detail & Related papers  (2024-06-19T22:30:08Z)
• Graph Adaptive Semantic Transfer for Cross-domain Sentiment Classification [68.06496970320595]
  Cross-domain sentiment classification (CDSC) aims to use the transferable semantics learned from the source domain to predict the sentiment of reviews in the unlabeled target domain. We present Graph Adaptive Semantic Transfer (GAST) model, an adaptive syntactic graph embedding method that is able to learn domain-invariant semantics from both word sequences and syntactic graphs.
  arXiv  Detail & Related papers  (2022-05-18T07:47:01Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.