Beyond Top Words: MonoTM for Topic Modeling with Interpretable Monosemantic Features

• URL: http://arxiv.org/abs/2609.09575v1
• Date: Wed, 09 Sep 2026 00:54:46 GMT
• Title: Beyond Top Words: MonoTM for Topic Modeling with Interpretable Monosemantic Features
• Abstract summary: We introduce textbfMonoTM, an interpretable topic modeling framework.<n>We show that document--topic mixture estimation and semantic interpretation favor different SAE configurations.<n>MonoTM estimates mixtures from the full SAE bag-of-features representation and, with them fixed, learns topic descriptors over a separate vocabulary of corpus-grounded semantic features.
• Score: 11.011725742250283
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Topic models summarize large text corpora, but top-ranked words often provide only a limited representation of topic semantics. Sparse autoencoders (SAEs) offer a way to move beyond word-level descriptors by extracting interpretable features from dense representations, yet how feature interpretability relates to topic-inference quality remains unclear. We introduce \textbf{MonoTM}, an interpretable topic modeling framework that decouples these roles. Across three benchmark corpora, we show that document--topic mixture estimation and semantic interpretation favor different SAE configurations and feature subsets. MonoTM estimates mixtures from the full SAE bag-of-features representation and, with them fixed, learns topic descriptors over a separate vocabulary of corpus-grounded semantic features. This design preserves global topic structure while representing topics with semantic units more meaningful than individual words, making them more useful for downstream corpus analysis.

Related papers

• Does Syntax Matter? A Graph-Augmented Variational Topic Model for Computational Social Sciences [51.56484100374058]
  This paper introduces the Structural Contextual Probabilistic Topic Model ( SCPTM), an architecture that incorporates syntactic dependency relations into topic inference.<n>We evaluate seven topic modeling techniques across four corpora differing in register and discourse structure.<n>Results show that SCPTM's neural architecture yields substantial gains in document-topic alignment over generative baselines.
  arXiv  Detail & Related papers  (2026-09-07T17:39:06Z)
• Speech Codec Probing from Semantic and Phonetic Perspectives [49.01048570474675]
  Speech tokenizers are essential for connecting speech to large language models (LLMs) in multimodal systems.<n> emerging evidence suggests that what is termed "semantic" in speech representations does not align with text-derived semantics.
  arXiv  Detail & Related papers  (2026-03-11T03:32:25Z)
• Semantic Regexes: Auto-Interpreting LLM Features with a Structured Language [29.636642657652455]
  We introduce semantices, structured language descriptions of large language model (LLM) features.<n>We find that semantices match the accuracy of natural language while yielding more concise and consistent feature descriptions.
  arXiv  Detail & Related papers  (2025-10-07T18:56:45Z)
• Question-Driven Analysis and Synthesis: Building Interpretable Thematic Trees with LLMs for Text Clustering and Controllable Generation [1.3750624267664158]
  We introduce Recursive Thematic Partitioning (RTP) to interactively build a binary tree.<n>Each node in the tree is a natural language question that semantically partitions the data, resulting in a fully interpretable taxonomy.<n>We show that RTP's question-driven hierarchy is more interpretable than the keyword-based topics from a strong baseline like BERTopic.
  arXiv  Detail & Related papers  (2025-09-26T11:27:22Z)
• Model Directions, Not Words: Mechanistic Topic Models Using Sparse Autoencoders [21.650904669171524]
  Mechanistic Topic Models (MTMs) operate on interpretable features learned by sparse autoencoders (SAEs)<n>MTMs enable controllable text generation using topic-based steering vectors.<n>To properly evaluate MTM topics against word-list-based approaches, we propose textittopic judge, an LLM-based pairwise comparison evaluation framework.
  arXiv  Detail & Related papers  (2025-07-31T03:17:43Z)
• Interactive Topic Models with Optimal Transport [75.26555710661908]
  We present EdTM, as an approach for label name supervised topic modeling. EdTM models topic modeling as an assignment problem while leveraging LM/LLM based document-topic affinities.
  arXiv  Detail & Related papers  (2024-06-28T13:57:27Z)
• Spatial Semantic Recurrent Mining for Referring Image Segmentation [63.34997546393106]
  We propose Stextsuperscript2RM to achieve high-quality cross-modality fusion. It follows a working strategy of trilogy: distributing language feature, spatial semantic recurrent coparsing, and parsed-semantic balancing. Our proposed method performs favorably against other state-of-the-art algorithms.
  arXiv  Detail & Related papers  (2024-05-15T00:17:48Z)
• A General and Flexible Multi-concept Parsing Framework for Multilingual Semantic Matching [60.51839859852572]
  We propose to resolve the text into multi concepts for multilingual semantic matching to liberate the model from the reliance on NER models. We conduct comprehensive experiments on English datasets QQP and MRPC, and Chinese dataset Medical-SM.
  arXiv  Detail & Related papers  (2024-03-05T13:55:16Z)
• Prompting Large Language Models for Topic Modeling [10.31712610860913]
  We propose PromptTopic, a novel topic modeling approach that harnesses the advanced language understanding of large language models (LLMs) It involves extracting topics at the sentence level from individual documents, then aggregating and condensing these topics into a predefined quantity, ultimately providing coherent topics for texts of varying lengths. We benchmark PromptTopic against the state-of-the-art baselines on three vastly diverse datasets, establishing its proficiency in discovering meaningful topics.
  arXiv  Detail & Related papers  (2023-12-15T11:15:05Z)
• Syntax and Semantics Meet in the "Middle": Probing the Syntax-Semantics Interface of LMs Through Agentivity [68.8204255655161]
  We present the semantic notion of agentivity as a case study for probing such interactions. This suggests LMs may potentially serve as more useful tools for linguistic annotation, theory testing, and discovery.
  arXiv  Detail & Related papers  (2023-05-29T16:24:01Z)
• Coordinated Topic Modeling [10.710176350043998]
  We propose a new problem called coordinated topic modeling that imitates human behavior while describing a text corpus. We design ECTM, an embedding-based coordinated topic model that effectively uses the reference representation to capture the target corpus-specific aspects. In ECTM, we introduce the topic- and document-level supervision with a self-training mechanism to solve the problem.
  arXiv  Detail & Related papers  (2022-10-16T15:10:54Z)
• Dialogue Meaning Representation for Task-Oriented Dialogue Systems [51.91615150842267]
  We propose Dialogue Meaning Representation (DMR), a flexible and easily extendable representation for task-oriented dialogue. Our representation contains a set of nodes and edges with inheritance hierarchy to represent rich semantics for compositional semantics and task-specific concepts. We propose two evaluation tasks to evaluate different machine learning based dialogue models, and further propose a novel coreference resolution model GNNCoref for the graph-based coreference resolution task.
  arXiv  Detail & Related papers  (2022-04-23T04:17:55Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.