Comment-level Topic Drift Analysis in the Reddit Corpus

• URL: http://arxiv.org/abs/2608.19133v1
• Date: Wed, 19 Aug 2026 17:27:25 GMT
• Title: Comment-level Topic Drift Analysis in the Reddit Corpus
• Abstract summary: We present a novel application of embedding-based dynamic topic modeling techniques to detect and quantify topic drift at the comment level in a massive corpus.<n>By leveraging pretrained language models to generate contextualized semantic embeddings for short text, we analyzed 12.7 billion Reddit comments spanning 2006 to 2022.<n>Key findings suggest that politically and socially contentious topics exhibit significant directional drift in embedding space.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We present a novel application of embedding-based dynamic topic modeling techniques to detect and quantify topic drift at the comment level in a massive corpus. By leveraging pretrained language models to generate contextualized semantic embeddings for short text, we analyzed 12.7 billion Reddit comments spanning 2006 to 2022. Using unsupervised methods on these embeddings, we identify dynamically evolving topic clusters over time. Our primary contribution is a methodology for analysis of semantic drift and discourse evolution in the embedding space itself. We also demonstrate modifications to existing methods that enable this analysis at scale, and we propose and demonstrate a null model comparison test to filter spurious dynamics. Key findings suggest that politically and socially contentious topics exhibit significant directional drift in embedding space, with inter-topic distances changing systematically over time beyond what the null model can explain, whereas domains such as music and sports remain comparatively stable.

Related papers

• Scale-Dependent Semantic Dynamics Revealed by Allan Deviation [0.0]
  We analyze the stability of meaning by treating ordered sentence embeddings as a displacement signal.<n>We find that while large language models successfully mimic the local scaling statistics of human text, they exhibit a systematic reduction in their stability horizon.
  arXiv  Detail & Related papers  (2026-01-29T13:10:59Z)
• Dynamic Topic Evolution with Temporal Decay and Attention in Large Language Models [3.4219049032524804]
  This paper proposes a modeling framework for dynamic topic evolution based on temporal large language models.<n>The proposed method provides a systematic solution for understanding dynamic semantic patterns in large-scale text.
  arXiv  Detail & Related papers  (2025-10-12T13:50:41Z)
• Automated Sentiment Classification and Topic Discovery in Large-Scale Social Media Streams [3.5279571333221913]
  We present a framework for large-scale sentiment and topic analysis of Twitter discourse.<n>Our pipeline begins with targeted data collection using conflict-specific keywords.<n>We examine the relationship between sentiment and contextual features such as timestamp, geolocation, and lexical content.
  arXiv  Detail & Related papers  (2025-05-03T18:04:57Z)
• Sequential Representation Learning via Static-Dynamic Conditional Disentanglement [58.19137637859017]
  This paper explores self-supervised disentangled representation learning within sequential data, focusing on separating time-independent and time-varying factors in videos. We propose a new model that breaks the usual independence assumption between those factors by explicitly accounting for the causal relationship between the static/dynamic variables. Experiments show that the proposed approach outperforms previous complex state-of-the-art techniques in scenarios where the dynamics of a scene are influenced by its content.
  arXiv  Detail & Related papers  (2024-08-10T17:04:39Z)
• Interactive Topic Models with Optimal Transport [75.26555710661908]
  We present EdTM, as an approach for label name supervised topic modeling. EdTM models topic modeling as an assignment problem while leveraging LM/LLM based document-topic affinities.
  arXiv  Detail & Related papers  (2024-06-28T13:57:27Z)
• TopicAdapt- An Inter-Corpora Topics Adaptation Approach [27.450275637652418]
  This paper proposes a neural topic model, TopicAdapt, that can adapt relevant topics from a related source corpus and also discover new topics in a target corpus that are absent in the source corpus. Experiments over multiple datasets from diverse domains show the superiority of the proposed model against the state-of-the-art topic models.
  arXiv  Detail & Related papers  (2023-10-08T02:56:44Z)
• Inverse Dynamics Pretraining Learns Good Representations for Multitask Imitation [66.86987509942607]
  We evaluate how such a paradigm should be done in imitation learning. We consider a setting where the pretraining corpus consists of multitask demonstrations. We argue that inverse dynamics modeling is well-suited to this setting.
  arXiv  Detail & Related papers  (2023-05-26T14:40:46Z)
• Topics in the Haystack: Extracting and Evaluating Topics beyond Coherence [0.0]
  We propose a method that incorporates a deeper understanding of both sentence and document themes. This allows our model to detect latent topics that may include uncommon words or neologisms. We present correlation coefficients with human identification of intruder words and achieve near-human level results at the word-intrusion task.
  arXiv  Detail & Related papers  (2023-03-30T12:24:25Z)
• Are Neural Topic Models Broken? [81.15470302729638]
  We study the relationship between automated and human evaluation of topic models. We find that neural topic models fare worse in both respects compared to an established classical method.
  arXiv  Detail & Related papers  (2022-10-28T14:38:50Z)
• Incorporating Dynamic Semantics into Pre-Trained Language Model for Aspect-based Sentiment Analysis [67.41078214475341]
  We propose Dynamic Re-weighting BERT (DR-BERT) to learn dynamic aspect-oriented semantics for ABSA. Specifically, we first take the Stack-BERT layers as a primary encoder to grasp the overall semantic of the sentence. We then fine-tune it by incorporating a lightweight Dynamic Re-weighting Adapter (DRA)
  arXiv  Detail & Related papers  (2022-03-30T14:48:46Z)
• How Far are We from Effective Context Modeling? An Exploratory Study on Semantic Parsing in Context [59.13515950353125]
  We present a grammar-based decoding semantic parsing and adapt typical context modeling methods on top of it. We evaluate 13 context modeling methods on two large cross-domain datasets, and our best model achieves state-of-the-art performances.
  arXiv  Detail & Related papers  (2020-02-03T11:28:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.