Online Density-Based Clustering for Real-Time Narrative Evolution Monitorin

• URL: http://arxiv.org/abs/2601.20680v1
• Date: Wed, 28 Jan 2026 15:07:30 GMT
• Title: Online Density-Based Clustering for Real-Time Narrative Evolution Monitorin
• Abstract summary: We investigate the replacement of HDBSCAN with online (streaming/incremental) clustering methods in a production narrative report generation pipeline.<n>This research addresses a critical gap between batch-oriented topic modeling frameworks and the streaming nature of social media monitoring.
• Score: 2.607403443115235
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Automated narrative intelligence systems for social media monitoring face significant scalability challenges when processing continuous data streams using traditional batch clustering algorithms. We investigate the replacement of HDBSCAN (offline clustering) with online (streaming/incremental) clustering methods in a production narrative report generation pipeline. The proposed system employs a three-stage architecture (data collection, modeling, dashboard generation) that processes thousands of multilingual social media documents daily. While HDBSCAN excels at discovering hierarchical density-based clusters and handling noise, its batch-only nature necessitates complete retraining for each time window, resulting in memory constraints, computational inefficiency, and inability to adapt to evolving narratives in real-time. This work evaluates a bunch of online clustering algorithms across dimensions of cluster quality preservation, computational efficiency, memory footprint, and integration compatibility with existing workflows. We propose evaluation criteria that balance traditional clustering metrics (Silhouette Coefficient, Davies-Bouldin Index) with narrative metrics (narrative distinctness, contingency and variance). Our methodology includes sliding-window simulations on historical datasets from Ukraine information space, enabling comparative analysis of algorithmic trade-offs in realistic operational contexts. This research addresses a critical gap between batch-oriented topic modeling frameworks and the streaming nature of social media monitoring, with implications for computational social science, crisis informatics, and narrative surveillance systems.

Related papers

• Amortized Neural Clustering of Time Series based on Statistical Features [7.076900172535537]
  This paper introduces an algorithm-agnostic approach to feature-based time series clustering via amortized neural inference.<n>By training neural networks to approximate the optimal partitioning rule from simulated data, the proposed framework reduces reliance on conventional clustering methods.<n>One version of the method can automatically determine the number of clusters, avoiding ad-hoc selection procedures.
  arXiv  Detail & Related papers  (2026-05-13T07:54:55Z)
• GO-GenZip: Goal-Oriented Generative Sampling and Hybrid Compression [50.76941829813458]
  Current network data telemetry pipelines consist of massive streams of fine-grained Key Performance Indicators (KPIs) from multiple distributed sources towards central aggregators.<n>This work presents a generative AI (GenAI)-driven sampling and hybrid compression framework that redesigns network telemetry from a goal-oriented perspective.
  arXiv  Detail & Related papers  (2026-03-20T16:33:15Z)
• Temporal Narrative Monitoring in Dynamic Information Environments [2.648998485166934]
  This work presents a system-oriented framework for modeling emerging narratives as temporally evolving semantic structures.<n>We apply the methodology to a real-world crisis event and evaluate system behavior through stratified cluster validation and temporal lifecycle analysis.
  arXiv  Detail & Related papers  (2026-03-18T11:30:17Z)
• Harnessing Data from Clustered LQR Systems: Personalized and Collaborative Policy Optimization [4.770545528276722]
  We show how clustering can be used in data-driven control to learn personalized policies.<n>Our work is the first to reveal how clustering can be used in data-driven control to learn personalized policies.
  arXiv  Detail & Related papers  (2025-11-21T18:45:53Z)
• In-Context Clustering with Large Language Models [50.25868718329313]
  ICC captures complex relationships among inputs through an attention mechanism.<n>We show that pretrained LLMs exhibit impressive zero-shot clustering capabilities on text-encoded numeric data.<n>Our work extends in-context learning to an unsupervised setting, showcasing the effectiveness and flexibility of LLMs for clustering.
  arXiv  Detail & Related papers  (2025-10-09T17:07:55Z)
• Adaptive and Robust DBSCAN with Multi-agent Reinforcement Learning [53.527506374566485]
  We propose a novel Adaptive and Robust DBSCAN with Multi-agent Reinforcement Learning cluster framework, namely AR-DBSCAN.<n>We show that AR-DBSCAN not only improves clustering accuracy by up to 144.1% and 175.3% in the NMI and ARI metrics, respectively, but also is capable of robustly finding dominant parameters.
  arXiv  Detail & Related papers  (2025-05-07T11:37:23Z)
• ACTGNN: Assessment of Clustering Tendency with Synthetically-Trained Graph Neural Networks [4.668678950572517]
  ACTGNN is a graph-based framework designed to assess clustering tendency by leveraging graph representations of data.<n>A Graph Neural Network (GNN) is trained exclusively on synthetic datasets, enabling robust learning of clustering structures.<n>Our results highlight the generalizability and effectiveness of the proposed approach, making it a promising tool for robust clustering tendency assessment.
  arXiv  Detail & Related papers  (2025-01-30T03:31:26Z)
• Federated Granger Causality Learning for Interdependent Clients with State Space Representation [0.6499759302108926]
  We develop a federated approach to learning Granger causality.<n>We propose augmenting the client models with the Granger causality information learned by the server.<n>We also study the convergence of the framework to a centralized oracle model.
  arXiv  Detail & Related papers  (2025-01-23T18:04:21Z)
• Resilience to the Flowing Unknown: an Open Set Recognition Framework for Data Streams [6.7236795813629]
  This work investigates the application of an Open Set Recognition framework that combines classification and clustering to address the textitover-occupied space problem in streaming scenarios.
  arXiv  Detail & Related papers  (2024-10-31T11:06:54Z)
• Contrastive Continual Multi-view Clustering with Filtered Structural Fusion [57.193645780552565]
  Multi-view clustering thrives in applications where views are collected in advance. It overlooks scenarios where data views are collected sequentially, i.e., real-time data. Some methods are proposed to handle it but are trapped in a stability-plasticity dilemma. We propose Contrastive Continual Multi-view Clustering with Filtered Structural Fusion.
  arXiv  Detail & Related papers  (2023-09-26T14:18:29Z)
• Deep Temporal Graph Clustering [77.02070768950145]
  We propose a general framework for deep Temporal Graph Clustering (GC) GC introduces deep clustering techniques to suit the interaction sequence-based batch-processing pattern of temporal graphs. Our framework can effectively improve the performance of existing temporal graph learning methods.
  arXiv  Detail & Related papers  (2023-05-18T06:17:50Z)
• Detection and Evaluation of Clusters within Sequential Data [58.720142291102135]
  Clustering algorithms for Block Markov Chains possess theoretical optimality guarantees. In particular, our sequential data is derived from human DNA, written text, animal movement data and financial markets. It is found that the Block Markov Chain model assumption can indeed produce meaningful insights in exploratory data analyses.
  arXiv  Detail & Related papers  (2022-10-04T15:22:39Z)
• Learning from Heterogeneous Data Based on Social Interactions over Graphs [58.34060409467834]
  This work proposes a decentralized architecture, where individual agents aim at solving a classification problem while observing streaming features of different dimensions. We show that the. strategy enables the agents to learn consistently under this highly-heterogeneous setting. We show that the. strategy enables the agents to learn consistently under this highly-heterogeneous setting.
  arXiv  Detail & Related papers  (2021-12-17T12:47:18Z)
• Scaling-up Distributed Processing of Data Streams for Machine Learning [10.581140430698103]
  This paper reviews recently developed methods that focus on large-scale distributed optimization in the compute- and bandwidth-limited regime. It focuses on methods that solve: (i) distributed convex problems, and (ii) distributed principal component analysis, which is a non problem with geometric structure that permits global convergence.
  arXiv  Detail & Related papers  (2020-05-18T16:28:54Z)
• ClusterVO: Clustering Moving Instances and Estimating Visual Odometry for Self and Surroundings [54.33327082243022]
  ClusterVO is a stereo Visual Odometry which simultaneously clusters and estimates the motion of both ego and surrounding rigid clusters/objects. Unlike previous solutions relying on batch input or imposing priors on scene structure or dynamic object models, ClusterVO is online, general and thus can be used in various scenarios including indoor scene understanding and autonomous driving.
  arXiv  Detail & Related papers  (2020-03-29T09:06:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.