Neural Total Variation Distance Estimators for Changepoint Detection in News Data

• URL: http://arxiv.org/abs/2506.18764v1
• Date: Mon, 23 Jun 2025 15:33:30 GMT
• Title: Neural Total Variation Distance Estimators for Changepoint Detection in News Data
• Authors: Csaba Zsolnai, Niels Lörch, Julian Arnold,
• Abstract summary: We leverage neural networks for changepoint detection in news data, introducing a method based on the so-called learning-by-confusion scheme.<n>We demonstrate the effectiveness of this method on both synthetic datasets and real-world data from The Guardian newspaper.<n>Our approach requires minimal domain knowledge, can autonomously discover significant shifts in public discourse, and yields a quantitative measure of change in content.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Detecting when public discourse shifts in response to major events is crucial for understanding societal dynamics. Real-world data is high-dimensional, sparse, and noisy, making changepoint detection in this domain a challenging endeavor. In this paper, we leverage neural networks for changepoint detection in news data, introducing a method based on the so-called learning-by-confusion scheme, which was originally developed for detecting phase transitions in physical systems. We train classifiers to distinguish between articles from different time periods. The resulting classification accuracy is used to estimate the total variation distance between underlying content distributions, where significant distances highlight changepoints. We demonstrate the effectiveness of this method on both synthetic datasets and real-world data from The Guardian newspaper, successfully identifying major historical events including 9/11, the COVID-19 pandemic, and presidential elections. Our approach requires minimal domain knowledge, can autonomously discover significant shifts in public discourse, and yields a quantitative measure of change in content, making it valuable for journalism, policy analysis, and crisis monitoring.

Related papers

• A Dataset for Semantic Segmentation in the Presence of Unknowns [49.795683850385956]
  Existing datasets allow evaluation of only knowns or unknowns - but not both.<n>We propose a novel anomaly segmentation dataset, ISSU, that features a diverse set of anomaly inputs from cluttered real-world environments.<n>The dataset is twice larger than existing anomaly segmentation datasets.
  arXiv  Detail & Related papers  (2025-03-28T10:31:01Z)
• PeFAD: A Parameter-Efficient Federated Framework for Time Series Anomaly Detection [51.20479454379662]
  We propose a. Federated Anomaly Detection framework named PeFAD with the increasing privacy concerns. We conduct extensive evaluations on four real datasets, where PeFAD outperforms existing state-of-the-art baselines by up to 28.74%.
  arXiv  Detail & Related papers  (2024-06-04T13:51:08Z)
• Change points detection in crime-related time series: an on-line fuzzy approach based on a shape space representation [0.0]
  We propose an on-line method for detecting and querying change points in crime-related time series. The method is able to accurately detect change points at very low computational costs.
  arXiv  Detail & Related papers  (2023-12-18T10:49:03Z)
• Domain Adaptive Synapse Detection with Weak Point Annotations [63.97144211520869]
  We present AdaSyn, a framework for domain adaptive synapse detection with weak point annotations. In the WASPSYN challenge at I SBI 2023, our method ranks the 1st place.
  arXiv  Detail & Related papers  (2023-08-31T05:05:53Z)
• Automatic Change-Point Detection in Time Series via Deep Learning [8.43086628139493]
  We show how to automatically generate new offline detection methods based on training a neural network. We present theory that quantifies the error rate for such an approach, and how it depends on the amount of training data. Our method also shows strong results in detecting and localising changes in activity based on accelerometer data.
  arXiv  Detail & Related papers  (2022-11-07T20:59:14Z)
• High dimensional change-point detection: a complete graph approach [0.0]
  We propose a complete graph-based, change-point detection algorithm to detect change of mean and variance from low to high-dimensional online data. Inspired by complete graph structure, we introduce graph-spanning ratios to map high-dimensional data into metrics. Our approach has high detection power with small and multiple scanning window, which allows timely detection of change-point in the online setting.
  arXiv  Detail & Related papers  (2022-03-16T15:59:20Z)
• On Generalizing Beyond Domains in Cross-Domain Continual Learning [91.56748415975683]
  Deep neural networks often suffer from catastrophic forgetting of previously learned knowledge after learning a new task. Our proposed approach learns new tasks under domain shift with accuracy boosts up to 10% on challenging datasets such as DomainNet and OfficeHome.
  arXiv  Detail & Related papers  (2022-03-08T09:57:48Z)
• WATCH: Wasserstein Change Point Detection for High-Dimensional Time Series Data [4.228718402877829]
  Change point detection methods have the ability to discover changes in an unsupervised fashion. We propose WATCH, a novel Wasserstein distance-based change point detection approach. An extensive evaluation shows that WATCH is capable of accurately identifying change points and outperforming state-of-the-art methods.
  arXiv  Detail & Related papers  (2022-01-18T16:55:29Z)
• Changepoint Analysis of Topic Proportions in Temporal Text Data [1.8262547855491456]
  We build a specialised temporal topic model with provisions for changepoints in the distribution of topic proportions. We use sample splitting to estimate topic polytopes first and then apply a likelihood ratio statistic. We obtain some historically well-known changepoints and discover some new ones.
  arXiv  Detail & Related papers  (2021-11-29T17:20:51Z)
• Combating Temporal Drift in Crisis with Adapted Embeddings [58.4558720264897]
  Language usage changes over time, and this can impact the effectiveness of NLP systems. This work investigates methods for adapting to changing discourse during crisis events.
  arXiv  Detail & Related papers  (2021-04-17T13:11:41Z)
• Pretrained equivariant features improve unsupervised landmark discovery [69.02115180674885]
  We formulate a two-step unsupervised approach that overcomes this challenge by first learning powerful pixel-based features. Our method produces state-of-the-art results in several challenging landmark detection datasets.
  arXiv  Detail & Related papers  (2021-04-07T05:42:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.