Quantitative Evaluation of Motif Sets in Time Series

• URL: http://arxiv.org/abs/2412.09346v1
• Date: Thu, 12 Dec 2024 15:13:34 GMT
• Title: Quantitative Evaluation of Motif Sets in Time Series
• Authors: Daan Van Wesenbeeck, Aras Yurtman, Wannes Meert, Hendrik Blockeel,
• Abstract summary: This paper introduces PROM and TSMD-Bench, a benchmark for quantitative evaluation of time series motif discovery methods.<n>ProM provides a more comprehensive evaluation than existing metrics, and TSMD-Bench is a more challenging benchmark than earlier ones.<n>The proposed approach enables large-scale, systematic performance comparisons in this field.
• Score: 6.65064657162217
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Time Series Motif Discovery (TSMD), which aims at finding recurring patterns in time series, is an important task in numerous application domains, and many methods for this task exist. These methods are usually evaluated qualitatively. A few metrics for quantitative evaluation, where discovered motifs are compared to some ground truth, have been proposed, but they typically make implicit assumptions that limit their applicability. This paper introduces PROM, a broadly applicable metric that overcomes those limitations, and TSMD-Bench, a benchmark for quantitative evaluation of time series motif discovery. Experiments with PROM and TSMD-Bench show that PROM provides a more comprehensive evaluation than existing metrics, that TSMD-Bench is a more challenging benchmark than earlier ones, and that the combination can help understand the relative performance of TSMD methods. More generally, the proposed approach enables large-scale, systematic performance comparisons in this field.

Related papers

• THUNDER: Tile-level Histopathology image UNDERstanding benchmark [32.185038017473396]
  THUNDER is a tile-level benchmark for digital pathology foundation models.<n>In this paper, we provide a comprehensive comparison of 23 foundation models on 16 different datasets.
  arXiv  Detail & Related papers  (2025-07-10T15:41:35Z)
• Enhancing LLM Reasoning for Time Series Classification by Tailored Thinking and Fused Decision [8.256998757769322]
  ReasonTSC is a framework designed to leverage LLM reasoning for time series classification.<n>It steers the model to think over the essential characteristics of time series data.<n>It integrates predictions and confidence scores from plug-in classifiers, e.g., domain-specific time series models, as in-context examples.
  arXiv  Detail & Related papers  (2025-06-01T03:15:54Z)
• Multi2: Multi-Agent Test-Time Scalable Framework for Multi-Document Processing [43.75154489681047]
  We propose a novel framework leveraging test-time scaling for Multi-Document Summarization (MDS)<n>Our approach employs prompt ensemble techniques to generate multiple candidate summaries using various prompts, then combines them with an aggregator to produce a refined summary.<n>To evaluate our method effectively, we also introduce two new LLM-based metrics: the Consistency-Aware Preference (CAP) score and LLM Atom-Content-Unit (LLM-ACU) score.
  arXiv  Detail & Related papers  (2025-02-27T23:34:47Z)
• Multiscale Dubuc: A New Similarity Measure for Time Series [1.024113475677323]
  We introduce the Multiscale Dubuc Distance measure and prove that it is a metric. We use 95 datasets from the UCR Time Series Classification Archive to compare MDD's performance with EuD, LCSS, and DTW. Our experiments show that MDD's overall success, without any case-specific customization, is comparable to DTW with optimized window sizes per dataset.
  arXiv  Detail & Related papers  (2024-11-15T18:38:18Z)
• Step-by-Step Reasoning for Math Problems via Twisted Sequential Monte Carlo [55.452453947359736]
  We introduce a novel verification method based on Twisted Sequential Monte Carlo (TSMC) We apply TSMC to Large Language Models by estimating the expected future rewards at partial solutions. This approach results in a more straightforward training target that eliminates the need for step-wise human annotations.
  arXiv  Detail & Related papers  (2024-10-02T18:17:54Z)
• Unleash The Power of Pre-Trained Language Models for Irregularly Sampled Time Series [22.87452807636833]
  This work explores the potential of PLMs for ISTS analysis. We present a unified PLM-based framework, ISTS-PLM, which integrates time-aware and variable-aware PLMs for comprehensive intra and inter-time series modeling.
  arXiv  Detail & Related papers  (2024-08-12T14:22:14Z)
• TSI-Bench: Benchmarking Time Series Imputation [52.27004336123575]
  TSI-Bench is a comprehensive benchmark suite for time series imputation utilizing deep learning techniques. The TSI-Bench pipeline standardizes experimental settings to enable fair evaluation of imputation algorithms. TSI-Bench innovatively provides a systematic paradigm to tailor time series forecasting algorithms for imputation purposes.
  arXiv  Detail & Related papers  (2024-06-18T16:07:33Z)
• How to Prune Your Language Model: Recovering Accuracy on the "Sparsity May Cry'' Benchmark [60.72725673114168]
  We revisit the question of accurate BERT-pruning during fine-tuning on downstream datasets. We propose a set of general guidelines for successful pruning, even on the challenging SMC benchmark.
  arXiv  Detail & Related papers  (2023-12-21T03:11:30Z)
• LoCoMotif: Discovering time-warped motifs in time series [7.265353600305124]
  Time Series Motif Discovery (TSMD) refers to the task of identifying patterns that occur multiple times in a time series. Existing methods for TSMD have one or more of the following limitations. We present a new method, LoCoMotif, that has none of these limitations.
  arXiv  Detail & Related papers  (2023-11-29T12:18:46Z)
• InfiMM-Eval: Complex Open-Ended Reasoning Evaluation For Multi-Modal Large Language Models [50.03163753638256]
  Multi-modal Large Language Models (MLLMs) are increasingly prominent in the field of artificial intelligence. Our benchmark comprises three key reasoning categories: deductive, abductive, and analogical reasoning. We evaluate a selection of representative MLLMs using this rigorously developed open-ended multi-step elaborate reasoning benchmark.
  arXiv  Detail & Related papers  (2023-11-20T07:06:31Z)
• Exploring Progress in Multivariate Time Series Forecasting: Comprehensive Benchmarking and Heterogeneity Analysis [70.78170766633039]
  We address the need for means of assessing MTS forecasting proposals reliably and fairly. BasicTS+ is a benchmark designed to enable fair, comprehensive, and reproducible comparison of MTS forecasting solutions. We apply BasicTS+ along with rich datasets to assess the capabilities of more than 45 MTS forecasting solutions.
  arXiv  Detail & Related papers  (2023-10-09T19:52:22Z)
• TRAM: Benchmarking Temporal Reasoning for Large Language Models [12.112914393948415]
  We introduce TRAM, a temporal reasoning benchmark composed of ten datasets. We evaluate popular language models like GPT-4 and Llama2 in zero-shot and few-shot scenarios. Our findings indicate that the best-performing model lags significantly behind human performance.
  arXiv  Detail & Related papers  (2023-10-02T00:59:07Z)
• Fuzzy clustering of ordinal time series based on two novel distances with economic applications [0.12891210250935145]
  Two novel distances between ordinal time series are introduced and used to construct fuzzy clustering procedures. The resulting clustering algorithms are computationally efficient and able to group series generated from similar processes. Two specific applications involving economic time series illustrate the usefulness of the proposed approaches.
  arXiv  Detail & Related papers  (2023-04-24T16:39:22Z)
• TaSPM: Targeted Sequential Pattern Mining [53.234101208024335]
  We propose a generic framework namely TaSPM, based on the fast CM-SPAM algorithm. We also propose several pruning strategies to reduce meaningless operations in mining processes. Experiments show that the novel targeted mining algorithm TaSPM can achieve faster running time and less memory consumption.
  arXiv  Detail & Related papers  (2022-02-26T17:49:47Z)
• Learning Robust State Abstractions for Hidden-Parameter Block MDPs [55.31018404591743]
  We leverage ideas of common structure from the HiP-MDP setting to enable robust state abstractions inspired by Block MDPs. We derive instantiations of this new framework for both multi-task reinforcement learning (MTRL) and meta-reinforcement learning (Meta-RL) settings.
  arXiv  Detail & Related papers  (2020-07-14T17:25:27Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.