How well does Classification Accuracy capture Concept Drift Detection Quality? An overview of Concept Drift Detection evaluation

• URL: http://arxiv.org/abs/2605.31186v1
• Date: Fri, 29 May 2026 11:55:23 GMT
• Title: How well does Classification Accuracy capture Concept Drift Detection Quality? An overview of Concept Drift Detection evaluation
• Abstract summary: We present an in-depth overview of the relationship between metrics for quantifying drift detection quality and classification performance.<n>Study aims to identify the most informative set of drift detection quality metrics and provide a deep understanding of the method's evaluation.
• Score: 0.2538209532048867
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Data streams are nowadays among the most frequently analyzed data structures, with the concept drift posing a major challenge encountered by processing systems. Despite the proposition of numerous solutions to counteract the accuracy degeneration due to concept drift, the scientific community has not yet established a unified framework for evaluating the concept drift detection task. Existing research often relies on classification quality metrics, but these can be affected by multiple factors and may not reliably reflect drift detection quality. In this work, we present an in-depth overview of the relationship between metrics for quantifying drift detection quality and classification performance in synthetic nonstationary data streams. The proposed research studies eight drift detection quality metrics in relation to the classifier's performance across seven synthetic data stream generation tools, additionally considering drift dynamics as a factor. The studies aim to identify the most informative set of drift detection quality metrics and provide a deep understanding of the method's evaluation.

Related papers

• Robust Slip Detection and Material Classification via Spatiotemporal Transformers on a Uniformly-Illuminated Visuo-Tactile Sensor [52.33204893988409]
  We propose a visuo-tactile sensor featuring customized uniform illumination, alongside a unified perception framework.<n>The proposed hardware-software framework provides high-fidelity feedback and a powerful multi-modal perception baseline for complex robotic manipulation.
  arXiv  Detail & Related papers  (2026-08-25T07:25:27Z)
• DriftScope: Measuring The Hidden Effects of Diffusion Model Adaptation [50.584834483403675]
  Adapting pre-trained text-to-image diffusion models, whether to learn new visual concepts or erase unwanted ones, is routinely evaluated on its intended effects alone.<n>We demonstrate that adaptation systematically damages semantically unrelated concepts in ways that aggregate metrics structurally cannot surface.<n>We introduce DriftScope, a prompt-level diagnostic tool that takes any two model checkpoints and returns a ranked list of tokens whose visual concepts have shifted most between them.
  arXiv  Detail & Related papers  (2026-06-30T20:57:51Z)
• Learner-based Concept Drift Detection: Analysis and Evaluation [0.0]
  The presence of concept drift poses a major challenge for many real-world applications.<n>This study examines theoretically the concept drift characteristics and numerous drift detection algorithms across several categories.
  arXiv  Detail & Related papers  (2026-06-18T13:31:58Z)
• A Framework for Evaluating and Benchmarking Concept Drift Detection Methods [18.74827879168727]
  We benchmark 14 widely used drift detection methods on 7 realworld datasets across 4 drift types.<n>Our results provide insights into the strengths and weaknesses of current drift detection approaches.
  arXiv  Detail & Related papers  (2026-06-05T19:02:17Z)
• A Comprehensive Library for Benchmarking Multi-class Visual Anomaly Detection [89.92916473403108]
  This paper proposes a comprehensive visual anomaly detection benchmark, ADer, which is a modular framework for new methods.<n>The benchmark includes multiple datasets from industrial and medical domains, implementing fifteen state-of-the-art methods and nine comprehensive metrics.<n>We objectively reveal the strengths and weaknesses of different methods and provide insights into the challenges and future directions of multi-class visual anomaly detection.
  arXiv  Detail & Related papers  (2024-06-05T13:40:07Z)
• A Neighbor-Searching Discrepancy-based Drift Detection Scheme for Learning Evolving Data [40.00357483768265]
  This work presents a novel real concept drift detection method based on Neighbor-Searching Discrepancy. The proposed method is able to detect real concept drift with high accuracy while ignoring virtual drift. It can also indicate the direction of the classification boundary change by identifying the invasion or retreat of a certain class.
  arXiv  Detail & Related papers  (2024-05-23T04:03:36Z)
• Methods for Generating Drift in Text Streams [49.3179290313959]
  Concept drift is a frequent phenomenon in real-world datasets and corresponds to changes in data distribution over time. This paper provides four textual drift generation methods to ease the production of datasets with labeled drifts. Results show that all methods have their performance degraded right after the drifts, and the incremental SVM is the fastest to run and recover the previous performance levels.
  arXiv  Detail & Related papers  (2024-03-18T23:48:33Z)
• Innovative Horizons in Aerial Imagery: LSKNet Meets DiffusionDet for Advanced Object Detection [55.2480439325792]
  We present an in-depth evaluation of an object detection model that integrates the LSKNet backbone with the DiffusionDet head. The proposed model achieves a mean average precision (MAP) of approximately 45.7%, which is a significant improvement. This advancement underscores the effectiveness of the proposed modifications and sets a new benchmark in aerial image analysis.
  arXiv  Detail & Related papers  (2023-11-21T19:49:13Z)
• A comprehensive analysis of concept drift locality in data streams [3.5897534810405403]
  Concept drift must be detected for effective model adaptation to evolving data properties. We present a novel categorization of concept drift based on its locality and scale. We conduct a comparative assessment of 9 state-of-the-art drift detectors across diverse difficulties.
  arXiv  Detail & Related papers  (2023-11-10T20:57:43Z)
• DeepfakeBench: A Comprehensive Benchmark of Deepfake Detection [55.70982767084996]
  A critical yet frequently overlooked challenge in the field of deepfake detection is the lack of a standardized, unified, comprehensive benchmark. We present the first comprehensive benchmark for deepfake detection, called DeepfakeBench, which offers three key contributions. DeepfakeBench contains 15 state-of-the-art detection methods, 9CL datasets, a series of deepfake detection evaluation protocols and analysis tools, as well as comprehensive evaluations.
  arXiv  Detail & Related papers  (2023-07-04T01:34:41Z)
• Are Concept Drift Detectors Reliable Alarming Systems? -- A Comparative Study [6.7961908135481615]
  Concept drift, also known as concept drift, impacts the performance of machine learning models. In this study, we assess the reliability of concept drift detectors to identify drift in time. Our findings aim to help practitioners understand which drift detector should be employed in different situations.
  arXiv  Detail & Related papers  (2022-11-23T16:31:15Z)
• Detecting Concept Drift in the Presence of Sparsity -- A Case Study of Automated Change Risk Assessment System [0.8021979227281782]
  Missing values, widely called as textitsparsity in literature, is a common characteristic of many real-world datasets. We study different patterns of missing values, various statistical and ML based data imputation methods for different kinds of sparsity. We then select the best concept drift detector given a dataset with missing values based on the different metrics.
  arXiv  Detail & Related papers  (2022-07-27T04:27:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.