A comprehensive analysis of concept drift locality in data streams
- URL: http://arxiv.org/abs/2311.06396v2
- Date: Sat, 9 Dec 2023 15:17:09 GMT
- Title: A comprehensive analysis of concept drift locality in data streams
- Authors: Gabriel J. Aguiar and Alberto Cano
- Abstract summary: Concept drift must be detected for effective model adaptation to evolving data properties.
We present a novel categorization of concept drift based on its locality and scale.
We conduct a comparative assessment of 9 state-of-the-art drift detectors across diverse difficulties.
- Score: 3.5897534810405403
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Adapting to drifting data streams is a significant challenge in online
learning. Concept drift must be detected for effective model adaptation to
evolving data properties. Concept drift can impact the data distribution
entirely or partially, which makes it difficult for drift detectors to
accurately identify the concept drift. Despite the numerous concept drift
detectors in the literature, standardized procedures and benchmarks for
comprehensive evaluation considering the locality of the drift are lacking. We
present a novel categorization of concept drift based on its locality and
scale. A systematic approach leads to a set of 2,760 benchmark problems,
reflecting various difficulty levels following our proposed categorization. We
conduct a comparative assessment of 9 state-of-the-art drift detectors across
diverse difficulties, highlighting their strengths and weaknesses for future
research. We examine how drift locality influences the classifier performance
and propose strategies for different drift categories to minimize the recovery
time. Lastly, we provide lessons learned and recommendations for future concept
drift research. Our benchmark data streams and experiments are publicly
available at https://github.com/gabrieljaguiar/locality-concept-drift.
Related papers
- Online Drift Detection with Maximum Concept Discrepancy [13.48123472458282]
We propose MCD-DD, a novel concept drift detection method based on maximum concept discrepancy.
Our method can adaptively identify varying forms of concept drift by contrastive learning of concept embeddings.
arXiv Detail & Related papers (2024-07-07T13:57:50Z) - A Neighbor-Searching Discrepancy-based Drift Detection Scheme for Learning Evolving Data [40.00357483768265]
This work presents a novel real concept drift detection method based on Neighbor-Searching Discrepancy.
The proposed method is able to detect real concept drift with high accuracy while ignoring virtual drift.
It can also indicate the direction of the classification boundary change by identifying the invasion or retreat of a certain class.
arXiv Detail & Related papers (2024-05-23T04:03:36Z) - Methods for Generating Drift in Text Streams [49.3179290313959]
Concept drift is a frequent phenomenon in real-world datasets and corresponds to changes in data distribution over time.
This paper provides four textual drift generation methods to ease the production of datasets with labeled drifts.
Results show that all methods have their performance degraded right after the drifts, and the incremental SVM is the fastest to run and recover the previous performance levels.
arXiv Detail & Related papers (2024-03-18T23:48:33Z) - Concept Drift Adaptation in Text Stream Mining Settings: A Comprehensive
Review [49.3179290313959]
This study performed a systematic literature review regarding concept drift adaptation in text stream scenarios.
We selected 40 papers to unravel aspects such as text drift categories, types of text drift detection, model update mechanism, the addressed stream mining tasks, types of text representations, and text representation update mechanism.
arXiv Detail & Related papers (2023-12-05T17:15:16Z) - Unsupervised Domain Adaptation for Self-Driving from Past Traversal
Features [69.47588461101925]
We propose a method to adapt 3D object detectors to new driving environments.
Our approach enhances LiDAR-based detection models using spatial quantized historical features.
Experiments on real-world datasets demonstrate significant improvements.
arXiv Detail & Related papers (2023-09-21T15:00:31Z) - Are Concept Drift Detectors Reliable Alarming Systems? -- A Comparative
Study [6.7961908135481615]
Concept drift, also known as concept drift, impacts the performance of machine learning models.
In this study, we assess the reliability of concept drift detectors to identify drift in time.
Our findings aim to help practitioners understand which drift detector should be employed in different situations.
arXiv Detail & Related papers (2022-11-23T16:31:15Z) - StreamYOLO: Real-time Object Detection for Streaming Perception [84.2559631820007]
We endow the models with the capacity of predicting the future, significantly improving the results for streaming perception.
We consider multiple velocities driving scene and propose Velocity-awared streaming AP (VsAP) to jointly evaluate the accuracy.
Our simple method achieves the state-of-the-art performance on Argoverse-HD dataset and improves the sAP and VsAP by 4.7% and 8.2% respectively.
arXiv Detail & Related papers (2022-07-21T12:03:02Z) - Automatic Learning to Detect Concept Drift [40.69280758487987]
We propose Meta-ADD, a novel framework that learns to classify concept drift by tracking the changed pattern of error rates.
Specifically, in the training phase, we extract meta-features based on the error rates of various concept drift, after which a meta-detector is developed via prototypical neural network.
In the detection phase, the learned meta-detector is fine-tuned to adapt to the corresponding data stream via stream-based active learning.
arXiv Detail & Related papers (2021-05-04T11:10:39Z) - Adversarial Concept Drift Detection under Poisoning Attacks for Robust
Data Stream Mining [15.49323098362628]
We propose a framework for robust concept drift detection in the presence of adversarial and poisoning attacks.
We introduce the taxonomy for two types of adversarial concept drifts, as well as a robust trainable drift detector.
We also introduce Relative Loss of Robustness - a novel measure for evaluating the performance of concept drift detectors under poisoning attacks.
arXiv Detail & Related papers (2020-09-20T18:46:31Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.