A Comprehensive Survey on Rare Event Prediction
- URL: http://arxiv.org/abs/2309.11356v1
- Date: Wed, 20 Sep 2023 14:36:57 GMT
- Title: A Comprehensive Survey on Rare Event Prediction
- Authors: Chathurangi Shyalika, Ruwan Wickramarachchi, Amit Sheth
- Abstract summary: This paper aims to identify gaps in the current literature and highlight the challenges of predicting rare events.
It also suggests potential research directions, which can help guide practitioners and researchers.
- Score: 1.8416014644193066
- License: http://creativecommons.org/licenses/by-sa/4.0/
- Abstract: Rare event prediction involves identifying and forecasting events with a low
probability using machine learning and data analysis. Due to the imbalanced
data distributions, where the frequency of common events vastly outweighs that
of rare events, it requires using specialized methods within each step of the
machine learning pipeline, i.e., from data processing to algorithms to
evaluation protocols. Predicting the occurrences of rare events is important
for real-world applications, such as Industry 4.0, and is an active research
area in statistical and machine learning. This paper comprehensively reviews
the current approaches for rare event prediction along four dimensions: rare
event data, data processing, algorithmic approaches, and evaluation approaches.
Specifically, we consider 73 datasets from different modalities (i.e.,
numerical, image, text, and audio), four major categories of data processing,
five major algorithmic groupings, and two broader evaluation approaches. This
paper aims to identify gaps in the current literature and highlight the
challenges of predicting rare events. It also suggests potential research
directions, which can help guide practitioners and researchers.
Related papers
- Evaluating the Role of Data Enrichment Approaches Towards Rare Event Analysis in Manufacturing [1.3980986259786223]
Rare events are occurrences that take place with a significantly lower frequency than more common regular events.
In manufacturing, predicting such events is particularly important, as they lead to unplanned downtime, shortening equipment lifespan, and high energy consumption.
This paper evaluates the role of data enrichment techniques combined with supervised machine-learning techniques for rare event detection and prediction.
arXiv Detail & Related papers (2024-07-01T00:05:56Z) - Stochastic Amortization: A Unified Approach to Accelerate Feature and
Data Attribution [67.28273187033693]
We show that training a network that directly predicts the desired output, known as amortization, is inexpensive and surprisingly effective.
This approach significantly accelerates several feature attribution and data valuation methods, often yielding an order of magnitude speedup over existing approaches.
arXiv Detail & Related papers (2024-01-29T03:42:37Z) - Event Detection in Time Series: Universal Deep Learning Approach [0.0]
Event detection in time series is a challenging task due to the prevalence of imbalanced datasets, rare events, and time interval-defined events.
We propose a novel supervised regression-based deep learning approach that offers several advantages over classification-based methods.
Our approach can effectively handle various types of events within a unified framework, including rare events and imbalanced datasets.
arXiv Detail & Related papers (2023-11-27T09:33:56Z) - Systematic Evaluation of Predictive Fairness [60.0947291284978]
Mitigating bias in training on biased datasets is an important open problem.
We examine the performance of various debiasing methods across multiple tasks.
We find that data conditions have a strong influence on relative model performance.
arXiv Detail & Related papers (2022-10-17T05:40:13Z) - Predicting Seriousness of Injury in a Traffic Accident: A New Imbalanced
Dataset and Benchmark [62.997667081978825]
The paper introduces a new dataset to assess the performance of machine learning algorithms in the prediction of the seriousness of injury in a traffic accident.
The dataset is created by aggregating publicly available datasets from the UK Department for Transport.
arXiv Detail & Related papers (2022-05-20T21:15:26Z) - Robust Event Classification Using Imperfect Real-world PMU Data [58.26737360525643]
We study robust event classification using imperfect real-world phasor measurement unit (PMU) data.
We develop a novel machine learning framework for training robust event classifiers.
arXiv Detail & Related papers (2021-10-19T17:41:43Z) - Deep Learning Schema-based Event Extraction: Literature Review and
Current Trends [60.29289298349322]
Event extraction technology based on deep learning has become a research hotspot.
This paper fills the gap by reviewing the state-of-the-art approaches, focusing on deep learning-based models.
arXiv Detail & Related papers (2021-07-05T16:32:45Z) - Event-Related Bias Removal for Real-time Disaster Events [67.2965372987723]
Social media has become an important tool to share information about crisis events such as natural disasters and mass attacks.
Detecting actionable posts that contain useful information requires rapid analysis of huge volume of data in real-time.
We train an adversarial neural model to remove latent event-specific biases and improve the performance on tweet importance classification.
arXiv Detail & Related papers (2020-11-02T02:03:07Z) - Unsupervised Event Detection, Clustering, and Use Case Exposition in
Micro-PMU Measurements [0.0]
We develop an unsupervised event detection method based on the concept of Generative Adversarial Networks (GAN)
We also propose a two-step unsupervised clustering method, based on a novel linear mixed integer programming formulation.
Results show that they can outperform the prevalent methods in the literature.
arXiv Detail & Related papers (2020-07-30T05:20:29Z) - Event Prediction in the Big Data Era: A Systematic Survey [7.3810864598379755]
Event prediction is becoming a viable option in the big data era.
This paper aims to provide a systematic and comprehensive survey of the technologies, applications, and evaluations of event prediction.
arXiv Detail & Related papers (2020-07-19T23:24:52Z) - Multi-label Prediction in Time Series Data using Deep Neural Networks [19.950094635430048]
This paper addresses a multi-label predictive fault classification problem for multidimensional time-series data.
The proposed algorithm is tested on two public benchmark datasets.
arXiv Detail & Related papers (2020-01-27T21:35:15Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.