Moving Metric Detection and Alerting System at eBay

• URL: http://arxiv.org/abs/2004.02360v2
• Date: Mon, 12 Dec 2022 17:16:25 GMT
• Title: Moving Metric Detection and Alerting System at eBay
• Authors: Zezhong Zhang, Keyu Nie and Ted Tao Yuan
• Abstract summary: At eBay, there are thousands of product health metrics for different domain teams to monitor. We built a two-phase alerting system to notify users with actionable alerts based on anomaly detection and alert retrieval.
• Score: 4.778341933013294
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: At eBay, there are thousands of product health metrics for different domain teams to monitor. We built a two-phase alerting system to notify users with actionable alerts based on anomaly detection and alert retrieval. In the first phase, we developed an efficient anomaly detection algorithm, called Moving Metric Detector (MMD), to identify potential alerts among metrics with distribution agnostic criteria. In the second alert retrieval phase, we built additional logic with feedbacks to select valid actionable alerts with point-wise ranking model and business rules. Compared with other trend and seasonality decomposition methods, our decomposer is faster and better to detect anomalies in unsupervised cases. Our two-phase approach dramatically improves alert precision and avoids alert spamming in eBay production.

Related papers

• Carbon Filter: Real-time Alert Triage Using Large Scale Clustering and Fast Search [6.830322979559498]
  "Alert fatigue" is one of the biggest challenges faced by the Security Operations Center (SOC) today. We present Carbon Filter, a statistical learning based system that dramatically reduces the number of alerts analysts need to manually review.
  arXiv  Detail & Related papers  (2024-05-07T22:06:24Z)
• A Supervised Embedding and Clustering Anomaly Detection method for classification of Mobile Network Faults [0.0]
  The paper introduces Supervised Embedding and Clustering Anomaly Detection (SEMC-AD) It is a method designed to efficiently identify faulty alarm logs in a mobile network and alleviate the challenges of manual monitoring. SEMC-AD achieves 99% anomaly detection, whereas random forest and XGBoost only detect 86% and 81% of anomalies, respectively.
  arXiv  Detail & Related papers  (2023-10-10T16:54:25Z)
• An optimization method for out-of-distribution anomaly detection models [6.075775003017512]
  Frequent false alarms impede the promotion of unsupervised anomaly detection algorithms in industrial applications. An SVM-based classifier is exploited as a post-processing module to identify false alarms from the anomaly map at the object level.
  arXiv  Detail & Related papers  (2023-02-02T08:29:10Z)
• A Robust and Explainable Data-Driven Anomaly Detection Approach For Power Electronics [56.86150790999639]
  We present two anomaly detection and classification approaches, namely the Matrix Profile algorithm and anomaly transformer. The Matrix Profile algorithm is shown to be well suited as a generalizable approach for detecting real-time anomalies in streaming time-series data. A series of custom filters is created and added to the detector to tune its sensitivity, recall, and detection accuracy.
  arXiv  Detail & Related papers  (2022-09-23T06:09:35Z)
• An Effective and Robust Detector for Logo Detection [58.448716977297565]
  Some attackers fool the well-trained logo detection model for infringement. A novel logo detector based on the mechanism of looking and thinking twice is proposed in this paper. We extend detectoRS algorithm to a cascade schema with an equalization loss function, multi-scale transformations, and adversarial data augmentation.
  arXiv  Detail & Related papers  (2021-08-01T10:17:53Z)
• ESAD: End-to-end Deep Semi-supervised Anomaly Detection [85.81138474858197]
  We propose a new objective function that measures the KL-divergence between normal and anomalous data. The proposed method significantly outperforms several state-of-the-arts on multiple benchmark datasets.
  arXiv  Detail & Related papers  (2020-12-09T08:16:35Z)
• No Need to Know Physics: Resilience of Process-based Model-free Anomaly Detection for Industrial Control Systems [95.54151664013011]
  We present a novel framework to generate adversarial spoofing signals that violate physical properties of the system. We analyze four anomaly detectors published at top security conferences.
  arXiv  Detail & Related papers  (2020-12-07T11:02:44Z)
• Machine Learning Applications in Misuse and Anomaly Detection [0.0]
  Machine learning and data mining algorithms play important roles in designing intrusion detection systems. Based on their approaches toward the detection of attacks in a network, intrusion detection systems can be broadly categorized into two types. In the misuse detection systems, an attack in a system is detected whenever the sequence of activities in the network matches with a known attack signature. In the anomaly detection approach, on the other hand, anomalous states in a system are identified based on a significant difference in the state transitions of the system from its normal states.
  arXiv  Detail & Related papers  (2020-09-10T19:52:00Z)
• ADSAGE: Anomaly Detection in Sequences of Attributed Graph Edges applied to insider threat detection at fine-grained level [0.5134435281973136]
  We introduce ADSAGE to detect anomalies in audit log events modeled as graph edges. Our method is the first to perform anomaly detection at edge level while supporting both edge sequences and attributes. We evaluate ADSAGE on authentication, email traffic and web browsing logs from the CERT insider threat datasets.
  arXiv  Detail & Related papers  (2020-07-14T12:05:05Z)
• Unsupervised Anomaly Detection with Adversarial Mirrored AutoEncoders [51.691585766702744]
  We propose a variant of Adversarial Autoencoder which uses a mirrored Wasserstein loss in the discriminator to enforce better semantic-level reconstruction. We put forward an alternative measure of anomaly score to replace the reconstruction-based metric. Our method outperforms the current state-of-the-art methods for anomaly detection on several OOD detection benchmarks.
  arXiv  Detail & Related papers  (2020-03-24T08:26:58Z)
• EHSOD: CAM-Guided End-to-end Hybrid-Supervised Object Detection with Cascade Refinement [53.69674636044927]
  We present EHSOD, an end-to-end hybrid-supervised object detection system. It can be trained in one shot on both fully and weakly-annotated data. It achieves comparable results on multiple object detection benchmarks with only 30% fully-annotated data.
  arXiv  Detail & Related papers  (2020-02-18T08:04:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.