Revisiting Non-separable Binary Classification and its Applications in Anomaly Detection

• URL: http://arxiv.org/abs/2312.01541v2
• Date: Tue, 18 Jun 2024 20:46:32 GMT
• Title: Revisiting Non-separable Binary Classification and its Applications in Anomaly Detection
• Authors: Matthew Lau, Ismaila Seck, Athanasios P Meliopoulos, Wenke Lee, Eugene Ndiaye,
• Abstract summary: We show that linear classification of XOR is possible. We propose equality separation, that adapts the SVM objective to distinguish data within or outside the margin. Our classifier can then be integrated into neural network pipelines with a smooth approximation.
• Score: 10.031370250511207
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The inability to linearly classify XOR has motivated much of deep learning. We revisit this age-old problem and show that linear classification of XOR is indeed possible. Instead of separating data between halfspaces, we propose a slightly different paradigm, equality separation, that adapts the SVM objective to distinguish data within or outside the margin. Our classifier can then be integrated into neural network pipelines with a smooth approximation. From its properties, we intuit that equality separation is suitable for anomaly detection. To formalize this notion, we introduce closing numbers, a quantitative measure on the capacity for classifiers to form closed decision regions for anomaly detection. Springboarding from this theoretical connection between binary classification and anomaly detection, we test our hypothesis on supervised anomaly detection experiments, showing that equality separation can detect both seen and unseen anomalies.

Related papers

• Revisiting Deep Ensemble Uncertainty for Enhanced Medical Anomaly Detection [34.14012444375776]
  We propose a Dual-space Uncertainty Estimation framework for medical anomaly detection. To balance agreement and disagreement for anomaly detection, we propose Redundancy-Aware Repulsion (RAR) We also develop Dual-Space Uncertainty (DSU) which utilizes the ensemble's uncertainty in input and output spaces.
  arXiv  Detail & Related papers  (2024-09-26T02:47:41Z)
• Anomaly Detection Under Uncertainty Using Distributionally Robust Optimization Approach [0.9217021281095907]
  Anomaly detection is defined as the problem of finding data points that do not follow the patterns of the majority. The one-class Support Vector Machines (SVM) method aims to find a decision boundary to distinguish between normal data points and anomalies. A distributionally robust chance-constrained model is proposed in which the probability of misclassification is low.
  arXiv  Detail & Related papers  (2023-12-03T06:13:22Z)
• Decoupling anomaly discrimination and representation learning: self-supervised learning for anomaly detection on attributed graph [18.753970895946814]
  DSLAD is a self-supervised method with anomaly discrimination and representation learning decoupled for anomaly detection. Experiments conducted on various six benchmark datasets reveal the effectiveness of DSLAD.
  arXiv  Detail & Related papers  (2023-04-11T12:23:40Z)
• Hybrid Open-set Segmentation with Synthetic Negative Data [0.0]
  Open-set segmentation can be conceived by complementing closed-set classification with anomaly detection. We propose a novel anomaly score that fuses generative and discriminative cues. Experiments reveal strong open-set performance in spite of negligible computational overhead.
  arXiv  Detail & Related papers  (2023-01-19T11:02:44Z)
• Hierarchical Semi-Supervised Contrastive Learning for Contamination-Resistant Anomaly Detection [81.07346419422605]
  Anomaly detection aims at identifying deviant samples from the normal data distribution. Contrastive learning has provided a successful way to sample representation that enables effective discrimination on anomalies. We propose a novel hierarchical semi-supervised contrastive learning framework, for contamination-resistant anomaly detection.
  arXiv  Detail & Related papers  (2022-07-24T18:49:26Z)
• SLA$^2$P: Self-supervised Anomaly Detection with Adversarial Perturbation [77.71161225100927]
  Anomaly detection is a fundamental yet challenging problem in machine learning. We propose a novel and powerful framework, dubbed as SLA$2$P, for unsupervised anomaly detection.
  arXiv  Detail & Related papers  (2021-11-25T03:53:43Z)
• Explainable Deep Few-shot Anomaly Detection with Deviation Networks [123.46611927225963]
  We introduce a novel weakly-supervised anomaly detection framework to train detection models. The proposed approach learns discriminative normality by leveraging the labeled anomalies and a prior probability. Our model is substantially more sample-efficient and robust, and performs significantly better than state-of-the-art competing methods in both closed-set and open-set settings.
  arXiv  Detail & Related papers  (2021-08-01T14:33:17Z)
• Visualizing Classifier Adjacency Relations: A Case Study in Speaker Verification and Voice Anti-Spoofing [72.4445825335561]
  We propose a simple method to derive 2D representation from detection scores produced by an arbitrary set of binary classifiers. Based upon rank correlations, our method facilitates a visual comparison of classifiers with arbitrary scores. While the approach is fully versatile and can be applied to any detection task, we demonstrate the method using scores produced by automatic speaker verification and voice anti-spoofing systems.
  arXiv  Detail & Related papers  (2021-06-11T13:03:33Z)
• A Hypothesis Testing Approach to Nonstationary Source Separation [15.193722258844517]
  The extraction of nonstationary signals from blind and semi-blind multivariate observations is a recurrent problem. Various methods for nonstationarity identification are reviewed and a new general framework based on hypothesis testing is proposed. The proposed method is applied to noninvasive fetal ECG extraction, as case study.
  arXiv  Detail & Related papers  (2021-05-14T16:58:55Z)
• A Unified Joint Maximum Mean Discrepancy for Domain Adaptation [73.44809425486767]
  This paper theoretically derives a unified form of JMMD that is easy to optimize. From the revealed unified JMMD, we illustrate that JMMD degrades the feature-label dependence that benefits to classification. We propose a novel MMD matrix to promote the dependence, and devise a novel label kernel that is robust to label distribution shift.
  arXiv  Detail & Related papers  (2021-01-25T09:46:14Z)
• Learning from Aggregate Observations [82.44304647051243]
  We study the problem of learning from aggregate observations where supervision signals are given to sets of instances. We present a general probabilistic framework that accommodates a variety of aggregate observations. Simple maximum likelihood solutions can be applied to various differentiable models.
  arXiv  Detail & Related papers  (2020-04-14T06:18:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.