Abstract Interpretation-Based Feature Importance for SVMs
- URL: http://arxiv.org/abs/2210.12456v1
- Date: Sat, 22 Oct 2022 13:57:44 GMT
- Title: Abstract Interpretation-Based Feature Importance for SVMs
- Authors: Abhinandan Pal, Francesco Ranzato, Caterina Urban, Marco Zanella
- Abstract summary: We propose a symbolic representation for support vector machines (SVMs) by means of abstract interpretation.
We derive a novel feature importance measure, called abstract feature importance (AFI), that does not depend in any way on a given dataset of the accuracy of the SVM.
Our experimental results show that, independently of the accuracy of the SVM, our AFI measure correlates much more strongly with the stability of the SVM to feature perturbations than feature importance measures widely available in machine learning software.
- Score: 8.879921160392737
- License: http://creativecommons.org/licenses/by-nc-nd/4.0/
- Abstract: We propose a symbolic representation for support vector machines (SVMs) by
means of abstract interpretation, a well-known and successful technique for
designing and implementing static program analyses. We leverage this
abstraction in two ways: (1) to enhance the interpretability of SVMs by
deriving a novel feature importance measure, called abstract feature importance
(AFI), that does not depend in any way on a given dataset of the accuracy of
the SVM and is very fast to compute, and (2) for verifying stability, notably
individual fairness, of SVMs and producing concrete counterexamples when the
verification fails. We implemented our approach and we empirically demonstrated
its effectiveness on SVMs based on linear and non-linear (polynomial and radial
basis function) kernels. Our experimental results show that, independently of
the accuracy of the SVM, our AFI measure correlates much more strongly with the
stability of the SVM to feature perturbations than feature importance measures
widely available in machine learning software such as permutation feature
importance. It thus gives better insight into the trustworthiness of SVMs.
Related papers
- Multiview learning with twin parametric margin SVM [0.0]
Multiview learning (MVL) seeks to leverage the benefits of diverse perspectives to complement each other.
We propose multiview twin parametric margin support vector machine (MvTPMSVM)
MvTPMSVM constructs parametric margin hyperplanes corresponding to both classes, aiming to regulate and manage the impact of the heteroscedastic noise structure.
arXiv Detail & Related papers (2024-08-04T10:16:11Z) - Understanding Self-attention Mechanism via Dynamical System Perspective [58.024376086269015]
Self-attention mechanism (SAM) is widely used in various fields of artificial intelligence.
We show that intrinsic stiffness phenomenon (SP) in the high-precision solution of ordinary differential equations (ODEs) also widely exists in high-performance neural networks (NN)
We show that the SAM is also a stiffness-aware step size adaptor that can enhance the model's representational ability to measure intrinsic SP.
arXiv Detail & Related papers (2023-08-19T08:17:41Z) - Enhancing Pattern Classification in Support Vector Machines through
Matrix Formulation [0.0]
The reliance on vector-based formulations in existing SVM-based models poses limitations regarding flexibility and ease of incorporating additional terms to handle specific challenges.
We introduce a matrix formulation for SVM that effectively addresses these constraints.
Experimental evaluations on multilabel and multiclass datasets demonstrate that Matrix SVM achieves superior time efficacy.
arXiv Detail & Related papers (2023-07-18T15:56:39Z) - Evaluating robustness of support vector machines with the Lagrangian
dual approach [6.868150350359336]
We propose a method to improve the verification performance for vector machines (SVMs) with nonlinear kernels.
We evaluate the adversarial robustness of SVMs with linear and nonlinear kernels on the MNIST and Fashion-MNIST datasets.
The experimental results show that the percentage of provable robustness obtained by our method on the test set is better than that of the state-of-the-art.
arXiv Detail & Related papers (2023-06-05T07:15:54Z) - Variational Distillation for Multi-View Learning [104.17551354374821]
We design several variational information bottlenecks to exploit two key characteristics for multi-view representation learning.
Under rigorously theoretical guarantee, our approach enables IB to grasp the intrinsic correlation between observations and semantic labels.
arXiv Detail & Related papers (2022-06-20T03:09:46Z) - Information-Theoretic Odometry Learning [83.36195426897768]
We propose a unified information theoretic framework for learning-motivated methods aimed at odometry estimation.
The proposed framework provides an elegant tool for performance evaluation and understanding in information-theoretic language.
arXiv Detail & Related papers (2022-03-11T02:37:35Z) - Chance constrained conic-segmentation support vector machine with
uncertain data [0.0]
Support vector machines (SVM) is one of the well known supervised classes of learning algorithms.
This paper studies CS-SVM when the data points are uncertain or mislabelled.
arXiv Detail & Related papers (2021-07-28T12:29:47Z) - Machine-Learning-Derived Entanglement Witnesses [55.76279816849472]
We show a correspondence between linear support vector machines (SVMs) and entanglement witnesses.
We use this correspondence to generate entanglement witnesses for bipartite and tripartite qubit (and qudit) target entangled states.
arXiv Detail & Related papers (2021-07-05T22:28:02Z) - Estimating Average Treatment Effects with Support Vector Machines [77.34726150561087]
Support vector machine (SVM) is one of the most popular classification algorithms in the machine learning literature.
We adapt SVM as a kernel-based weighting procedure that minimizes the maximum mean discrepancy between the treatment and control groups.
We characterize the bias of causal effect estimation arising from this trade-off, connecting the proposed SVM procedure to the existing kernel balancing methods.
arXiv Detail & Related papers (2021-02-23T20:22:56Z) - On Coresets for Support Vector Machines [61.928187390362176]
A coreset is a small, representative subset of the original data points.
We show that our algorithm can be used to extend the applicability of any off-the-shelf SVM solver to streaming, distributed, and dynamic data settings.
arXiv Detail & Related papers (2020-02-15T23:25:12Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.