Active Feature Selection for the Mutual Information Criterion
- URL: http://arxiv.org/abs/2012.06979v1
- Date: Sun, 13 Dec 2020 06:40:35 GMT
- Title: Active Feature Selection for the Mutual Information Criterion
- Authors: Shachar Schnapp and Sivan Sabato
- Abstract summary: We study active feature selection, a novel feature selection setting in which unlabeled data is available.
We focus on feature selection using the classical mutual information criterion, which selects the $k$ features with the largest mutual information with the label.
In the active feature selection setting, the goal is to use significantly fewer labels than the data set size and still find $k$ features whose mutual information with the label based on the emphentire data set is large.
- Score: 21.376800678915558
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: We study active feature selection, a novel feature selection setting in which
unlabeled data is available, but the budget for labels is limited, and the
examples to label can be actively selected by the algorithm. We focus on
feature selection using the classical mutual information criterion, which
selects the $k$ features with the largest mutual information with the label. In
the active feature selection setting, the goal is to use significantly fewer
labels than the data set size and still find $k$ features whose mutual
information with the label based on the \emph{entire} data set is large. We
explain and experimentally study the choices that we make in the algorithm, and
show that they lead to a successful algorithm, compared to other more naive
approaches. Our design draws on insights which relate the problem of active
feature selection to the study of pure-exploration multi-armed bandits
settings. While we focus here on mutual information, our general methodology
can be adapted to other feature-quality measures as well. The code is available
at the following url: https://github.com/ShacharSchnapp/ActiveFeatureSelection.
Related papers
- Cost-constrained multi-label group feature selection using shadow features [1.87071394890391]
We consider the problem of feature selection in multi-label classification, considering the costs assigned to groups of features.
In this task, the goal is to select a subset of features that will be useful for predicting the label vector, but at the same time, the cost associated with the selected features will not exceed the assumed budget.
arXiv Detail & Related papers (2024-08-03T19:31:59Z) - Multi-Label Feature Selection Using Adaptive and Transformed Relevance [0.0]
This paper presents a novel information-theoretical filter-based multi-label feature selection, called ATR, with a new function.
ATR ranks features considering individual labels as well as abstract label space discriminative powers.
Our experiments affirm the scalability of ATR for benchmarks characterized by extensive feature and label spaces.
arXiv Detail & Related papers (2023-09-26T09:01:38Z) - Exploiting Diversity of Unlabeled Data for Label-Efficient
Semi-Supervised Active Learning [57.436224561482966]
Active learning is a research area that addresses the issues of expensive labeling by selecting the most important samples for labeling.
We introduce a new diversity-based initial dataset selection algorithm to select the most informative set of samples for initial labeling in the active learning setting.
Also, we propose a novel active learning query strategy, which uses diversity-based sampling on consistency-based embeddings.
arXiv Detail & Related papers (2022-07-25T16:11:55Z) - Parallel feature selection based on the trace ratio criterion [4.30274561163157]
This work presents a novel parallel feature selection approach for classification, namely Parallel Feature Selection using Trace criterion (PFST)
Our method uses trace criterion, a measure of class separability used in Fisher's Discriminant Analysis, to evaluate feature usefulness.
The experiments show that our method can produce a small set of features in a fraction of the amount of time by the other methods under comparison.
arXiv Detail & Related papers (2022-03-03T10:50:33Z) - Active metric learning and classification using similarity queries [21.589707834542338]
We show that a novel unified query framework can be applied to any problem in which a key component is learning a representation of the data that reflects similarity.
We demonstrate the effectiveness of the proposed strategy on two tasks -- active metric learning and active classification.
arXiv Detail & Related papers (2022-02-04T03:34:29Z) - Compactness Score: A Fast Filter Method for Unsupervised Feature
Selection [66.84571085643928]
We propose a fast unsupervised feature selection method, named as, Compactness Score (CSUFS) to select desired features.
Our proposed algorithm seems to be more accurate and efficient compared with existing algorithms.
arXiv Detail & Related papers (2022-01-31T13:01:37Z) - Few-shot Learning for Unsupervised Feature Selection [59.75321498170363]
We propose a few-shot learning method for unsupervised feature selection.
The proposed method can select a subset of relevant features in a target task given a few unlabeled target instances.
We experimentally demonstrate that the proposed method outperforms existing feature selection methods.
arXiv Detail & Related papers (2021-07-02T03:52:51Z) - Active Learning for Noisy Data Streams Using Weak and Strong Labelers [3.9370369973510746]
We consider a novel weak and strong labeler problem inspired by humans natural ability for labeling.
We propose an on-line active learning algorithm that consists of four steps: filtering, adding diversity, informative sample selection, and labeler selection.
We derive a decision function that measures the information gain by combining the informativeness of individual samples and model confidence.
arXiv Detail & Related papers (2020-10-27T09:18:35Z) - Online Active Model Selection for Pre-trained Classifiers [72.84853880948894]
We design an online selective sampling approach that actively selects informative examples to label and outputs the best model with high probability at any round.
Our algorithm can be used for online prediction tasks for both adversarial and streams.
arXiv Detail & Related papers (2020-10-19T19:53:15Z) - SPL-MLL: Selecting Predictable Landmarks for Multi-Label Learning [87.27700889147144]
We propose to select a small subset of labels as landmarks which are easy to predict according to input (predictable) and can well recover the other possible labels (representative)
We employ the Alternating Direction Method (ADM) to solve our problem. Empirical studies on real-world datasets show that our method achieves superior classification performance over other state-of-the-art methods.
arXiv Detail & Related papers (2020-08-16T11:07:44Z) - Optimal Clustering from Noisy Binary Feedback [75.17453757892152]
We study the problem of clustering a set of items from binary user feedback.
We devise an algorithm with a minimal cluster recovery error rate.
For adaptive selection, we develop an algorithm inspired by the derivation of the information-theoretical error lower bounds.
arXiv Detail & Related papers (2019-10-14T09:18:26Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.