Related papers: R\'{e}nyi Entropy Bounds on the Active Learning Cost-Performance Tradeoff

R\'{e}nyi Entropy Bounds on the Active Learning Cost-Performance Tradeoff

URL: http://arxiv.org/abs/2002.02025v1
Date: Wed, 5 Feb 2020 22:38:35 GMT
Title: R\'{e}nyi Entropy Bounds on the Active Learning Cost-Performance Tradeoff
Authors: Vahid Jamali, Antonia Tulino, Jaime Llorca, and Elza Erkip
Abstract summary: Semi-supervised classification studies how to combine the statistical knowledge of the often abundant unlabeled data with the often limited labeled data in order to maximize overall classification accuracy. In this paper, we initiate the non-asymptotic analysis of the optimal policy for semi-supervised classification with actively obtained labeled data. We provide the first characterization of the jointly optimal active learning and semi-supervised classification policy, in terms of the cost-performance tradeoff driven by the label query budget and overall classification accuracy.
Score: 27.436483977171328
License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
Abstract: Semi-supervised classification, one of the most prominent fields in machine learning, studies how to combine the statistical knowledge of the often abundant unlabeled data with the often limited labeled data in order to maximize overall classification accuracy. In this context, the process of actively choosing the data to be labeled is referred to as active learning. In this paper, we initiate the non-asymptotic analysis of the optimal policy for semi-supervised classification with actively obtained labeled data. Considering a general Bayesian classification model, we provide the first characterization of the jointly optimal active learning and semi-supervised classification policy, in terms of the cost-performance tradeoff driven by the label query budget (number of data items to be labeled) and overall classification accuracy. Leveraging recent results on the R\'enyi Entropy, we derive tight information-theoretic bounds on such active learning cost-performance tradeoff.