Maximizing Conditional Entropy for Batch-Mode Active Learning of Perceptual Metrics

• URL: http://arxiv.org/abs/2102.07365v2
• Date: Tue, 16 Feb 2021 04:11:42 GMT
• Title: Maximizing Conditional Entropy for Batch-Mode Active Learning of Perceptual Metrics
• Authors: Priyadarshini Kumari, Sidhdhartha Chaudhuri, Vivek Borkar, Subhasis Chaudhuri
• Abstract summary: We present a novel approach for batch mode active metric learning using the Maximum Entropy Principle. We take advantage of the monotonically increasing submodular entropy function to construct an efficient greedy algorithm. Our approach is the first batch-mode active metric learning method to define a unified score that balances informativeness and diversity for an entire batch of triplets.
• Score: 14.777274711706653
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Active metric learning is the problem of incrementally selecting batches of training data (typically, ordered triplets) to annotate, in order to progressively improve a learned model of a metric over some input domain as rapidly as possible. Standard approaches, which independently select each triplet in a batch, are susceptible to highly correlated batches with many redundant triplets and hence low overall utility. While there has been recent work on selecting decorrelated batches for metric learning \cite{kumari2020batch}, these methods rely on ad hoc heuristics to estimate the correlation between two triplets at a time. We present a novel approach for batch mode active metric learning using the Maximum Entropy Principle that seeks to collectively select batches with maximum joint entropy, which captures both the informativeness and the diversity of the triplets. The entropy is derived from the second-order statistics estimated by dropout. We take advantage of the monotonically increasing submodular entropy function to construct an efficient greedy algorithm based on Gram-Schmidt orthogonalization that is provably $\left( 1 - \frac{1}{e} \right)$-optimal. Our approach is the first batch-mode active metric learning method to define a unified score that balances informativeness and diversity for an entire batch of triplets. Experiments with several real-world datasets demonstrate that our algorithm is robust and consistently outperforms the state-of-the-art.

Related papers

• Interpretable Triplet Importance for Personalized Ranking [5.409302364904161]
  We propose a shapely value-based method to measure the triplet importance in an interpretable manner. Our model consistently outperforms the state-of-the-art methods.
  arXiv  Detail & Related papers  (2024-07-28T11:46:55Z)
• Multi-threshold Deep Metric Learning for Facial Expression Recognition [60.26967776920412]
  We present the multi-threshold deep metric learning technique, which avoids the difficult threshold validation. We find that each threshold of the triplet loss intrinsically determines a distinctive distribution of inter-class variations. It makes the embedding layer, which is composed of a set of slices, a more informative and discriminative feature.
  arXiv  Detail & Related papers  (2024-06-24T08:27:31Z)
• BatchGFN: Generative Flow Networks for Batch Active Learning [80.73649229919454]
  BatchGFN is a novel approach for pool-based active learning that uses generative flow networks to sample sets of data points proportional to a batch reward. We show our approach enables principled sampling near-optimal utility batches at inference time with a single forward pass per point in the batch in toy regression problems.
  arXiv  Detail & Related papers  (2023-06-26T20:41:36Z)
• On the Stability and Generalization of Triplet Learning [55.75784102837832]
  Triplet learning, i.e. learning from triplet data, has attracted much attention in computer vision tasks. This paper investigates the generalization guarantees of triplet learning by leveraging the stability analysis.
  arXiv  Detail & Related papers  (2023-02-20T07:32:50Z)
• Walk-and-Relate: A Random-Walk-based Algorithm for Representation Learning on Sparse Knowledge Graphs [5.444459446244819]
  We propose an efficient method to augment the number of triplets to address the problem of data sparsity. We also provide approaches to accurately and efficiently filter out informative metapaths from the possible set of metapaths. The proposed approaches are model-agnostic, and the augmented training dataset can be used with any KG embedding approach out of the box.
  arXiv  Detail & Related papers  (2022-09-19T05:35:23Z)
• Efficient and Near-Optimal Smoothed Online Learning for Generalized Linear Functions [28.30744223973527]
  We give a computationally efficient algorithm that is the first to enjoy the statistically optimal log(T/sigma) regret for realizable K-wise linear classification. We develop a novel characterization of the geometry of the disagreement region induced by generalized linear classifiers.
  arXiv  Detail & Related papers  (2022-05-25T21:31:36Z)
• A Lagrangian Duality Approach to Active Learning [119.36233726867992]
  We consider the batch active learning problem, where only a subset of the training data is labeled. We formulate the learning problem using constrained optimization, where each constraint bounds the performance of the model on labeled samples. We show, via numerical experiments, that our proposed approach performs similarly to or better than state-of-the-art active learning methods.
  arXiv  Detail & Related papers  (2022-02-08T19:18:49Z)
• Removing Bias in Multi-modal Classifiers: Regularization by Maximizing Functional Entropies [88.0813215220342]
  Some modalities can more easily contribute to the classification results than others. We develop a method based on the log-Sobolev inequality, which bounds the functional entropy with the functional-Fisher-information. On the two challenging multi-modal datasets VQA-CPv2 and SocialIQ, we obtain state-of-the-art results while more uniformly exploiting the modalities.
  arXiv  Detail & Related papers  (2020-10-21T07:40:33Z)
• Batch Decorrelation for Active Metric Learning [21.99577268213412]
  We present an active learning strategy for training parametric models of distance metrics, given triplet-based similarity assessments. In contrast to prior work on class-based learning, we focus on em metrics that express the em degree of (dis)similarity between objects.
  arXiv  Detail & Related papers  (2020-05-20T12:47:48Z)
• Multi-Objective Matrix Normalization for Fine-grained Visual Recognition [153.49014114484424]
  Bilinear pooling achieves great success in fine-grained visual recognition (FGVC) Recent methods have shown that the matrix power normalization can stabilize the second-order information in bilinear features. We propose an efficient Multi-Objective Matrix Normalization (MOMN) method that can simultaneously normalize a bilinear representation.
  arXiv  Detail & Related papers  (2020-03-30T08:40:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.