Pool-based sequential active learning with multi kernels

• URL: http://arxiv.org/abs/2010.11421v1
• Date: Thu, 22 Oct 2020 03:54:41 GMT
• Title: Pool-based sequential active learning with multi kernels
• Authors: Jeongmin Chae, Songnam Hong
• Abstract summary: We study a pool-based sequential active learning (AL) in which one sample is queried at each time from a large pool of unlabeled data. We propose two selection criteria, named expected- Kerneldiscrepancy (EKD) and expected- Kernel-loss (EKL) Also, it is identified that the proposed EKD and EKL successfully generalize the concepts of popular query-by-committee (QBC) and expected-model-change (EMC)
• Score: 10.203602318836444
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We study a pool-based sequential active learning (AL), in which one sample is queried at each time from a large pool of unlabeled data according to a selection criterion. For this framework, we propose two selection criteria, named expected-kernel-discrepancy (EKD) and expected-kernel-loss (EKL), by leveraging the particular structure of multiple kernel learning (MKL). Also, it is identified that the proposed EKD and EKL successfully generalize the concepts of popular query-by-committee (QBC) and expected-model-change (EMC), respectively. Via experimental results with real-data sets, we verify the effectiveness of the proposed criteria compared with the existing methods.

Related papers

• Concept-Aware Batch Sampling Improves Language-Image Pretraining [78.53540190580189]
  Concept-Aware Batch Sampling (CABS) is a simple yet effective batch sampling framework that flexibly constructs batches on-the-fly.<n>We show that CABS significantly benefits CLIP/SigLIP model classes and yields highly performant models.<n>Overall, CABS represents a strong open-source alternative to proprietary online data curation algorithms.
  arXiv  Detail & Related papers  (2025-11-25T18:58:07Z)
• KITE: Kernelized and Information Theoretic Exemplars for In-Context Learning [30.471243464952625]
  In-context learning (ICL) has emerged as a powerful paradigm for adapting large language models to new and data-scarce tasks.<n>We study the problem of example selection in ICL from a principled, information theory-driven perspective.<n>We derive a principled surrogate objective that is approximately submodular, enabling the use of a greedy algorithm with an approximation guarantee.
  arXiv  Detail & Related papers  (2025-09-19T06:50:03Z)
• Learning Discrete Bayesian Networks with Hierarchical Dirichlet Shrinkage [52.914168158222765]
  We detail a comprehensive Bayesian framework for learning DBNs.<n>We give a novel Markov chain Monte Carlo (MCMC) algorithm utilizing parallel Langevin proposals to generate exact posterior samples.<n>We apply our methodology to uncover prognostic network structure from primary breast cancer samples.
  arXiv  Detail & Related papers  (2025-09-16T17:24:35Z)
• Large Language Models are Demonstration Pre-Selectors for Themselves [57.101804269100185]
  In-context learning (ICL) with large language models (LLMs) delivers strong few-shot performance by choosing few-shot demonstrations from the entire training data.<n>FEw yet Essential Demonstration prE-selectoR is a novel pre-selection framework that identifies a representative subset of demonstrations.<n>FEw yet Essential Demonstration prE-selectoR can reduce training data size by over 20% while maintaining performance.
  arXiv  Detail & Related papers  (2025-06-06T12:29:03Z)
• Q-Learning with Clustered-SMART (cSMART) Data: Examining Moderators in the Construction of Clustered Adaptive Interventions [3.9650359172757743]
  A clustered adaptive intervention (cAI) is a sequence of decision rules that guides practitioners on how best to tailor cluster-level intervention to improve outcomes.<n>We introduce a clustered Q-learning framework with the M-out-of-N Cluster Bootstrap to evaluate whether a set of candidate tailoring variables may be useful in defining an optimal cAI.
  arXiv  Detail & Related papers  (2025-05-01T19:24:39Z)
• Copula-based mixture model identification for subgroup clustering with imaging applications [2.285847431713438]
  We consider the more flexible Copula-Based Mixture Models (CBMMs) for clustering. CBMMs allow heterogeneous component distributions composed by flexible choices of marginal and copula forms.
  arXiv  Detail & Related papers  (2025-02-12T16:30:39Z)
• Demonstration Selection for In-Context Learning via Reinforcement Learning [16.103533806505403]
  Relevance-Diversity Enhanced Selection (RDES) is an innovative approach to optimize the selection of diverse reference demonstrations.<n>RDES employs frameworks like Q-learning and a PPO-based variant to dynamically identify demonstrations that maximize diversity.<n>We demonstrate that RDES significantly enhances performance compared to ten established baselines.
  arXiv  Detail & Related papers  (2024-12-05T08:33:52Z)
• CART: A Generative Cross-Modal Retrieval Framework with Coarse-To-Fine Semantic Modeling [53.97609687516371]
  Cross-modal retrieval aims to search for instances, which are semantically related to the query through the interaction of different modal data.<n>Traditional solutions utilize a single-tower or dual-tower framework to explicitly compute the score between queries and candidates.<n>We propose a generative cross-modal retrieval framework (CART) based on coarse-to-fine semantic modeling.
  arXiv  Detail & Related papers  (2024-06-25T12:47:04Z)
• Diversified Batch Selection for Training Acceleration [68.67164304377732]
  A prevalent research line, known as online batch selection, explores selecting informative subsets during the training process. vanilla reference-model-free methods involve independently scoring and selecting data in a sample-wise manner. We propose Diversified Batch Selection (DivBS), which is reference-model-free and can efficiently select diverse and representative samples.
  arXiv  Detail & Related papers  (2024-06-07T12:12:20Z)
• On the Use of Relative Validity Indices for Comparing Clustering Approaches [0.6990493129893111]
  Relative Validity Indices are widely used for evaluating and optimising clustering outcomes. There is a growing trend in the literature to use RVIs when selecting a Similarity Paradigm (SP) for clustering. This study presents the first comprehensive investigation into the reliability of RVIs for SP selection.
  arXiv  Detail & Related papers  (2024-04-16T07:39:54Z)
• Rethinking Clustered Federated Learning in NOMA Enhanced Wireless Networks [60.09912912343705]
  This study explores the benefits of integrating the novel clustered federated learning (CFL) approach with non-independent and identically distributed (non-IID) datasets. A detailed theoretical analysis of the generalization gap that measures the degree of non-IID in the data distribution is presented. Solutions to address the challenges posed by non-IID conditions are proposed with the analysis of the properties.
  arXiv  Detail & Related papers  (2024-03-05T17:49:09Z)
• Learning under Singularity: An Information Criterion improving WBIC and sBIC [1.0878040851637998]
  We introduce a novel Information Criterion (IC), termed Learning under Singularity (LS) LS is effective without regularity constraints and demonstrates stability. This approach offers a flexible and robust method for model selection, free from regularity constraints.
  arXiv  Detail & Related papers  (2024-02-20T07:09:39Z)
• Counterfactual Explanations Using Optimization With Constraint Learning [0.0]
  We propose a generic and flexible approach to counterfactual explanations using optimization with constraint learning (CE-OCL) Specifically, we discuss how we can leverage an optimization with constraint learning framework for the generation of counterfactual explanations. We also propose two novel modeling approaches to address data manifold closeness and diversity, which are two key criteria for practical counterfactual explanations.
  arXiv  Detail & Related papers  (2022-09-22T13:27:21Z)
• A One-shot Framework for Distributed Clustered Learning in Heterogeneous Environments [54.172993875654015]
  The paper proposes a family of communication efficient methods for distributed learning in heterogeneous environments. One-shot approach, based on local computations at the users and a clustering based aggregation step at the server is shown to provide strong learning guarantees. For strongly convex problems it is shown that, as long as the number of data points per user is above a threshold, the proposed approach achieves order-optimal mean-squared error rates in terms of the sample size.
  arXiv  Detail & Related papers  (2022-09-22T09:04:10Z)
• Using Representation Expressiveness and Learnability to Evaluate Self-Supervised Learning Methods [61.49061000562676]
  We introduce Cluster Learnability (CL) to assess learnability. CL is measured in terms of the performance of a KNN trained to predict labels obtained by clustering the representations with K-means. We find that CL better correlates with in-distribution model performance than other competing recent evaluation schemes.
  arXiv  Detail & Related papers  (2022-06-02T19:05:13Z)
• Multiple-criteria Based Active Learning with Fixed-size Determinantal Point Processes [43.71112693633952]
  We introduce a multiple-criteria based active learning algorithm, which incorporates three complementary criteria, i.e., informativeness, representativeness and diversity. We show that our method performs significantly better and is more stable than other multiple-criteria based AL algorithms.
  arXiv  Detail & Related papers  (2021-07-04T13:22:54Z)
• Joint Contrastive Learning with Infinite Possibilities [114.45811348666898]
  This paper explores useful modifications of the recent development in contrastive learning via novel probabilistic modeling. We derive a particular form of contrastive loss named Joint Contrastive Learning (JCL)
  arXiv  Detail & Related papers  (2020-09-30T16:24:21Z)
• Selective Inference for Latent Block Models [50.83356836818667]
  This study provides a selective inference method for latent block models. We construct a statistical test on a set of row and column cluster memberships of a latent block model. The proposed exact and approximated tests work effectively, compared to the naive test that did not take the selective bias into account.
  arXiv  Detail & Related papers  (2020-05-27T10:44:19Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.