Cohort-Based Active Modality Acquisition

• URL: http://arxiv.org/abs/2505.16791v1
• Date: Thu, 22 May 2025 15:32:50 GMT
• Title: Cohort-Based Active Modality Acquisition
• Abstract summary: We introduce cohort-based Active Modality Acquisition (CAMA) to formalize the challenge of selecting which samples should receive additional modalities.<n>We derive acquisition strategies that leverage a combination of generative imputation and discriminative modeling.<n>Experiments on common multimodal datasets demonstrate that our proposed imputation-based strategies can more effectively guide the acquisition of new samples.
• Score: 0.27271717351505986
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Real-world machine learning applications often involve data from multiple modalities that must be integrated effectively to make robust predictions. However, in many practical settings, not all modalities are available for every sample, and acquiring additional modalities can be costly. This raises the question: which samples should be prioritized for additional modality acquisition when resources are limited? While prior work has explored individual-level acquisition strategies and training-time active learning paradigms, test-time and cohort-based acquisition remain underexplored despite their importance in many real-world settings. We introduce Cohort-based Active Modality Acquisition (CAMA), a novel test-time setting to formalize the challenge of selecting which samples should receive additional modalities. We derive acquisition strategies that leverage a combination of generative imputation and discriminative modeling to estimate the expected benefit of acquiring missing modalities based on common evaluation metrics. We also introduce upper-bound heuristics that provide performance ceilings to benchmark acquisition strategies. Experiments on common multimodal datasets demonstrate that our proposed imputation-based strategies can more effectively guide the acquisition of new samples in comparison to those relying solely on unimodal information, entropy guidance, and random selections. Our work provides an effective solution for optimizing modality acquisition at the cohort level, enabling better utilization of resources in constrained settings.

Related papers

• Data-dependent Exploration for Online Reinforcement Learning from Human Feedback [50.34161049551627]
  Online reinforcement learning from human feedback (RLHF) has emerged as a promising paradigm for aligning large language models (LLMs) by continuously collecting new preference feedback during training.<n>Existing exploration strategies often derive bonuses via on-policy expectations, which are difficult to estimate reliably from the limited historical preference data available during training.<n>We propose data-dependent exploration for preference optimization (DEPO), a simple and scalable method that leverages historical data to construct an extra uncertainty bonus for high-uncertainty regions.
  arXiv  Detail & Related papers  (2026-05-06T03:56:45Z)
• Do We Truly Need So Many Samples? Multi-LLM Repeated Sampling Efficiently Scales Test-Time Compute [54.22256089592864]
  This paper presents a simple, effective, and cost-efficient strategy to improve LLM performance by scaling test-time compute.<n>Our strategy builds upon the repeated-sampling-then-voting framework, with a novel twist: incorporating multiple models, even weaker ones, to leverage their complementary strengths.
  arXiv  Detail & Related papers  (2025-04-01T13:13:43Z)
• Prototype-Guided Cross-Modal Knowledge Enhancement for Adaptive Survival Prediction [12.14067843712642]
  ProSurv is a prototype-guided cross-modal knowledge enhancement framework.<n>It eliminates the dependency on paired data and enables robust learning and adaptive survival prediction.<n>Extensive experiments on four public datasets demonstrate the superiority of ProSurv over state-of-the-art methods.
  arXiv  Detail & Related papers  (2025-03-13T11:38:11Z)
• Towards Modality Generalization: A Benchmark and Prospective Analysis [56.84045461854789]
  This paper introduces Modality Generalization (MG), which focuses on enabling models to generalize to unseen modalities.<n>We propose a comprehensive benchmark featuring multi-modal algorithms and adapt existing methods that focus on generalization.<n>Our work provides a foundation for advancing robust and adaptable multi-modal models, enabling them to handle unseen modalities in realistic scenarios.
  arXiv  Detail & Related papers  (2024-12-24T08:38:35Z)
• Model-Free Active Exploration in Reinforcement Learning [53.786439742572995]
  We study the problem of exploration in Reinforcement Learning and present a novel model-free solution. Our strategy is able to identify efficient policies faster than state-of-the-art exploration approaches.
  arXiv  Detail & Related papers  (2024-06-30T19:00:49Z)
• Diversified Batch Selection for Training Acceleration [68.67164304377732]
  A prevalent research line, known as online batch selection, explores selecting informative subsets during the training process. vanilla reference-model-free methods involve independently scoring and selecting data in a sample-wise manner. We propose Diversified Batch Selection (DivBS), which is reference-model-free and can efficiently select diverse and representative samples.
  arXiv  Detail & Related papers  (2024-06-07T12:12:20Z)
• Borrowing Treasures from Neighbors: In-Context Learning for Multimodal Learning with Missing Modalities and Data Scarcity [9.811378971225727]
  This paper extends the current research into missing modalities to the low-data regime. It is often expensive to get full-modality data and sufficient annotated training samples. We propose to use retrieval-augmented in-context learning to address these two crucial issues.
  arXiv  Detail & Related papers  (2024-03-14T14:19:48Z)
• Bridging Diversity and Uncertainty in Active learning with Self-Supervised Pre-Training [21.57943896942296]
  This study addresses the integration of diversity-based and uncertainty-based sampling strategies in active learning.<n>We introduce a straightforward called TCM that mitigates the cold start problem while maintaining strong performance across various data levels.
  arXiv  Detail & Related papers  (2024-03-06T14:18:24Z)
• Learning to Rank for Active Learning via Multi-Task Bilevel Optimization [29.207101107965563]
  We propose a novel approach for active learning, which aims to select batches of unlabeled instances through a learned surrogate model for data acquisition. A key challenge in this approach is developing an acquisition function that generalizes well, as the history of data, which forms part of the utility function's input, grows over time.
  arXiv  Detail & Related papers  (2023-10-25T22:50:09Z)
• Towards Balanced Active Learning for Multimodal Classification [15.338417969382212]
  Training multimodal networks requires a vast amount of data due to their larger parameter space compared to unimodal networks. Active learning is a widely used technique for reducing data annotation costs by selecting only those samples that could contribute to improving model performance. Current active learning strategies are mostly designed for unimodal tasks, and when applied to multimodal data, they often result in biased sample selection from the dominant modality.
  arXiv  Detail & Related papers  (2023-06-14T07:23:36Z)
• HyperImpute: Generalized Iterative Imputation with Automatic Model Selection [77.86861638371926]
  We propose a generalized iterative imputation framework for adaptively and automatically configuring column-wise models. We provide a concrete implementation with out-of-the-box learners, simulators, and interfaces.
  arXiv  Detail & Related papers  (2022-06-15T19:10:35Z)
• SURF: Semi-supervised Reward Learning with Data Augmentation for Feedback-efficient Preference-based Reinforcement Learning [168.89470249446023]
  We present SURF, a semi-supervised reward learning framework that utilizes a large amount of unlabeled samples with data augmentation. In order to leverage unlabeled samples for reward learning, we infer pseudo-labels of the unlabeled samples based on the confidence of the preference predictor. Our experiments demonstrate that our approach significantly improves the feedback-efficiency of the preference-based method on a variety of locomotion and robotic manipulation tasks.
  arXiv  Detail & Related papers  (2022-03-18T16:50:38Z)
• Single-Modal Entropy based Active Learning for Visual Question Answering [75.1682163844354]
  We address Active Learning in the multi-modal setting of Visual Question Answering (VQA) In light of the multi-modal inputs, image and question, we propose a novel method for effective sample acquisition. Our novel idea is simple to implement, cost-efficient, and readily adaptable to other multi-modal tasks.
  arXiv  Detail & Related papers  (2021-10-21T05:38:45Z)
• Optimal Ensemble Construction for Multi-Study Prediction with Applications to COVID-19 Excess Mortality Estimation [7.02598981483736]
  Multi-study ensembling uses a two-stage strategy which fits study-specific models and estimates ensemble weights separately. This approach ignores the ensemble properties at the model-fitting stage, potentially resulting in a loss of efficiency. We show that when little data is available for a country before the onset of the pandemic, leveraging data from other countries can substantially improve prediction accuracy.
  arXiv  Detail & Related papers  (2021-09-19T16:52:41Z)
• Semi-supervised Batch Active Learning via Bilevel Optimization [89.37476066973336]
  We formulate our approach as a data summarization problem via bilevel optimization. We show that our method is highly effective in keyword detection tasks in the regime when only few labeled samples are available.
  arXiv  Detail & Related papers  (2020-10-19T16:53:24Z)
• Meta-Reinforcement Learning Robust to Distributional Shift via Model Identification and Experience Relabeling [126.69933134648541]
  We present a meta-reinforcement learning algorithm that is both efficient and extrapolates well when faced with out-of-distribution tasks at test time. Our method is based on a simple insight: we recognize that dynamics models can be adapted efficiently and consistently with off-policy data.
  arXiv  Detail & Related papers  (2020-06-12T13:34:46Z)
• Toward Optimal Probabilistic Active Learning Using a Bayesian Approach [4.380488084997317]
  Active learning aims at reducing the labeling costs by an efficient and effective allocation of costly labeling resources. By reformulating existing selection strategies within our proposed model, we can explain which aspects are not covered in current state-of-the-art.
  arXiv  Detail & Related papers  (2020-06-02T15:59:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.