GRaSp: Automatic Example Optimization for In-Context Learning in Low-Data Tasks

• URL: http://arxiv.org/abs/2605.07454v1
• Date: Fri, 08 May 2026 09:00:28 GMT
• Title: GRaSp: Automatic Example Optimization for In-Context Learning in Low-Data Tasks
• Abstract summary: In-context learning enables large language models to adapt to new tasks, but their performance is sensitive to the selected examples.<n>We propose GRaSp, a three-stage framework for automatic in-context example optimization.<n>By first generating a large synthetic candidate pool, then structuring it with clustering and dimensionality reduction, and finally using genetic algorithms to find the optimal in-context examples, the framework shows consistent improvements on the NER task.
• Score: 0.3374875022248866
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In-context learning enables large language models to adapt to new tasks, but their performance is highly sensitive to the selected examples. Finding effective demonstrations is particularly difficult in domain-specific, low-data settings where high-quality examples are scarce. We propose GRaSp, a three-stage framework for automatic in-context example optimization. By first generating a large synthetic candidate pool, then structuring it with clustering and dimensionality reduction, and finally using genetic algorithms to find the optimal in-context examples, the framework shows consistent improvements on the NER task. We also introduce a custom diversity-adaptive mutation mechanism, allowing it to transition from the initial broad inter-cluster exploration to focused intra-cluster refinement as the population converges. We evaluate GRaSp on financial named entity recognition (FiNER-139), comparing synthetic and human-annotated candidate pools across pool sizes of 500 and 5000. With non-synthetic data, GRaSp achieves 45.84% micro-F1, consistently outperforming both zero-shot and random few-shot baselines. Synthetic data matches the random baseline but does not exceed it, suggesting that distributional variety in the candidate pool is critical for generalization.

Related papers

• Spokes: Optimizing for Diverse Pretraining Data Selection [82.66872118512403]
  We introduce a probabilistic diversification framework based on the G-Vendi score, optimized via exponentiated gradient descent.<n>Our method produces subsets substantially more diverse than those obtained via random sampling.<n>We evaluate our approach on FineWeb and DCLM, where it consistently outperforms existing methods.
  arXiv  Detail & Related papers  (2026-06-13T09:28:46Z)
• Replacing Parameters with Preferences: Federated Alignment of Heterogeneous Vision-Language Models [63.70401095689976]
  Vision-Language Models (VLMs) have broad potential in privacy-sensitive domains such as healthcare and finance.<n>We propose MoR, a federated alignment framework that combines GRPO with Mixture-of-Rewards for heterogeneousVLMs.<n>MoR consistently outperforms federated alignment baselines in generalization and cross-client adaptability.
  arXiv  Detail & Related papers  (2026-05-05T07:02:50Z)
• Too Correct to Learn: Reinforcement Learning on Saturated Reasoning Data [55.84428098924793]
  We propose Constrained Uniform Top-K Sampling (CUTS), a parameter-free decoding strategy enforcing structure-preserving exploration.<n>We integrate this into Mixed-CUTS, a training framework synergizing exploitative and exploratory rollouts to amplify intra-group advantage variance.<n> Notably, Mixed-CUTS improves Pass@1 accuracy on the challenging AIME25 benchmark by up to 15.1% over standard GRPO.
  arXiv  Detail & Related papers  (2026-04-20T16:43:28Z)
• Prismatic Synthesis: Gradient-based Data Diversification Boosts Generalization in LLM Reasoning [77.120955854093]
  We show that data diversity can be a strong predictor of generalization in language models.<n>We introduce G-Vendi, a metric that quantifies diversity via the entropy of model-induced gradients.<n>We present Prismatic Synthesis, a framework for generating diverse synthetic data.
  arXiv  Detail & Related papers  (2025-05-26T16:05:10Z)
• Combatting Dimensional Collapse in LLM Pre-Training Data via Diversified File Selection [65.96556073745197]
  DiverSified File selection algorithm (DiSF) is proposed to select the most decorrelated text files in the feature space.<n>DiSF saves 98.5% of 590M training files in SlimPajama, outperforming the full-data pre-training within a 50B training budget.
  arXiv  Detail & Related papers  (2025-04-29T11:13:18Z)
• SoftSRV: Learn to Generate Targeted Synthetic Data [35.20014396557643]
  We present a novel framework, SoftSRV, that is used to generate targeted synthetic fine-tuning data for task-specific models.<n>We find that SoftSRV improves upon typical prompt engineering approaches, generating targeted data that leads to fine-tuned models.
  arXiv  Detail & Related papers  (2024-10-21T21:48:49Z)
• Unsupervised Domain Adaption for Neural Information Retrieval [18.97486314518283]
  We compare synthetic annotation by query generation using Large Language Models or rule-based string manipulation. We find that Large Language Models outperform rule-based methods in all scenarios by a large margin. In addition we explore several sizes of open Large Language Models to generate synthetic data and find that a medium-sized model suffices.
  arXiv  Detail & Related papers  (2023-10-13T18:27:33Z)
• Target-Aware Generative Augmentations for Single-Shot Adaptation [21.840653627684855]
  We propose a new approach to adapting models from a source domain to a target domain. SiSTA fine-tunes a generative model from the source domain using a single-shot target, and then employs novel sampling strategies for curating synthetic target data. We find that SiSTA produces significantly improved generalization over existing baselines in face detection and multi-class object recognition.
  arXiv  Detail & Related papers  (2023-05-22T17:46:26Z)
• Single-Shot Domain Adaptation via Target-Aware Generative Augmentation [21.17396588958938]
  We argue that augmentations utilized by existing methods are insufficient to handle large distribution shifts. We propose SiSTA (Single-Shot Target Augmentations), which first fine-tunes a generative model from the source domain using a single-shot target. We find that SiSTA produces improvements as high as 20% over existing baselines under challenging shifts in face detection.
  arXiv  Detail & Related papers  (2022-10-29T20:53:57Z)
• Score-based Generative Modeling in Latent Space [93.8985523558869]
  Score-based generative models (SGMs) have recently demonstrated impressive results in terms of both sample quality and distribution coverage. Here, we propose the Latent Score-based Generative Model (LSGM), a novel approach that trains SGMs in a latent space. Moving from data to latent space allows us to train more expressive generative models, apply SGMs to non-continuous data, and learn smoother SGMs in a smaller space.
  arXiv  Detail & Related papers  (2021-06-10T17:26:35Z)
• Recent Developments Combining Ensemble Smoother and Deep Generative Networks for Facies History Matching [58.720142291102135]
  This research project focuses on the use of autoencoders networks to construct a continuous parameterization for facies models. We benchmark seven different formulations, including VAE, generative adversarial network (GAN), Wasserstein GAN, variational auto-encoding GAN, principal component analysis (PCA) with cycle GAN, PCA with transfer style network and VAE with style loss.
  arXiv  Detail & Related papers  (2020-05-08T21:32:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.