Convergent Emergence of In-Context Learning Across Modalities

• URL: http://arxiv.org/abs/2609.14011v1
• Date: Sat, 12 Sep 2026 15:55:30 GMT
• Title: Convergent Emergence of In-Context Learning Across Modalities
• Abstract summary: Few-shot in-context learning (ICL) has been extensively studied in large language models trained for next-token prediction on human text.<n>We show that paired-mapping ICL emerges across six modalities, surpasses controlled baselines, and has correlated per-task effects across five of them.
• Score: 35.460104283005926
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Few-shot in-context learning (ICL), the capacity of a model to infer abstract patterns from input-output examples provided in its prompt and apply them to new inputs, has been extensively studied in large language models trained for next-token prediction on human text. Recently, few-shot ICL has been demonstrated in autoregressive genomic models as well. This raises a question: does ICL emerge broadly across domains, and if so, what common structure is shared? To address both, we develop a controlled cross-modality framework that instantiates the same task suite in a variety of modalities to test what we call the Convergent Emergence Hypothesis: the idea that few-shot ICL, when it emerges, shares a common cross-modality difficulty profile - i.e., tasks that benefit from ICL in one modality tend to benefit in others. We show that paired-mapping ICL emerges across six modalities (language, genome, integer sequences, time series, images, and proteins), surpasses controlled baselines, and has correlated per-task effects across five of them. Together, these results provide support for the Convergent Emergence Hypothesis in some modalities, but not all.

Related papers

• TwinICL: Diagnosing Multimodal In-Context Learning through Paired Counterfactuals [49.800443965874756]
  In-context learning (ICL) enables models to infer tasks from demonstrations.<n>Existing benchmarks generally lack matched text and image versions needed to compare ICL performance across modalities.<n>We introduce TwinICL, a procedurally generated benchmark providing such pairs for controlled comparison.
  arXiv  Detail & Related papers  (2026-09-14T04:42:42Z)
• Many-Shot CoT-ICL: Making In-Context Learning Truly Learn [58.439517684779936]
  In-context learning (ICL) adapts large language models to new tasks by conditioning on demonstrations in the prompt without parameter updates.<n>We study many-shot chain-of-thought in-context learning (CoT-ICL) for reasoning and show that standard many-shot rules do not transfer.
  arXiv  Detail & Related papers  (2026-05-13T13:30:12Z)
• PLUME: Latent Reasoning Based Universal Multimodal Embedding [52.35354073629127]
  Universal multimodal embedding (UME) maps heterogeneous inputs into a shared retrieval space with a single model.<n>Recent approaches improve UME by generating explicit chain-of-thought (CoT) rationales before extracting embeddings.<n>We propose PLUME, a latent reasoning framework that advances UME by replacing verbalized CoT with a short autoregressive rollout of continuous latent states.
  arXiv  Detail & Related papers  (2026-04-02T14:04:53Z)
• Demonstrations, CoT, and Prompting: A Theoretical Analysis of ICL [9.83462430479849]
  In-Context Learning (ICL) enables pretrained LLMs to adapt to downstream tasks by conditioning on a small set of input-output demonstrations.<n>We analyze how ICL performance is governed by (i) the quality of selected demonstrations, (ii) an intrinsic ICL capability of the pretrained model, and (iii) the degree of distribution shift.
  arXiv  Detail & Related papers  (2026-03-20T03:33:04Z)
• Beyond the Prompt in Large Language Models: Comprehension, In-Context Learning, and Chain-of-Thought [15.598263332303612]
  Large Language Models (LLMs) have demonstrated remarkable proficiency across diverse tasks.<n>This study dives into the foundations of these observations by addressing three critical questions.
  arXiv  Detail & Related papers  (2026-02-16T10:57:26Z)
• UniT: Unified Multimodal Chain-of-Thought Test-time Scaling [85.590774707406]
  Unified models can handle both multimodal understanding and generation within a single architecture, yet they typically operate in a single pass without iteratively refining their outputs.<n>We introduce UniT, a framework for multimodal test-time scaling that enables a single unified model to reason, verify, and refine across multiple rounds.
  arXiv  Detail & Related papers  (2026-02-12T18:59:49Z)
• Contextualize-then-Aggregate: Circuits for In-Context Learning in Gemma-2 2B [51.74607395697567]
  In-Context Learning (ICL) is an intriguing ability of large language models (LLMs)<n>We use causal interventions to identify information flow in Gemma-2 2B for five naturalistic ICL tasks.<n>We find that the model infers task information using a two-step strategy we call contextualize-then-aggregate.
  arXiv  Detail & Related papers  (2025-03-31T18:33:55Z)
• In-Context Learning with Hypothesis-Class Guidance [9.323460118695932]
  In this work, we propose ICL with hypothesis-class guidance (ICL-HCG)<n>Under our framework ICL-HCG, we conduct extensive experiments to explore: (i) a variety of generalization abilities to new hypothesis classes; (ii) different model architectures; (iii) sample complexity; (iv) the role of instruction; and (vi) the effect of pretraining hypothesis diversity.
  arXiv  Detail & Related papers  (2025-02-27T05:52:06Z)
• Towards Auto-Regressive Next-Token Prediction: In-Context Learning Emerges from Generalization [26.9153121765435]
  Large language models (LLMs) have demonstrated remarkable in-context learning abilities.<n>This paper investigates how ICL emerges and the impact of pre-training phase on ICL.<n>Our theory is supported by experiments on numerical linear dynamic systems, synthetic GINC and real-world language datasets.
  arXiv  Detail & Related papers  (2025-02-24T10:26:29Z)
• Competition Dynamics Shape Algorithmic Phases of In-Context Learning [10.974593590868533]
  In-Context Learning (ICL) has significantly expanded the general-purpose nature of large language models.<n>We propose a synthetic sequence modeling task that involves learning to simulate a finite mixture of Markov chains.<n>We show we can explain a model's behavior by decomposing it into four broad algorithms that combine a fuzzy retrieval vs. inference approach with either unigram or bigram statistics.
  arXiv  Detail & Related papers  (2024-12-01T23:35:53Z)
• In-context Learning Generalizes, But Not Always Robustly: The Case of Syntax [36.98247762224868]
  In-context learning (ICL) is now a common method for teaching large language models (LLMs) new tasks. Do models infer the underlying structure of the task defined by the context, or do they rely on superficial generalizations that only generalize to identically distributed examples? In experiments with models from the GPT, PaLM, and Llama 2 families, we find large variance across LMs. The variance is explained more by the composition of the pre-training corpus and supervision methods than by model size.
  arXiv  Detail & Related papers  (2023-11-13T23:52:43Z)
• Synergies between Disentanglement and Sparsity: Generalization and Identifiability in Multi-Task Learning [79.83792914684985]
  We prove a new identifiability result that provides conditions under which maximally sparse base-predictors yield disentangled representations. Motivated by this theoretical result, we propose a practical approach to learn disentangled representations based on a sparsity-promoting bi-level optimization problem.
  arXiv  Detail & Related papers  (2022-11-26T21:02:09Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.