UniICL: Systematizing Unified Multimodal In-context Learning through a Capability-Oriented Taxonomy

• URL: http://arxiv.org/abs/2603.24690v1
• Date: Wed, 25 Mar 2026 18:09:33 GMT
• Title: UniICL: Systematizing Unified Multimodal In-context Learning through a Capability-Oriented Taxonomy
• Abstract summary: In-context Learning efficacy is often non-monotonic and highly task-dependent.<n>We introduce a six-level capability-oriented taxonomy that categorizes the functional role of demonstrations.<n>We construct UniICL-760K, a large-scale corpus featuring curated 8-shot In-context Learning episodes.<n>As an architectural intervention, we propose the Context-Adaptive Prototype Modulator.
• Score: 99.95663439922937
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: In-context Learning enables training-free adaptation via demonstrations but remains highly sensitive to example selection and formatting. In unified multimodal models spanning understanding and generation, this sensitivity is exacerbated by cross-modal interference and varying cognitive demands. Consequently, In-context Learning efficacy is often non-monotonic and highly task-dependent. To diagnose these behaviors, we introduce a six-level capability-oriented taxonomy that categorizes the functional role of demonstrations from basic perception to high-order discernment. Guided by this cognitive framework, we construct UniICL-760K, a large-scale corpus featuring curated 8-shot In-context Learning episodes across 15 subtasks, alongside UniICL-Bench for rigorous, controlled evaluation. As an architectural intervention to stabilize few-shot adaptation, we propose the Context-Adaptive Prototype Modulator, a lightweight, plug-and-play module. Evaluations on UniICL-Bench show that our approach yields highly competitive unified results, outperforming larger-parameter multimodal large language model baselines on most understanding In-context Learning tasks. Data and code will be available soon at https://github.com/xuyicheng-zju/UniICL.

Related papers

• Unified Hallucination Fuzzing for Multimodal Large Language Models [44.4785399881315]
  Hallucination remains a persistent challenge for Multimodal Large Language Models (MLLMs)<n>Existing evaluations, predominantly based on static benchmarks, suffer from narrow taxonomical coverage and rapid performance saturation.<n>We present a systematic evaluation framework integrating a comprehensive benchmark with self-evolving stress testing.
  arXiv  Detail & Related papers  (2026-07-15T17:57:37Z)
• Hyper-ICL: Attention Calibration with Hyperbolic Anchor Distillation for Multimodal In-Context Learning [10.293023172874213]
  Hyper-ICL is a training-based framework for demonstration-free multimodal ICL.<n>It reconstructs demonstration effects directly without requiring ICDs at inference time.<n>It consistently improves accuracy and stability over vanilla ICL and existing state-of-the-art methods.
  arXiv  Detail & Related papers  (2026-06-03T04:32:09Z)
• Many-Shot CoT-ICL: Making In-Context Learning Truly Learn [58.439517684779936]
  In-context learning (ICL) adapts large language models to new tasks by conditioning on demonstrations in the prompt without parameter updates.<n>We study many-shot chain-of-thought in-context learning (CoT-ICL) for reasoning and show that standard many-shot rules do not transfer.
  arXiv  Detail & Related papers  (2026-05-13T13:30:12Z)
• Joint Semantic Token Selection and Prompt Optimization for Interpretable Prompt Learning [57.91658393667469]
  Interpretable Prompt Learning is a hybrid framework that alternates between discrete semantic token selection and continuous prompt optimization.<n>Our framework is plug-and-play, allowing seamless integration with existing prompt learning methods.
  arXiv  Detail & Related papers  (2026-05-06T02:38:59Z)
• HIFICL: High-Fidelity In-Context Learning for Multimodal Tasks [6.981025167004503]
  We introduce High-Fidelity In-Context Learning (HIFICL) to more faithfully model the ICL mechanism.<n>HIFICL consists of three key components: 1) a set of "virtual key-value pairs" to act as a learnable context, 2) a low-rank factorization for stable and regularized training, and 3) a simple end-to-end training objective.
  arXiv  Detail & Related papers  (2026-03-13T08:03:35Z)
• Schema for In-Context Learning [0.7850388075652649]
  In-context learning (ICL) enables language models to adapt to new tasks by conditioning on demonstration examples.<n>Inspired by cognitive science, we introduce SCHEMA ACTIVATED IN CONTEXT (SA-ICL)<n>This framework extracts the representation of the building blocks of cognition for the reasoning process instilled from prior examples.<n>We show that SA-ICL consistently boosts performance, up to 36.19 percent, when the single demonstration example is of high quality.
  arXiv  Detail & Related papers  (2025-10-14T21:00:15Z)
• Foundation Model for Skeleton-Based Human Action Understanding [56.89025287217221]
  This paper presents a Unified Skeleton-based Dense Representation Learning framework.<n>USDRL consists of a Transformer-based Dense Spatio-Temporal (DSTE), Multi-Grained Feature Decorrelation (MG-FD), and Multi-Perspective Consistency Training (MPCT)
  arXiv  Detail & Related papers  (2025-08-18T02:42:16Z)
• PUMA: Layer-Pruned Language Model for Efficient Unified Multimodal Retrieval with Modality-Adaptive Learning [54.73049408950049]
  We propose a Layer-Pruned Language Model for Efficient Unified Multimodal Retrieval with Modality-Adaptive Learning.<n>Our approach improves unified multimodal retrieval from both structural and learning perspectives.
  arXiv  Detail & Related papers  (2025-07-10T16:47:25Z)
• Will Pre-Training Ever End? A First Step Toward Next-Generation Foundation MLLMs via Self-Improving Systematic Cognition [89.50068130832635]
  Self-Improving cognition (SIcog) is a self-learning framework for constructing next-generation foundation MLLMs by multimodal knowledge.<n>We propose Chain-of-Description for step-by-step visual understanding and integrate structured Chain-of-Thought (CoT) reasoning to support in-depth multimodal reasoning.<n>Experiments demonstrate SIcog's effectiveness in developing MLLMs with enhanced multimodal cognition.
  arXiv  Detail & Related papers  (2025-03-16T00:25:13Z)
• ELICIT: LLM Augmentation via External In-Context Capability [16.237679215248196]
  alg is a framework consisting of two modules designed to effectively store and reuse task vectors.<n>alg serves as a plug-and-play performance booster to enable adaptive elicitation of model capabilities.
  arXiv  Detail & Related papers  (2024-10-12T03:19:06Z)
• Scaling In-Context Demonstrations with Structured Attention [75.41845145597875]
  We propose a better architectural design for in-context learning. Structured Attention for In-Context Learning replaces the full-attention by a structured attention mechanism. We show that SAICL achieves comparable or better performance than full attention while obtaining up to 3.4x inference speed-up.
  arXiv  Detail & Related papers  (2023-07-05T23:26:01Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.