Adaptive Constraint Propagation: Scaling Structured Inference for Large Language Models via Meta-Reinforcement Learning

• URL: http://arxiv.org/abs/2601.00095v2
• Date: Tue, 06 Jan 2026 03:24:45 GMT
• Title: Adaptive Constraint Propagation: Scaling Structured Inference for Large Language Models via Meta-Reinforcement Learning
• Abstract summary: MetaJuLS learns universal constraint propagation policies applicable across languages and tasks without task-specific retraining.<n>On Universal Dependencies across 10 languages, MetaJuLS adapts to new languages and tasks with 5--10 gradient steps (5--15 seconds) rather than requiring hours of task-specific training.<n>Mechanistic analysis reveals the policy discovers human-like parsing strategies (easy-first) and novel non-intuitives.
• Score: 6.908972852063454
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Large language models increasingly require structured inference, from JSON schema enforcement to multi-lingual parsing, where outputs must satisfy complex constraints. We introduce MetaJuLS, a meta-reinforcement learning approach that learns universal constraint propagation policies applicable across languages and tasks without task-specific retraining. By formulating structured inference as adaptive constraint propagation and training a Graph Attention Network with meta-learning, MetaJuLS achieves 1.5--2.0$\times$ speedups over GPU-optimized baselines while maintaining within 0.2\% accuracy of state-of-the-art parsers. On Universal Dependencies across 10 languages and LLM-constrained generation (LogicBench, GSM8K-Constrained), MetaJuLS demonstrates rapid cross-domain adaptation: a policy trained on English parsing adapts to new languages and tasks with 5--10 gradient steps (5--15 seconds) rather than requiring hours of task-specific training. Mechanistic analysis reveals the policy discovers human-like parsing strategies (easy-first) and novel non-intuitive heuristics. By reducing propagation steps in LLM deployments, MetaJuLS contributes to Green AI by directly reducing inference carbon footprint.

Related papers

• Parameter Alignment Mitigates Catastrophic Forgetting in Multilingual Expert Language Models [11.557479638024674]
  Organizing training around language families reduces cross-language interference but cannot alone prevent forgetting of the general knowledge needed for downstream tasks.<n>We present a suite of five layer-aware parameter alignment strategies: hard layer freezing, soft regularization, post-hoc weight reversion, and model merging.<n>We systematically compare our alignment strategies against two unregularized CPT baselines on benchmarks spanning 32 training languages from five language families, plus held-out languages.
  arXiv  Detail & Related papers  (2026-05-29T19:18:20Z)
• Learning to Route Languages for Multilingual Policy Optimization [50.044784120224335]
  Large language models are trained on heterogeneous multilingual corpora.<n>We propose language-routed policy optimization (LRPO), an online policy optimization framework that treats language as a selectable variable.
  arXiv  Detail & Related papers  (2026-05-25T02:28:41Z)
• COMPASS: COntinual Multilingual PEFT with Adaptive Semantic Sampling [0.023074632109535153]
  We introduce a data-centric framework for adapting large language models to target languages.<n>We use a distribution-aware sampling strategy to identify semantic gaps between existing training data and a target usage distribution.<n>We extend this into a continual learning framework, which monitors for data distribution shifts in production and dynamically updates adapters to prevent model staleness.
  arXiv  Detail & Related papers  (2026-04-22T16:07:10Z)
• LangGap: Diagnosing and Closing the Language Gap in Vision-Language-Action Models [4.54067274409672]
  Vision-Language-Action (VLA) models achieve over 95% success on standard benchmarks.<n>We find that current state-of-the-art VLA models largely ignore language instructions.<n>This paper constructs the LangGap benchmark, based on a four-dimensional semantic perturbation method.
  arXiv  Detail & Related papers  (2026-02-28T10:53:33Z)
• SpidR-Adapt: A Universal Speech Representation Model for Few-Shot Adaptation [40.55805997909858]
  We introduce SpidR-Adapt for rapid adaptation to new languages using minimal unlabeled data.<n>We construct a multi-task adaptive pre-training protocol which formulates the adaptation process as a bi-level optimization framework.<n> Empirically, SpidR-Adapt achieves rapid gains in phonemic discriminability and spoken language modeling.
  arXiv  Detail & Related papers  (2025-12-24T14:33:16Z)
• SCOPE: Language Models as One-Time Teacher for Hierarchical Planning in Text Environments [4.375012768093524]
  Long-term planning in text-based environments presents significant challenges due to open-ended action spaces, ambiguous observations, and sparse feedback.<n>Recent research suggests that large language models (LLMs) encode rich semantic knowledge about the world, which can be valuable for guiding agents in high-level reasoning and planning across both embodied and purely textual settings.<n>Existing approaches often depend heavily on querying LLMs during training and inference, making them computationally expensive and difficult to deploy efficiently.<n>We introduce SCOPE (Subgoal-COnditioned Pretraining for Efficient planning), a one-shot hierarchical planner that leverages LLM-generated subgoal
  arXiv  Detail & Related papers  (2025-12-10T18:26:14Z)
• Multi-Task Reinforcement Learning with Language-Encoded Gated Policy Networks [0.6345523830122167]
  Multi-task reinforcement learning often relies on task metadata to guide behavior across diverse objectives.<n>We present Lexical Policy Networks (LEXPOL), a language-conditioned mixture-of-policies architecture for multi-task RL.<n>LEXPOL encodes task metadata with a text encoder and uses a learned gating module to select or blend among sub-policies.
  arXiv  Detail & Related papers  (2025-10-07T17:12:24Z)
• Beyond In-Context Learning: Aligning Long-form Generation of Large Language Models via Task-Inherent Attribute Guidelines [71.14354526117958]
  In-context learning (ICL) is an important yet not fully understood ability of pre-trained large language models (LLMs)<n>We present LongGuide, which efficiently generates two parallel streams of guidelines capturing task language and format properties.<n>LongGuide automatically selects the best combination of guidelines, improving both strong open- and closed-source LLMs by over 5% in both zero- and few-shot settings.
  arXiv  Detail & Related papers  (2025-06-02T02:35:24Z)
• Enhancing LLM Language Adaption through Cross-lingual In-Context Pre-training [57.62126373849383]
  Cross-lingual In-context Pre-training (CrossIC-PT) is a simple and scalable approach that enhances cross-lingual transfer.<n>We construct CrossIC-PT samples by interleaving semantic-related bilingual Wikipedia documents into a single context window.<n> Experimental results demonstrate that CrossIC-PT improves multilingual performance on three models across six target languages.
  arXiv  Detail & Related papers  (2025-04-29T07:24:25Z)
• Optimizing Low-Resource Language Model Training: Comprehensive Analysis of Multi-Epoch, Multi-Lingual, and Two-Stage Approaches [3.809045695573932]
  Existing works adopt multi-epoch, multi-lingual, and two-stage training to utilize the limited target language corpus efficiently. We exhaustively explore training setups for low-resource language LLM, combining these three approaches. As the amount of target language corpus decreases, the optimal training approach shifts from monolingual single-stage training to multi-lingual two-stage training at a compute budget dependent threshold.
  arXiv  Detail & Related papers  (2024-10-16T07:45:56Z)
• Learning to Learn with Contrastive Meta-Objective [48.27877062976768]
  We propose to exploit task identity as additional supervision in meta-training.<n>The proposed ConML is evaluating and optimizing the contrastive meta-objective.<n>We demonstrate that ConML integrates seamlessly with existing meta-learners, as well as in-context learning models.
  arXiv  Detail & Related papers  (2024-10-08T12:22:10Z)
• SMILE: Speech Meta In-Context Learning for Low-Resource Language Automatic Speech Recognition [55.2480439325792]
  Speech Meta In-Context LEarning (SMILE) is an innovative framework that combines meta-learning with speech in-context learning (SICL)<n>We show that SMILE consistently outperforms baseline methods in training-free few-shot multilingual ASR tasks.
  arXiv  Detail & Related papers  (2024-09-16T16:04:16Z)
• Large Language Models as Generalizable Policies for Embodied Tasks [50.870491905776305]
  We show that large language models (LLMs) can be adapted to be generalizable policies for embodied visual tasks. Our approach, called Large LAnguage model Reinforcement Learning Policy (LLaRP), adapts a pre-trained frozen LLM to take as input text instructions and visual egocentric observations and output actions directly in the environment.
  arXiv  Detail & Related papers  (2023-10-26T18:32:05Z)
• Soft Language Clustering for Multilingual Model Pre-training [57.18058739931463]
  We propose XLM-P, which contextually retrieves prompts as flexible guidance for encoding instances conditionally. Our XLM-P enables (1) lightweight modeling of language-invariant and language-specific knowledge across languages, and (2) easy integration with other multilingual pre-training methods.
  arXiv  Detail & Related papers  (2023-06-13T08:08:08Z)
• Pre-training Text Representations as Meta Learning [113.3361289756749]
  We introduce a learning algorithm which directly optimize model's ability to learn text representations for effective learning of downstream tasks. We show that there is an intrinsic connection between multi-task pre-training and model-agnostic meta-learning with a sequence of meta-train steps.
  arXiv  Detail & Related papers  (2020-04-12T09:05:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.