LongAct: Harnessing Intrinsic Activation Patterns for Long-Context Reinforcement Learning

• URL: http://arxiv.org/abs/2604.14922v1
• Date: Thu, 16 Apr 2026 12:06:59 GMT
• Title: LongAct: Harnessing Intrinsic Activation Patterns for Long-Context Reinforcement Learning
• Abstract summary: LongAct is a strategy that shifts from uniform to saliency-guided sparse updates.<n>By selectively updating only the weights associated with these significant activations, LongAct achieves an approximate 8% improvement on LongBench v2.
• Score: 54.6434454250226
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Reinforcement Learning (RL) has emerged as a critical driver for enhancing the reasoning capabilities of Large Language Models (LLMs). While recent advancements have focused on reward engineering or data synthesis, few studies exploit the model's intrinsic representation characteristics to guide the training process. In this paper, we first observe the presence of high-magnitude activations within the query and key vectors when processing long contexts. Drawing inspiration from model quantization -- which establishes the criticality of such high-magnitude activations -- and the insight that long-context reasoning inherently exhibits a sparse structure, we hypothesize that these weights serve as the pivotal drivers for effective model optimization. Based on this insight, we propose LongAct, a strategy that shifts from uniform to saliency-guided sparse updates. By selectively updating only the weights associated with these significant activations, LongAct achieves an approximate 8% improvement on LongBench v2 and enhances generalization on the RULER benchmark. Furthermore, our method exhibits remarkable universality, consistently boosting performance across diverse RL algorithms such as GRPO and DAPO. Extensive ablation studies suggest that focusing on these salient features is key to unlocking long-context potential.

Related papers

• Representation Learning Enables Scalable Multitask Deep Reinforcement Learning [32.45608314596038]
  We show that combining predictive, model-based representations with high-capacity value function approximation is sufficient to achieve strong performance.<n>We evaluate a simple model-free algorithm, MR.Q, coupled with auxiliary predictive objectives into a scalable actor-critic architecture.
  arXiv  Detail & Related papers  (2026-06-04T01:09:20Z)
• Why Does Reinforcement Learning Generalize? A Feature-Level Mechanistic Study of Post-Training in Large Language Models [51.45122910254346]
  Reinforcement learning (RL)-based post-training often improves the reasoning performance of large language models beyond the training domain.<n>Supervised fine-tuning (SFT) frequently leads to general forgetting capabilities.<n>We present a feature-level mechanistic analysis methodology to probe RL generalization using a controlled experimental setup.
  arXiv  Detail & Related papers  (2026-04-27T21:22:34Z)
• Insight-V++: Towards Advanced Long-Chain Visual Reasoning with Multimodal Large Language Models [65.4947731385794]
  We present a unified multi-agent visual reasoning framework that evolves from our foundational image-centric model, Insight-V.<n>We introduce two novel algorithms, ST-GRPO and J-GRPO, which enhance spatial-temporal reasoning and improve evaluative robustness.<n>Experiments on base models like LLaVA-NeXT and Qwen2.5-VL demonstrate significant performance gains across challenging image and video reasoning benchmarks.
  arXiv  Detail & Related papers  (2026-03-18T15:28:07Z)
• LongR: Unleashing Long-Context Reasoning via Reinforcement Learning with Dense Utility Rewards [57.993003392037174]
  LongR is a framework that enhances long-context performance by integrating a dynamic "Think-and-Read" mechanism.<n>LongR consistently enhances performance across diverse RL algorithms.
  arXiv  Detail & Related papers  (2026-02-05T15:26:47Z)
• Nemotron-Cascade: Scaling Cascaded Reinforcement Learning for General-Purpose Reasoning Models [71.9060068259379]
  We propose cascaded domain-wise reinforcement learning to build general-purpose reasoning models.<n>Our 14B model, after RL, outperforms its SFT teacher, DeepSeek-R1-0528, on LiveCodeBench v5/v6 Pro and silver-medal performance in the 2025 International Olympiad in Informatics (IOI)
  arXiv  Detail & Related papers  (2025-12-15T18:02:35Z)
• Tailored Primitive Initialization is the Secret Key to Reinforcement Learning [61.29280885291581]
  Reinforcement learning (RL) has emerged as a powerful paradigm for enhancing the reasoning capabilities of large language models (LLMs)<n>We argue that initializing LLMs with diverse, high-quality reasoning primitives is essential for achieving stable and sample-efficient RL training.<n>We propose Tailor, a finetuning pipeline that automatically discovers and curates novel reasoning primitives.
  arXiv  Detail & Related papers  (2025-11-16T03:12:40Z)
• Scaling Up RL: Unlocking Diverse Reasoning in LLMs via Prolonged Training [121.5858973157225]
  We investigate the effects of prolonged reinforcement learning on a small language model across a diverse set of reasoning domains.<n>We introduce controlled KL regularization, clipping ratio, and periodic reference policy resets as critical components for unlocking long-term performance gains.<n>Our model achieves significant improvements over strong baselines, including +14.7% on math, +13.9% on coding, and +54.8% on logic puzzle tasks.
  arXiv  Detail & Related papers  (2025-07-16T17:59:24Z)
• Activation Control for Efficiently Eliciting Long Chain-of-thought Ability of Language Models [45.938663388013445]
  We show that a small set of high-impact activations in the last few layers governs long-form reasoning attributes.<n>By simply amplifying these activations and inserting "wait" tokens, we can invoke the long CoT ability without any training.
  arXiv  Detail & Related papers  (2025-05-23T10:07:18Z)
• Benchmarking General-Purpose In-Context Learning [19.40952728849431]
  In-context learning (ICL) empowers generative models to address new tasks effectively and efficiently on the fly. In this paper, we study extending ICL to address a broader range of tasks with an extended learning horizon and higher improvement potential. We introduce two benchmarks specifically crafted to train and evaluate GPICL functionalities.
  arXiv  Detail & Related papers  (2024-05-27T14:50:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.