TIAO: Token Importance-Aware Policy Optimization for Text Summarization

• URL: http://arxiv.org/abs/2609.16748v1
• Date: Tue, 15 Sep 2026 07:23:07 GMT
• Title: TIAO: Token Importance-Aware Policy Optimization for Text Summarization
• Abstract summary: We propose Token Importance-Aware Policy Optimization (TIAO), a novel reinforcement learning strategy that explicitly leverages token-importance awareness.<n>TIAO identifies core tokens based on token dependency and reweights a trajectory's advantage according to its overall dependencies.<n>Experiments on the real world dataset show that our TIAO achieves highly competitive results.
• Score: 3.362110408874633
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Text summarization requires models to condense content while preserving key qualities such as consistency and coherence. Large language models (LLMs) have shown strong performance on this task and can be further improved through reinforcement learning (RL). However, most existing methods apply reward signals directly to undifferentiated token sequences, overlooking the varying importance of individual tokens to word and sentence level quality in summarization. In this paper, we propose Token Importance-Aware Policy Optimization (TIAO), a novel reinforcement learning strategy that explicitly leverages token-importance awareness. Specifically, TIAO identifies core tokens based on token dependency and reweights a trajectory's advantage according to its overall dependencies. Experiments on the real world dataset show that our TIAO achieves highly competitive results, and that a 7B foundation model enhanced by TIAO performs comparably to GPT-4 and GPT-5-nano. Code is available at https://github.com/TechCloud-x/TIAO

Related papers

• PRPO: Perception-Reinforced Policy Optimization via Token-Level Dynamic Advantage Reshaping [5.89473045822308]
  Reinforcement Learning with Verifiable Rewards (RLVR) has become an effective paradigm for improving the reasoning capability of Large Vision-Language Models (LVLMs)<n>Existing RLVR methods rely on trajectory-level outcome rewards, which assign identical learning signals across all generated tokens.<n>We propose Perception-Reinforced Policy Optimization (PRPO), a token-level reinforcement learning framework that explicitly identifies and reinforces pivotal perceptual tokens.
  arXiv  Detail & Related papers  (2026-06-07T16:06:29Z)
• Unlearning What Matters: Token-Level Attribution for Precise Language Model Unlearning [5.454773103061359]
  TokenUnlearn is a token-level attribution framework that identifies and selectively targets critical tokens.<n>Our approach combines knowledge-aware signals via masking, and entropy-aware signals to yield importance scores for precise token selection.
  arXiv  Detail & Related papers  (2026-05-01T02:59:03Z)
• Beyond Attention Magnitude: Leveraging Inter-layer Rank Consistency for Efficient Vision-Language-Action Models [75.27089289058613]
  We introduce textbfTIES (textbfTau-guided textbfInter-layer textbfEfficient textbfSelection), a dynamic framework guided by inter-layer token ranking consistency.<n>On the CogACT + SIMPLER benchmark, TIES improves average success rates by 6% while reducing token usage by 78%, and demonstrate strong generalization across diverse decoders and benchmarks.
  arXiv  Detail & Related papers  (2026-03-26T02:13:03Z)
• Forgetting: A New Mechanism Towards Better Large Language Model Fine-tuning [51.92313556418432]
  Supervised fine-tuning (SFT) plays a critical role for pretrained large language models (LLMs)<n>We suggest categorizing tokens within each corpus into two parts -- positive and negative tokens -- based on whether they are useful to improve model performance.<n>We conduct experiments on well-established benchmarks, finding that this forgetting mechanism not only improves overall model performance and also facilitate more diverse model responses.
  arXiv  Detail & Related papers  (2025-08-06T11:22:23Z)
• Resource-Efficient Adaptation of Large Language Models for Text Embeddings via Prompt Engineering and Contrastive Fine-tuning [3.9914181590063884]
  Large Language Models (LLMs) have become a cornerstone in Natural Language Processing (NLP)<n>We explore several adaptation strategies for pre-trained, decoder-only LLMs.
  arXiv  Detail & Related papers  (2025-07-30T14:49:30Z)
• Not All Preferences are What You Need for Post-Training: Selective Alignment Strategy for Preference Optimization [0.0]
  Post-training alignment of large language models (LLMs) is a critical challenge, as not all tokens contribute equally to model performance.<n>This paper introduces a selective alignment strategy that prioritizes high-impact tokens within preference pairs.<n>By focusing on these informative tokens, our approach reduces computational overhead and enhances alignment fidelity.
  arXiv  Detail & Related papers  (2025-07-10T12:58:45Z)
• IGD: Token Decisiveness Modeling via Information Gain in LLMs for Personalized Recommendation [79.22388408461458]
  We introduce an Information Gain-based Decisiveness-aware Token handling (IGD) strategy that integrates token decisiveness into both tuning and decoding.<n>IGD consistently improves recommendation accuracy, achieving significant gains on widely used ranking metrics compared to strong baselines.
  arXiv  Detail & Related papers  (2025-06-16T08:28:19Z)
• Token Assorted: Mixing Latent and Text Tokens for Improved Language Model Reasoning [53.57895922042783]
  Large Language Models (LLMs) excel at reasoning and planning when trained on chainof-thought (CoT) data.<n>We propose a hybrid representation of the reasoning process, where we partially abstract away the initial reasoning steps using latent discrete tokens.
  arXiv  Detail & Related papers  (2025-02-05T15:33:00Z)
• KNN Transformer with Pyramid Prompts for Few-Shot Learning [52.735070934075736]
  Few-Shot Learning aims to recognize new classes with limited labeled data. Recent studies have attempted to address the challenge of rare samples with textual prompts to modulate visual features.
  arXiv  Detail & Related papers  (2024-10-14T07:39:30Z)
• SEP: Self-Enhanced Prompt Tuning for Visual-Language Model [93.94454894142413]
  We introduce a novel approach named Self-Enhanced Prompt Tuning (SEP) SEP explicitly incorporates discriminative prior knowledge to enhance both textual-level and visual-level embeddings. Comprehensive evaluations across various benchmarks and tasks confirm SEP's efficacy in prompt tuning.
  arXiv  Detail & Related papers  (2024-05-24T13:35:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.