Enhancing Chain-of-Thought Reasoning with Critical Representation Fine-tuning

• URL: http://arxiv.org/abs/2507.10085v1
• Date: Mon, 14 Jul 2025 09:11:33 GMT
• Title: Enhancing Chain-of-Thought Reasoning with Critical Representation Fine-tuning
• Authors: Chenxi Huang, Shaotian Yan, Liang Xie, Binbin Lin, Sinan Fan, Yue Xin, Deng Cai, Chen Shen, Jieping Ye,
• Abstract summary: Representation Fine-tuning (ReFT) has attracted widespread attention for significantly improving parameter efficiency by editing representation space alone.<n>We propose Critical Representation Fine-Tuning (CRFT), a novel method that identifies and optimize these critical representations through information flow analysis.<n>Our method is validated across eight benchmarks for arithmetic and commonsense reasoning, using LLaMA and Mistral model families.
• Score: 37.16998362490576
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Representation Fine-tuning (ReFT), a recently proposed Parameter-Efficient Fine-Tuning (PEFT) method, has attracted widespread attention for significantly improving parameter efficiency by editing representation space alone. In this work, we investigate applying ReFT to complex reasoning tasks. However, directly using the native ReFT method, which modifies fixed representations at the beginning and end of each layer, yields suboptimal performance, as these fixed-position representations have uncertain impact on the outputs. We observe that, in complex reasoning tasks, there often exist certain critical representations. These representations either integrate significant information from preceding layers or regulate subsequent layer representations. Through layer-by-layer propagation, they exert a substantial influence on the final output. Naturally, fine-tuning these critical representations has the potential to greatly enhance reasoning performance. Building upon these insights, we propose Critical Representation Fine-Tuning (CRFT), a novel method that identifies and optimizes these critical representations through information flow analysis. CRFT operates within a supervised learning framework, dynamically optimizing critical representations in a low-rank linear subspace while freezing the base model. The effectiveness and efficiency of our method are validated across eight benchmarks for arithmetic and commonsense reasoning, using LLaMA and Mistral model families. Furthermore, our method also adapts effectively to few-shot settings, boosting one-shot accuracy by 16.4%. Our work highlights the untapped potential of representation-level optimization for CoT reasoning, offering a lightweight yet powerful alternative to traditional PEFT methods.

Related papers

• Implicit Neural Representation-Based Continuous Single Image Super Resolution: An Empirical Study [50.15623093332659]
  Implicit neural representation (INR) has become the standard approach for arbitrary-scale image super-resolution (ASSR)<n>We compare existing techniques across diverse settings and present aggregated performance results on multiple image quality metrics.<n>We examine a new loss function that penalizes intensity variations while preserving edges, textures, and finer details during training.
  arXiv  Detail & Related papers  (2026-01-25T07:09:20Z)
• Bias-Restrained Prefix Representation Finetuning for Mathematical Reasoning [33.28699044085956]
  Representation finetuning (ReFT) methods improve efficiency by freezing model weights and optimizing internal representations with fewer parameters than PEFT.<n>ReFT exhibits a significant performance decline on mathematical reasoning tasks.<n>This paper proposes Bias-REstrained Prefix Representation FineTuning (BREP ReFT), which enhances ReFT's mathematical reasoning capability.
  arXiv  Detail & Related papers  (2025-11-13T05:15:36Z)
• Meaningless Tokens, Meaningful Gains: How Activation Shifts Enhance LLM Reasoning [53.35553353785948]
  Motivated by the puzzling observation that inserting long sequences of meaningless tokens before the query prompt can consistently enhance reasoning LLM performance, this work analyzes the underlying mechanism driving this phenomenon.<n>We find that the improvements arise from a redistribution of activations in the LLM's layers, where near zero activations become less frequent while large magnitude activations increase.<n>We propose a lightweight inference-time technique that modifies activations directly without altering the input sequence.
  arXiv  Detail & Related papers  (2025-10-01T15:39:38Z)
• FedReFT: Federated Representation Fine-Tuning with All-But-Me Aggregation [12.544628972135905]
  We introduce Federated Representation Fine-Tuning (FedReFT), a novel approach to fine-tune the client's hidden representation.<n>FedReFT applies sparse intervention layers to steer hidden representations directly, offering a lightweight and semantically rich fine-tuning alternative.<n>We evaluate FedReFT on commonsense reasoning, arithmetic reasoning, instruction-tuning, and GLUE.
  arXiv  Detail & Related papers  (2025-08-27T22:03:19Z)
• Attention, Please! Revisiting Attentive Probing Through the Lens of Efficiency [20.320991233039965]
  As fine-tuning becomes impractical at scale, probing is emerging as the preferred evaluation protocol.<n>This motivates the need for attentive probing, an alternative that uses attention to selectively aggregate patch-level features.<n>We present the first comprehensive study of existing methods, analyzing their design choices and benchmarking their performance.
  arXiv  Detail & Related papers  (2025-06-11T21:10:26Z)
• Weight Spectra Induced Efficient Model Adaptation [54.8615621415845]
  Fine-tuning large-scale foundation models incurs prohibitive computational costs.<n>We show that fine-tuning predominantly amplifies the top singular values while leaving the remainder largely intact.<n>We propose a novel method that leverages learnable rescaling of top singular directions.
  arXiv  Detail & Related papers  (2025-05-29T05:03:29Z)
• Next Token Perception Score: Analytical Assessment of your LLM Perception Skills [12.093755170926762]
  Next Token Perception Score (NTPS) is a score derived under a linear setting that measures the overlap between autoregressive and perception feature subspaces.<n>We show that NTPS increases following low-rank adaptation (LoRA) fine-tuning, especially in large models.<n>Our results offer both theoretical insights and practical tools for analytically assessing perception skills.
  arXiv  Detail & Related papers  (2025-05-22T17:18:51Z)
• Efficient Diffusion as Low Light Enhancer [63.789138528062225]
  Reflectance-Aware Trajectory Refinement (RATR) is a simple yet effective module to refine the teacher trajectory using the reflectance component of images. textbfReflectance-aware textbfDiffusion with textbfDistilled textbfTrajectory (textbfReDDiT) is an efficient and flexible distillation framework tailored for Low-Light Image Enhancement (LLIE)
  arXiv  Detail & Related papers  (2024-10-16T08:07:18Z)
• In-context Demonstration Matters: On Prompt Optimization for Pseudo-Supervision Refinement [71.60563181678323]
  Large language models (LLMs) have achieved great success across diverse tasks, and fine-tuning is sometimes needed to further enhance generation quality.<n>To handle these challenges, a direct solution is to generate high-confidence'' data from unsupervised downstream tasks.<n>We propose a novel approach, pseudo-supervised demonstrations aligned prompt optimization (PAPO) algorithm, which jointly refines both the prompt and the overall pseudo-supervision.
  arXiv  Detail & Related papers  (2024-10-04T03:39:28Z)
• See Further for Parameter Efficient Fine-tuning by Standing on the Shoulders of Decomposition [56.87609859444084]
  parameter-efficient fine-tuning (PEFT) focuses on optimizing a select subset of parameters while keeping the rest fixed, significantly lowering computational and storage overheads.<n>We take the first step to unify all approaches by dissecting them from a decomposition perspective.<n>We introduce two novel PEFT methods alongside a simple yet effective framework designed to enhance the performance of PEFT techniques across various applications.
  arXiv  Detail & Related papers  (2024-07-07T15:44:42Z)
• Efficient Model-agnostic Alignment via Bayesian Persuasion [13.42367964190663]
  We introduce a model-agnostic and lightweight Bayesian Persuasion Alignment framework. In the persuasion process, the small model (Advisor) observes the information item (i.e., state) and persuades large models (Receiver) to elicit improved responses. We show that GPT-2 can significantly improve the performance of various models, achieving an average enhancement of 16.1% in mathematical reasoning ability and 13.7% in code generation.
  arXiv  Detail & Related papers  (2024-05-29T02:57:07Z)
• LoRA Meets Dropout under a Unified Framework [38.5176197615878]
  Large language models (LLMs) have emerged as essential elements in numerous NLP applications. Various dropout methods, initially designed for full finetuning with all the parameters updated, alleviates overfitting associated with excessive parameter redundancy. We introduce a unified framework for a comprehensive investigation, which instantiates these methods based on dropping position, structural pattern and compensation measure.
  arXiv  Detail & Related papers  (2024-02-25T07:09:10Z)
• Manipulating Feature Visualizations with Gradient Slingshots [53.94925202421929]
  Feature Visualization (FV) is a widely used technique for interpreting the concepts learned by Deep Neural Networks (DNNs)<n>We introduce a novel method, Gradient Slingshots, that enables manipulation of FV without modifying the model architecture or significantly degrading its performance.
  arXiv  Detail & Related papers  (2024-01-11T18:57:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.