Neuron-Guided Fine-Tuning: Unlocking Efficient Alignment Mechanisms for Large Language Models

• URL: http://arxiv.org/abs/2609.05913v1
• Date: Sat, 05 Sep 2026 06:13:24 GMT
• Title: Neuron-Guided Fine-Tuning: Unlocking Efficient Alignment Mechanisms for Large Language Models
• Abstract summary: Existing Supervised Fine-Tuning paradigms are often plagued by parameter redundancy, inconsistent data quality, and catastrophic forgetting.<n>We propose Neuron-Guided Fine-Tuning (NGFT), a holistic framework that leverages neuron activation patterns as a universal proxy to unify the fine-tuning lifecycle.
• Score: 73.88918843965155
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Existing Supervised Fine-Tuning paradigms, particularly Full Parameter Fine-Tuning are often plagued by parameter redundancy, inconsistent data quality, and catastrophic forgetting, which current methods typically address in isolation and lack a unified optimization signal to bridge data selection, parameter updates, and knowledge preservation. To address this, we propose Neuron-Guided Fine-Tuning (NGFT), a holistic framework that leverages neuron activation patterns as a universal proxy to unify the fine-tuning lifecycle. NGFT operates via three synergistic mechanisms: (1) Adaptive Task-Specific Neuron Selection, which identifies essential neurons in a single forward pass to concentrate updates and reduce redundancy; (2) Activation-Based Data Selection, which prioritizes information-dense samples that maximize contribution to key neurons; and (3) Neuron Activation Alignment, a novel loss function that anchors activations to pre-trained states, deepening representation learning and preserving general knowledge. Experimental results across three models across both domain-specific and general benchmarks demonstrate that NGFT significantly outperforms existing mainstream fine-tuning methods in both efficiency and performance, while effectively mitigating catastrophic forgetting.

Related papers

• General Self-Prediction Enhancement for Spiking Neurons [71.01912385372577]
  Spiking Neural Networks (SNNs) are highly energy-efficient due to event-driven, sparse computation, but their training is challenged by spike non-differentiability and trade-offs among performance, efficiency, and biological plausibility.<n>We propose a self-prediction enhanced spiking neuron method that generates an internal prediction current from its input-output history to modulate membrane potential.<n>This design offers dual advantages, it creates a continuous gradient path that alleviates vanishing gradients and boosts training stability and accuracy, while also aligning with biological principles, which resembles distal dendritic modulation and error-driven synaptic plasticity.
  arXiv  Detail & Related papers  (2026-01-29T15:08:48Z)
• NeuroAda: Activating Each Neuron's Potential for Parameter-Efficient Fine-Tuning [18.1179807699825]
  NeuroAda is a novel PEFT method that enables fine-grained model finetuning while maintaining high memory efficiency.<n>We show that NeuroAda achieves state-of-the-art performance with little as $leq0.02%$ trainable parameters while reducing memory usage by up to 60%.
  arXiv  Detail & Related papers  (2025-10-21T17:59:24Z)
• Detecting and Pruning Prominent but Detrimental Neurons in Large Language Models [68.57424628540907]
  Large language models (LLMs) often develop learned mechanisms specialized to specific datasets.<n>We introduce a fine-tuning approach designed to enhance generalization by identifying and pruning neurons associated with dataset-specific mechanisms.<n>Our method employs Integrated Gradients to quantify each neuron's influence on high-confidence predictions, pinpointing those that disproportionately contribute to dataset-specific performance.
  arXiv  Detail & Related papers  (2025-07-12T08:10:10Z)
• TMN: A Lightweight Neuron Model for Efficient Nonlinear Spike Representation [7.524721345903027]
  Spike trains serve as the primary medium for information transmission in Spiking Neural Networks.<n>Existing encoding schemes based on spike counts or timing often face severe limitations under low-timestep constraints.<n>We propose the Ternary Momentum Neuron (TMN), a novel neuron model featuring two key innovations.
  arXiv  Detail & Related papers  (2024-08-30T12:39:25Z)
• Let's Focus on Neuron: Neuron-Level Supervised Fine-tuning for Large Language Model [43.107778640669544]
  Large Language Models (LLMs) are composed of neurons that exhibit various behaviors and roles. Recent studies have revealed that not all neurons are active across different datasets. We introduce Neuron-Level Fine-Tuning (NeFT), a novel approach that refines the granularity of parameter training down to the individual neuron.
  arXiv  Detail & Related papers  (2024-03-18T09:55:01Z)
• Predicting Infant Brain Connectivity with Federated Multi-Trajectory GNNs using Scarce Data [54.55126643084341]
  Existing deep learning solutions suffer from three major limitations. We introduce FedGmTE-Net++, a federated graph-based multi-trajectory evolution network. Using the power of federation, we aggregate local learnings among diverse hospitals with limited datasets.
  arXiv  Detail & Related papers  (2024-01-01T10:20:01Z)
• Accelerating Scalable Graph Neural Network Inference with Node-Adaptive Propagation [80.227864832092]
  Graph neural networks (GNNs) have exhibited exceptional efficacy in a diverse array of applications. The sheer size of large-scale graphs presents a significant challenge to real-time inference with GNNs. We propose an online propagation framework and two novel node-adaptive propagation methods.
  arXiv  Detail & Related papers  (2023-10-17T05:03:00Z)
• Benign Overfitting in Deep Neural Networks under Lazy Training [72.28294823115502]
  We show that when the data distribution is well-separated, DNNs can achieve Bayes-optimal test error for classification. Our results indicate that interpolating with smoother functions leads to better generalization.
  arXiv  Detail & Related papers  (2023-05-30T19:37:44Z)
• Bayesian optimization for sparse neural networks with trainable activation functions [0.0]
  We propose a trainable activation function whose parameters need to be estimated. A fully Bayesian model is developed to automatically estimate from the learning data both the model weights and activation function parameters.
  arXiv  Detail & Related papers  (2023-04-10T08:44:44Z)
• Multi-Objective Optimisation of Cortical Spiking Neural Networks With Genetic Algorithms [0.7995360025953929]
  Spiking neural networks (SNNs) communicate through the all-or-none spiking activity of neurons. Previous work using genetic algorithm (GA) optimisation on an efficient SNN model was limited to a single parameter and objective. This work applied a version of GA, called non-dominated sorting GA (NSGA-III), to demonstrate the feasibility of performing multi-objective optimisation on the same SNN.
  arXiv  Detail & Related papers  (2021-05-14T13:35:39Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.