Distributed Sparse Interventions in Language Models

• URL: http://arxiv.org/abs/2607.07128v1
• Date: Wed, 08 Jul 2026 08:19:11 GMT
• Title: Distributed Sparse Interventions in Language Models
• Abstract summary: We introduce Distributed Sparse Interventions (DSI) to study the role of model components in task behaviour.<n>DSI considers nonlinearities and interactions between neurons across layers to identify sparse sets of neurons that elicit task-relevant computations.<n>We demonstrate that DSI can activate task behaviour in instruction-tuned language models by localising and intervening on as few as 0.01% of neurons.
• Score: 5.980111878930065
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Language models perform a wide range of tasks at varying levels of abstraction with the capacity to flexibly infer tasks from context, execute multiple tasks simultaneously, and select among competing tasks. To study the role of model components in task behaviour, their causal influence can be investigated through interventions. Prior work on model steering has largely focused on interventions along global directions in activation space, modeling task representations as approximately linear and additive. By studying interventions at the neuron level, we find substantial, neuron-specific nonlinear effects on model outputs that are not captured by current steering approaches. We introduce Distributed Sparse Interventions (DSI), an intervention approach that considers nonlinearities and interactions between neurons across layers to identify sparse sets of neurons that elicit task-relevant computations. Across a range of tasks, we demonstrate that DSI can activate task behaviour in instruction-tuned language models by localising and intervening on as few as 0.01% of neurons, highlighting the effectiveness of sparse, distributed interventions in the neuron basis. Additionally, adopting a set-based perspective enables computations over the identified neuron sets, offering insights into the roles of individual neurons by analysing their effects across tasks. Through sparse interventions, DSI enables fine-grained control over model behaviour, localisation of task-relevant neuron sets, and furthers our understanding of task composition.

Related papers

• Tiny Brains, Giant Impact: Uncovering the Keystone Neurons of LLM with Just a Few Prompts [57.98275914877019]
  We show that across a wide range of open-weight Transformers, a subset of neurons remains consistently highly activated during inference across tasks of multiple capability dimensions.<n>By probing along the cross-task activation strength, an extremely sparse subset is isolated, whose removal causes a collapse in model behavior.<n>Our analysis reveals that keystone neurons are a stable and intrinsic neuron subset of the model that is largely established during pretraining.
  arXiv  Detail & Related papers  (2026-05-24T03:31:07Z)
• From Heads to Neurons: Causal Attribution and Steering in Multi-Task Vision-Language Models [10.052877942432783]
  HONES is a gradient-free framework for task-aware neuron attribution and steering in vision-language models.<n>We show that HONES outperforms existing methods in identifying task-critical neurons and improves model performance after steering.
  arXiv  Detail & Related papers  (2026-04-20T08:21:06Z)
• Neuronal Group Communication for Efficient Neural representation [85.36421257648294]
  This paper addresses the question of how to build large neural systems that learn efficient, modular, and interpretable representations.<n>We propose Neuronal Group Communication (NGC), a theory-driven framework that reimagines a neural network as a dynamical system of interacting neuronal groups.<n>NGC treats weights as transient interactions between embedding-like neuronal states, with neural computation unfolding through iterative communication among groups of neurons.
  arXiv  Detail & Related papers  (2025-10-19T14:23:35Z)
• To See a World in a Spark of Neuron: Disentangling Multi-task Interference for Training-free Model Merging [16.81093103067372]
  We present the first study that relies on neuronal mechanisms for model merging.<n>We introduce NeuroMerging, a novel merging framework developed to mitigate task interference within neuronal subspaces.<n>Our findings highlighted the importance of aligning neuronal mechanisms in model merging.
  arXiv  Detail & Related papers  (2025-03-07T11:00:24Z)
• Neural Models of Task Adaptation: A Tutorial on Spiking Networks for Executive Control [0.0]
  This tutorial presents a step-by-step approach to constructing a spiking neural network (SNN) that simulates task-switching dynamics.<n>The model incorporates biologically realistic features, including lateral inhibition, adaptive synaptic weights, and precise parameterization within physiologically relevant ranges.<n>By following this tutorial, researchers can develop and extend biologically inspired SNN models for studying cognitive processes and neural adaptation.
  arXiv  Detail & Related papers  (2025-03-05T00:44:34Z)
• Towards Understanding Multi-Task Learning (Generalization) of LLMs via Detecting and Exploring Task-Specific Neurons [45.04661608619081]
  We detect task-sensitive neurons in large language models (LLMs) via gradient attribution on task-specific data.<n>We find that the overlap of task-specific neurons is strongly associated with generalization and specialization across tasks.<n>We propose a neuron-level continuous fine-tuning method that only fine-tunes the current task-specific neurons during continuous learning.
  arXiv  Detail & Related papers  (2024-07-09T01:27:35Z)
• STNDT: Modeling Neural Population Activity with a Spatiotemporal Transformer [19.329190789275565]
  We introduce SpatioTemporal Neural Data Transformer (STNDT), an NDT-based architecture that explicitly models responses of individual neurons. We show that our model achieves state-of-the-art performance on ensemble level in estimating neural activities across four neural datasets.
  arXiv  Detail & Related papers  (2022-06-09T18:54:23Z)
• Overcoming the Domain Gap in Contrastive Learning of Neural Action Representations [60.47807856873544]
  A fundamental goal in neuroscience is to understand the relationship between neural activity and behavior. We generated a new multimodal dataset consisting of the spontaneous behaviors generated by fruit flies. This dataset and our new set of augmentations promise to accelerate the application of self-supervised learning methods in neuroscience.
  arXiv  Detail & Related papers  (2021-11-29T15:27:51Z)
• Multi-Task Neural Processes [105.22406384964144]
  We develop multi-task neural processes, a new variant of neural processes for multi-task learning. In particular, we propose to explore transferable knowledge from related tasks in the function space to provide inductive bias for improving each individual task. Results demonstrate the effectiveness of multi-task neural processes in transferring useful knowledge among tasks for multi-task learning.
  arXiv  Detail & Related papers  (2021-11-10T17:27:46Z)
• Learning Neural Causal Models with Active Interventions [83.44636110899742]
  We introduce an active intervention-targeting mechanism which enables a quick identification of the underlying causal structure of the data-generating process. Our method significantly reduces the required number of interactions compared with random intervention targeting. We demonstrate superior performance on multiple benchmarks from simulated to real-world data.
  arXiv  Detail & Related papers  (2021-09-06T13:10:37Z)
• And/or trade-off in artificial neurons: impact on adversarial robustness [91.3755431537592]
  Presence of sufficient number of OR-like neurons in a network can lead to classification brittleness and increased vulnerability to adversarial attacks. We define AND-like neurons and propose measures to increase their proportion in the network. Experimental results on the MNIST dataset suggest that our approach holds promise as a direction for further exploration.
  arXiv  Detail & Related papers  (2021-02-15T08:19:05Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.