Jumping Ahead: Improving Reconstruction Fidelity with JumpReLU Sparse Autoencoders

• URL: http://arxiv.org/abs/2407.14435v3
• Date: Thu, 1 Aug 2024 17:42:04 GMT
• Title: Jumping Ahead: Improving Reconstruction Fidelity with JumpReLU Sparse Autoencoders
• Authors: Senthooran Rajamanoharan, Tom Lieberum, Nicolas Sonnerat, Arthur Conmy, Vikrant Varma, János Kramár, Neel Nanda,
• Abstract summary: We introduce JumpReLU SAEs, which achieve state-of-the-art reconstruction fidelity at a given sparsity level on Gemma 2 9B activations. We show that this improvement does not come at the cost of interpretability through manual and automated interpretability studies.
• Score: 4.4110204540437365
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Sparse autoencoders (SAEs) are a promising unsupervised approach for identifying causally relevant and interpretable linear features in a language model's (LM) activations. To be useful for downstream tasks, SAEs need to decompose LM activations faithfully; yet to be interpretable the decomposition must be sparse -- two objectives that are in tension. In this paper, we introduce JumpReLU SAEs, which achieve state-of-the-art reconstruction fidelity at a given sparsity level on Gemma 2 9B activations, compared to other recent advances such as Gated and TopK SAEs. We also show that this improvement does not come at the cost of interpretability through manual and automated interpretability studies. JumpReLU SAEs are a simple modification of vanilla (ReLU) SAEs -- where we replace the ReLU with a discontinuous JumpReLU activation function -- and are similarly efficient to train and run. By utilising straight-through-estimators (STEs) in a principled manner, we show how it is possible to train JumpReLU SAEs effectively despite the discontinuous JumpReLU function introduced in the SAE's forward pass. Similarly, we use STEs to directly train L0 to be sparse, instead of training on proxies such as L1, avoiding problems like shrinkage.

Related papers

• SALSA: Speedy ASR-LLM Synchronous Aggregation [40.91241351045586]
  We propose SALSA, which couples the decoder layers of the ASR to the LLM decoder, while synchronously advancing both decoders. We evaluate SALSA on 8 low-resource languages in the FLEURS benchmark, yielding substantial WER reductions of up to 38%.
  arXiv  Detail & Related papers  (2024-08-29T14:00:57Z)
• Model Surgery: Modulating LLM's Behavior Via Simple Parameter Editing [63.20133320524577]
  Large Language Models (LLMs) have demonstrated great potential as generalist assistants. It is crucial that these models exhibit desirable behavioral traits, such as non-toxicity and resilience against jailbreak attempts. In this paper, we observe that directly editing a small subset of parameters can effectively modulate specific behaviors of LLMs.
  arXiv  Detail & Related papers  (2024-07-11T17:52:03Z)
• Improving Dictionary Learning with Gated Sparse Autoencoders [8.3037652157611]
  Gated Sparse Autoencoder (Gated SAE) is a technique for unsupervised discovery of interpretable features in language models' (LMs) activations. In SAEs, the L1 penalty used to encourage sparsity introduces many undesirable biases, such as shrinkage. In training SAEs on LMs of up to 7B parameters, Gated SAEs solve shrinkage, and require half as many firing features to achieve comparable reconstruction fidelity.
  arXiv  Detail & Related papers  (2024-04-24T17:47:22Z)
• ResLoRA: Identity Residual Mapping in Low-Rank Adaption [96.59370314485074]
  We propose ResLoRA, an improved framework of low-rank adaptation (LoRA) Our method can achieve better results in fewer training steps without any extra trainable parameters or inference cost compared to LoRA. The experiments on NLG, NLU, and text-to-image tasks demonstrate the effectiveness of our method.
  arXiv  Detail & Related papers  (2024-02-28T04:33:20Z)
• Stable Distillation: Regularizing Continued Pre-training for Low-Resource Automatic Speech Recognition [54.9235160379917]
  Stable Distillation is a simple and novel approach for SSL-based continued pre-training. It boosts ASR performance in the target domain where both labeled and unlabeled data are limited.
  arXiv  Detail & Related papers  (2023-12-20T06:02:12Z)
• ML-LMCL: Mutual Learning and Large-Margin Contrastive Learning for Improving ASR Robustness in Spoken Language Understanding [55.39105863825107]
  We propose Mutual Learning and Large-Margin Contrastive Learning (ML-LMCL) to improve automatic speech recognition (ASR) robustness. In fine-tuning, we apply mutual learning and train two SLU models on the manual transcripts and the ASR transcripts, respectively. Experiments on three datasets show that ML-LMCL outperforms existing models and achieves new state-of-the-art performance.
  arXiv  Detail & Related papers  (2023-11-19T16:53:35Z)
• Re-Reading Improves Reasoning in Large Language Models [87.46256176508376]
  We introduce a simple, yet general and effective prompting method, Re2, to enhance the reasoning capabilities of off-the-shelf Large Language Models (LLMs) Unlike most thought-eliciting prompting methods, such as Chain-of-Thought (CoT), Re2 shifts the focus to the input by processing questions twice, thereby enhancing the understanding process. We evaluate Re2 on extensive reasoning benchmarks across 14 datasets, spanning 112 experiments, to validate its effectiveness and generality.
  arXiv  Detail & Related papers  (2023-09-12T14:36:23Z)
• Language Reward Modulation for Pretraining Reinforcement Learning [61.76572261146311]
  We propose leveraging the capabilities of LRFs as a pretraining signal for reinforcement learning. Our VLM pretraining approach, which is a departure from previous attempts to use LRFs, can warmstart sample-efficient learning on robot manipulation tasks.
  arXiv  Detail & Related papers  (2023-08-23T17:37:51Z)
• Does Zero-Shot Reinforcement Learning Exist? [11.741744003560095]
  A zero-shot RL agent is an agent that can solve any RL task instantly with no additional planning or learning. This marks a shift from the reward-centric RL paradigm towards "controllable" agents. Strategies for approximate zero-shot RL ave been suggested using successor features (SFs) or forward-backward (FB) representations.
  arXiv  Detail & Related papers  (2022-09-29T16:54:05Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.