Cryptanalytic Extraction of Isolated Bias-Free GLU Feed-Forward Blocks by Antipodal Separation

• URL: http://arxiv.org/abs/2608.06631v1
• Date: Thu, 06 Aug 2026 22:41:40 GMT
• Title: Cryptanalytic Extraction of Isolated Bias-Free GLU Feed-Forward Blocks by Antipodal Separation
• Abstract summary: Cryptanalytic extraction has been demonstrated for ReLU networks, for networks using componentwise activations such as GELU or SiLU, and for a Transformer's final projection matrix.<n>These methods do not recover the bias-free Gated Linear Unit feed-forward blocks used in many modern language models.<n>We give a constructive, multi-stage forward-query recovery primitive for isolated bias-free GLU blocks.
• Score: 6.560534698339666
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Cryptanalytic extraction has been demonstrated for ReLU networks, for networks using componentwise activations such as GELU or SiLU, and for a Transformer's final projection matrix. These methods do not recover the bias-free Gated Linear Unit (GLU) feed-forward blocks used in many modern language models. Such a block multiplies an activated linear projection by a second learned linear projection within each hidden unit, a two-branch structure absent from the network classes and final-layer setting addressed by those methods. We give a constructive, multi-stage forward-query recovery primitive for isolated bias-free GLU blocks. Finite-difference curvature supplies gate-direction candidates, and paired observations at x and -x separate gate magnitude, orientation, and value-branch coupling. Across high-precision targets, six Qwen layers, an 8,192-unit Llama subproblem, and a full-dimensional Gemma block all reach sub-percent median validation error. Four finite-precision configurations remain below 5 percent median error, but none reproduces every stored weight. These isolated-block experiments are not an end-to-end model-API attack: deriving the required internal block responses from final model outputs remains unsolved.

Related papers

• SHIFT-LLM: Distribution Shift Correction in Depth-Pruned LLMs [18.494055578192015]
  We introduce SHIFT-LLM, a training-free post-pruning correction framework that inserts a Linear Residual Adapter at each pruning site.<n>Each LRA preserves the identity pathway of the original residual block and adds a lightweight affine residual correction.<n>Experiments on five model families, six layer-selection criteria, and seven zero-shot benchmarks show that SHIFT-LLM consistently recovers accuracy lost to depth pruning.
  arXiv  Detail & Related papers  (2026-08-25T19:01:58Z)
• TANGO: Token-Aggregated Nonlinear Gating Operators for Natural and Formal Language Modeling [0.0]
  We introduce the TANGO model (Token-Aggregated Gating Operators)<n>Each source token produces a SwiGLU vector gate.<n>We compare TANGO with Recurrent and Untied Transformer++, full-attention GAU, and FLASH.
  arXiv  Detail & Related papers  (2026-08-22T22:04:39Z)
• CoCurve: Cross-Module Co-Pruning Curvature for Training-Free Structured LLM Pruning [47.406985340436925]
  CoCurve is a calibration-only, fine-tuning-free method that prunes attention and FFN units jointly.<n>It reduces to one budgeted quadratic program, solved in a single shot under a shared attention--FFN budget, with no labels, fine-tuning, or recovery.
  arXiv  Detail & Related papers  (2026-07-20T05:27:57Z)
• Constrained Decoding for Diffusion Language Models via Efficient Inference over Finite Automata [57.27430779838529]
  We present an exact and tractable algorithm for sampling from the constrained mean-field posterior under any constraint expressible as a finite automaton.<n>The approach guarantees constraint satisfaction by construction, supports both greedy and sampling-based decoding, and is compatible with parallel and block-wise decoding.<n> Empirical evaluations on Dream-7B and LLaDA-8 show substantial accuracy gains across various tasks.
  arXiv  Detail & Related papers  (2026-07-08T05:48:57Z)
• GEMS: Geometric Constraints Enable Multi-Semantic Superposition in LLMs [5.018957704056544]
  Activation steering controls model behavior by modifying intermediate hidden states at inference time without retraining.<n>We show that when multiple semantic directions are superposed without constraints, the model collapses.<n>We propose GEMS, a training-free method that maps each source to a corresponding geometric constraint.
  arXiv  Detail & Related papers  (2026-06-18T08:43:33Z)
• Closed-Form Spectral Regularization for Multi-Task Model Merging [96.82449201305234]
  Model merging combines several independently fine-tuned experts into a single multi-task model without any training data.<n>State-of-the-art merging methods formulate merging as a layer-wise interference problem.<n>We propose SWUDI, a closed-form method that combines a soft exponential filter, which matches the gradient-flow trajectory of iterative descent.
  arXiv  Detail & Related papers  (2026-06-05T14:00:47Z)
• Pressure-Testing Deception Probes in LLMs: Scaling, Robustness, and the Geometry of Deceptive Representations [5.218766876318545]
  Linear probes trained on LLM activations are increasingly proposed as deception-detection metrics, yet report AUROC exceeding 0.96 on clean benchmarks while collapsing under distributional shift.<n>This paper systematically pressure-tests probe-based metrics across the Gemma 3 model family (1B-27B parameters)<n>We test four hypotheses about deception encoding: (1) single linear direction, (2) multi-dimensional subspace, (3) convex conic hull, (4) entropy proxy.
  arXiv  Detail & Related papers  (2026-05-27T04:51:55Z)
• Efficient Long-Context Modeling in Diffusion Language Models via Block Approximate Sparse Attention [58.484607892210015]
  Diffusion Language Models (DLMs) enable globally coherent, bidirectional, and controllable text generation.<n>Many existing block-sparse attention methods select blocks by fixed sampling patterns over the high-resolution attention space.<n>We propose the Block Approximate Sparse Attention framework (BA-Att) with block-wise pre-downsampled operation.
  arXiv  Detail & Related papers  (2026-05-19T12:01:13Z)
• DDCL: Deep Dual Competitive Learning: A Differentiable End-to-End Framework for Unsupervised Prototype-Based Representation Learning [0.0]
  A persistent structural weakness in deep clustering is the disconnect between feature learning and cluster assignment.<n>This paper introduces Deep Dual Competitive Learning, the first fully differentiable end-to-end framework for unsupervised representation learning.
  arXiv  Detail & Related papers  (2026-04-02T08:02:27Z)
• FLARE: Toward Universal Dataset Purification against Backdoor Attacks [16.97677097266535]
  Deep neural networks (DNNs) are susceptible to backdoor attacks.<n> adversaries poison datasets with adversary-specified triggers to implant hidden backdoors.<n>We propose FLARE, a universal purification method to counter various backdoor attacks.
  arXiv  Detail & Related papers  (2024-11-29T05:34:21Z)
• On the Effective Number of Linear Regions in Shallow Univariate ReLU Networks: Convergence Guarantees and Implicit Bias [50.84569563188485]
  We show that gradient flow converges in direction when labels are determined by the sign of a target network with $r$ neurons. Our result may already hold for mild over- parameterization, where the width is $tildemathcalO(r)$ and independent of the sample size.
  arXiv  Detail & Related papers  (2022-05-18T16:57:10Z)
• GDP: Stabilized Neural Network Pruning via Gates with Differentiable Polarization [84.57695474130273]
  Gate-based or importance-based pruning methods aim to remove channels whose importance is smallest. GDP can be plugged before convolutional layers without bells and whistles, to control the on-and-off of each channel. Experiments conducted over CIFAR-10 and ImageNet datasets show that the proposed GDP achieves the state-of-the-art performance.
  arXiv  Detail & Related papers  (2021-09-06T03:17:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.