LayerTracer: A Joint Task-Particle and Vulnerable-Layer Analysis framework for Arbitrary Large Language Model Architectures

• URL: http://arxiv.org/abs/2604.20556v1
• Date: Wed, 22 Apr 2026 13:38:15 GMT
• Title: LayerTracer: A Joint Task-Particle and Vulnerable-Layer Analysis framework for Arbitrary Large Language Model Architectures
• Abstract summary: This paper proposes LayerTracer, an architecture-agnostic end-to-end analysis framework compatible with any Large Language Models (LLMs) architecture.<n>By extracting hidden states layer-by-layer and mapping them to vocabulary probability distributions, it achieves joint analysis of task particle localization and layer vulnerability quantification.
• Score: 2.175079806379993
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Currently, Large Language Models (LLMs) feature a diversified architectural landscape, including traditional Transformer, GateDeltaNet, and Mamba. However, the evolutionary laws of hierarchical representations, task knowledge formation positions, and network robustness bottleneck mechanisms in various LLM architectures remain unclear, posing core challenges for hybrid architecture design and model optimization. This paper proposes LayerTracer, an architecture-agnostic end-to-end analysis framework compatible with any LLM architecture. By extracting hidden states layer-by-layer and mapping them to vocabulary probability distributions, it achieves joint analysis of task particle localization and layer vulnerability quantification. We define the task particle as the key layer where the target token probability first rises significantly, representing the model's task execution starting point, and the vulnerable layer is defined as the layer with the maximum Jensen-Shannon (JS) divergence between output distributions before and after mask perturbation, reflecting its sensitivity to disturbances. Experiments on models of different parameter scales show that task particles mainly appear in the deep layers of the model regardless of parameter size, while larger-parameter models exhibit stronger hierarchical robustness. LayerTracer provides a scientific basis for layer division, module ratio, and gating switching of hybrid architectures, effectively optimizing model performance. It accurately locates task-effective layers and stability bottlenecks, offering universal support for LLM structure design and interpretability research.

Related papers

• ABLE: Representing and Mapping LLMs via Attribution-Based Large-model Embedding [23.27523995613204]
  We propose ABLE (Attribution-Based Large-model Embedding), a framework that leverages the interpretability space to construct model representations.<n>By aggregating gradient-based feature attributions via a tokenizer-agnostic word-level alignment, ABLE captures model-specific input-sensitivity patterns.<n>Our training-free approach achieves competitive or superior performance in relation prediction, model routing, and benchmark score prediction.
  arXiv  Detail & Related papers  (2026-04-20T04:55:11Z)
• MoEMambaMIL: Structure-Aware Selective State Space Modeling for Whole-Slide Image Analysis [0.7898424058509471]
  MoEMambaMIL is a structure-aware framework for whole-slide image (WSI) analysis.<n>It integrates region-nested selective scanning with mixture-of-experts (MoE) modeling.<n>MoEMambaMIL achieves the best performance across 9 downstream tasks.
  arXiv  Detail & Related papers  (2026-03-06T15:28:07Z)
• RooflineBench: A Benchmarking Framework for On-Device LLMs via Roofline Analysis [53.90240071275054]
  The transition toward localized intelligence through Small Language Models (SLMs) has intensified the need for rigorous performance characterization on resource-constrained edge hardware.<n>We propose a systematic framework that unifies architectural primitives and hardware constraints through the lens of operational intensity (OI)<n>By defining an inference-potential region, we introduce the Relative Inference Potential as a novel metric to compare efficiency differences between Large Language Models (LLMs) on the same hardware substrate.
  arXiv  Detail & Related papers  (2026-02-12T03:02:22Z)
• Closed-Loop LLM Discovery of Non-Standard Channel Priors in Vision Models [48.83701310501069]
  Large Language Models (LLMs) offer a transformative approach to Neural Architecture Search (NAS)<n>We formulate the search as a sequence of conditional code generation tasks, where an LLM refines architectural specifications based on performance telemetry.<n>We generate a vast corpus of valid, shape-consistent architectures via Abstract Syntax Tree (AST) mutations.<n> Experimental results on CIFAR-100 validate the efficacy of this approach, demonstrating that the model yields statistically significant improvements in accuracy.
  arXiv  Detail & Related papers  (2026-01-13T13:00:30Z)
• Hierarchical Alignment: Surgical Fine-Tuning via Functional Layer Specialization in Large Language Models [4.935224714809964]
  We introduce Hierarchical Alignment, a novel method that applies targeted DPO to distinct functional blocks of a model's layers.<n>Specifically, aligning the local layers (Local-Align) enhances grammatical fluency.<n> aligning the global layers (Global-Align) improves factual consistency as hypothesized but also proves to be the most effective strategy for enhancing logical coherence.
  arXiv  Detail & Related papers  (2025-10-14T00:58:34Z)
• GCRPNet: Graph-Enhanced Contextual and Regional Perception Network for Salient Object Detection in Optical Remote Sensing Images [68.33481681452675]
  We propose a graph-enhanced contextual and regional perception network (GCRPNet)<n>It builds upon the Mamba architecture to simultaneously capture long-range dependencies and enhance regional feature representation.<n>It performs adaptive patch scanning on feature maps processed via multi-scale convolutions, thereby capturing rich local region information.
  arXiv  Detail & Related papers  (2025-08-14T11:31:43Z)
• FreSca: Scaling in Frequency Space Enhances Diffusion Models [55.75504192166779]
  This paper explores frequency-based control within latent diffusion models.<n>We introduce FreSca, a novel framework that decomposes noise difference into low- and high-frequency components.<n>FreSca operates without any model retraining or architectural change, offering model- and task-agnostic control.
  arXiv  Detail & Related papers  (2025-04-02T22:03:11Z)
• Instruction-Guided Autoregressive Neural Network Parameter Generation [49.800239140036496]
  We propose IGPG, an autoregressive framework that unifies parameter synthesis across diverse tasks and architectures.<n>By autoregressively generating neural network weights' tokens, IGPG ensures inter-layer coherence and enables efficient adaptation across models and datasets.<n>Experiments on multiple datasets demonstrate that IGPG consolidates diverse pretrained models into a single, flexible generative framework.
  arXiv  Detail & Related papers  (2025-04-02T05:50:19Z)
• Spectral Architecture Search for Neural Network Models [0.0]
  We present a novel architecture search protocol which exploits the spectral attributes of the inter-layer transfer matrices.<n>We show that the newly proposed method yields a self-emerging architecture with a minimal degree of expressivity to handle the task under investigation.
  arXiv  Detail & Related papers  (2025-04-01T15:14:30Z)
• Model Hemorrhage and the Robustness Limits of Large Language Models [119.46442117681147]
  Large language models (LLMs) demonstrate strong performance across natural language processing tasks, yet undergo significant performance degradation when modified for deployment.<n>We define this phenomenon as model hemorrhage - performance decline caused by parameter alterations and architectural changes.
  arXiv  Detail & Related papers  (2025-03-31T10:16:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.