SPEAR: Structure Property Explainability with Attention Regularization

• URL: http://arxiv.org/abs/2608.13826v1
• Date: Thu, 13 Aug 2026 23:24:43 GMT
• Title: SPEAR: Structure Property Explainability with Attention Regularization
• Abstract summary: We introduce SPEAR (Structure Property Explainability with Attention Regularization), a framework that constrains attention distributions during training to improve their stability, selectivity, and physical interpretability.<n> SPEAR augments attention based regression with a learnable temperature that controls attention concentration and a smoothness penalty that enforces coherence across neighboring spectral positions.<n>We show that attention regularization produces smooth, contiguous attribution profiles aligned with causal features while preserving predictive accuracy.
• Score: 0.6055934915901143
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Machine learning is increasingly used to learn structure property relationships from spectroscopic and diffraction data, yet its adoption in materials discovery is often limited by poor interpretability of model predictions. Although attention mechanisms are frequently treated as inherently explainable, unregularized attention can yield unstable, fragmented, or intensity driven attribution patterns that obscure the physical origin of these relationships. Here we introduce SPEAR (Structure Property Explainability with Attention Regularization), a framework that constrains attention distributions during training to improve their stability, selectivity, and physical interpretability. SPEAR augments attention based regression with a learnable temperature that controls attention concentration and a smoothness penalty that enforces coherence across neighboring spectral positions, treating attention as a learnable explanatory object rather than a post hoc visualization. Using synthetic spectral benchmarks with known generative structure, we show that attention regularization produces smooth, contiguous attribution profiles aligned with causal features while preserving predictive accuracy. Applied to experimental X ray diffraction data from a combinatorial rare earth zirconate thin film library, the regularized model selectively emphasizes physically relevant diffraction features and decouples feature importance from raw peak intensity. The reflection it identified prompted a reassessment of our earlier structural analysis, revealing a correlation between the 220 peak position, the tetragonal distortion that accommodates cation size disorder, and the local thermal conductivity. Attention regularization therefore provides a principled training constraint for explainable structure property regression, yielding mechanistically meaningful explanations without sacrificing predictive performance.

Related papers

• NVExplain: Explaining Time Series Forecasting with Latent Trajectory Analysis and Structure-Preserving Surrogates [2.0320459617688793]
  Time series forecasting models are widely used in high-stakes settings.<n> existing post-hoc methods often ignore temporal dependence and fail to provide horizon-specific explanations.<n>We propose a model-agnostic explainability framework that explains forecasting predictions by attributing each forecast horizon to temporally relevant historical lags.
  arXiv  Detail & Related papers  (2026-08-25T19:14:12Z)
• PhysAttNet: Enhancing Predictive Performance in Industrial and Astrophysical Time Series via Physics-Informed Attention [2.4965742139895313]
  PhysAttNet is a physics-informed attention framework for time series forecasting.<n>It augments a lightweight CNN forecaster with an attention head guided by domain-informed regularization.<n>It improves forecasting accuracy, generalization, and prediction performance on structurally important events.
  arXiv  Detail & Related papers  (2026-08-07T18:09:31Z)
• Improving Adversarial Robustness of Attribution via Implicit Regularization [9.78455276849371]
  adversarial robustness of attributions is a fundamental requirement for reliable explainability in deep learning.<n>We show that robustness can arise implicitly from the learning dynamics of standard gradient descent.<n>Our results highlight learning dynamics as a principled and practical mechanism for robust explainability.
  arXiv  Detail & Related papers  (2026-05-28T14:19:39Z)
• K-Models: a Flexible and Interpretable Method for Ordinal Clustering with Application to Antigen-Antibody Interaction Profiles [65.21921642886367]
  K-Models is a novel framework that integrates ordinal constraints and estimates key underlying elements of the random process generating the observed functional profiles.<n>The proposed method is evaluated through simulations and real-world applications.
  arXiv  Detail & Related papers  (2026-05-14T13:35:44Z)
• SLASH the Sink: Sharpening Structural Attention Inside LLMs [55.75072688988806]
  Large Language Models (LLMs) show remarkable semantic understanding but struggle with structural understanding when processing graph topologies in a serialized format.<n>We propose StructuraL Attention SHarpening (SLASH), which amplifies this internal structural understanding via a plug-and-play attention redistribution.
  arXiv  Detail & Related papers  (2026-05-11T12:59:07Z)
• Explicit Dropout: Deterministic Regularization for Transformer Architectures [55.09895958546215]
  Dropout is a widely used regularization technique in deep learning, but its effects are typically realized through masking.<n>We propose a deterministic formulation that expresses dropout as an additive regularizer directly incorporated into the training loss.
  arXiv  Detail & Related papers  (2026-04-22T12:45:51Z)
• Causal-INSIGHT: Probing Temporal Models to Extract Causal Structure [4.718641165826564]
  Causal-INSIGHT is a model-agnostic, post-hoc interpretation framework for extracting model-implied (predictor-dependent) directed, time-lagged influence structure from trained temporal predictors.<n>We introduce Qbic, a sparsity-aware graph selection criterion that balances predictive fidelity and structural complexity without requiring ground-truth graph labels.
  arXiv  Detail & Related papers  (2026-03-26T14:16:37Z)
• Geometric Observability Index: An Operator-Theoretic Framework for Per-Feature Sensitivity, Weak Observability, and Dynamic Effects in SE(3) Pose Estimation [0.552480439325792]
  We present a unified operator-theoretic framework for analyzing per-feature sensitivity in camera pose estimation on the Lie group SE(3).<n>We extend influence function theory to matrix Lie groups and derive an intrinsic perturbation operator for left-trivialized M-estimators on SE(3).<n>The resulting Geometric Observability Index (GOI) quantifies the contribution of a single measurement through the curvature operator and the Lie algebraic structure of the observable subspace.
  arXiv  Detail & Related papers  (2026-02-05T12:12:00Z)
• Defining Energy Indicators for Impact Identification on Aerospace Composites: A Physics-Informed Machine Learning Perspective [0.4104921880358479]
  Energy estimation is critical to impact identification on aerospace composites.<n>This study introduces a physics-informed framework that embeds domain knowledge into machine learning.
  arXiv  Detail & Related papers  (2025-11-03T13:58:03Z)
• Transformers Learn Faster with Semantic Focus [57.97235825738412]
  We study sparse transformers in terms of learnability and generalization.<n>We find that input-dependent sparse attention models appear to converge faster and generalize better than standard attention models.
  arXiv  Detail & Related papers  (2025-06-17T01:19:28Z)
• Nonparametric Partial Disentanglement via Mechanism Sparsity: Sparse Actions, Interventions and Sparse Temporal Dependencies [58.179981892921056]
  This work introduces a novel principle for disentanglement we call mechanism sparsity regularization. We propose a representation learning method that induces disentanglement by simultaneously learning the latent factors. We show that the latent factors can be recovered by regularizing the learned causal graph to be sparse.
  arXiv  Detail & Related papers  (2024-01-10T02:38:21Z)
• Discovering Latent Causal Variables via Mechanism Sparsity: A New Principle for Nonlinear ICA [81.4991350761909]
  Independent component analysis (ICA) refers to an ensemble of methods which formalize this goal and provide estimation procedure for practical application. We show that the latent variables can be recovered up to a permutation if one regularizes the latent mechanisms to be sparse.
  arXiv  Detail & Related papers  (2021-07-21T14:22:14Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.