The Quadrilateral Loss: Additivity as a Measurable Behavior of Dense Neural Networks

• URL: http://arxiv.org/abs/2607.20201v1
• Date: Wed, 22 Jul 2026 14:23:37 GMT
• Title: The Quadrilateral Loss: Additivity as a Measurable Behavior of Dense Neural Networks
• Abstract summary: We introduce the quadrilateral loss, a differentiable penalty that treats additivity as a measurable behavior instead.<n>We compare routes to exact additivity, spanning structural masks, behavioral penalties, weight decay, backfitting, the shared-section model, and bagged boosted stumps.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Additive models buy interpretability by forbidding feature interactions, a constraint that neural instantiations enforce architecturally. We introduce the quadrilateral loss, a differentiable penalty that treats additivity as a measurable behavior instead: a second-order mixed difference on pairs of training points swapping one coordinate, which vanishes if and only if the coordinate carries no interaction, remains informative for piecewise-linear networks, and equals in expectation the per-coordinate interaction mass of the interventional Shapley-GAM. The loss turns additivity into a dial - most learned interactions prove removable almost for free, and on small datasets a moderate penalty improves accuracy and additivity simultaneously - and into an online observable: its per-feature surrender curves show, across seeds and datasets, that pre-regularization interaction magnitude barely predicts what a regularized model retains, undermining post-hoc interaction rankings. Against this instrument we compare routes to exact additivity, spanning structural masks, behavioral penalties (optionally crystallized into exact structure), weight decay, backfitting, the shared-section model, and bagged boosted stumps: constraining behavior before structure dominates weight-space constraints, rankings reverse between data regimes, and converging routes agree on the shape functions themselves. Three silent failure modes we document share one anatomy: guarantees imported into settings that quietly void their preconditions.

Related papers

• Feature Priming in Online Linear Regression: Sparse-Regret Lower Bounds and a Tight Univariate Rate [47.36630149538994]
  In high-dimensional online prediction, the best predictor may depend on only a few features, so regret should scale with sparsity rather than the ambient dimension.<n>Warmuth and Amid asked at COLT 2023 whether any of three such rules admits a competitive online regret guarantee.<n>Our analysis identifies a common obstruction: cheap nuisance causes the refit to underweight the truly predictive coordinate.
  arXiv  Detail & Related papers  (2026-08-18T09:32:03Z)
• Drift and Dependence: Layer-wise Information-Theoretic Bounds for Replay-Based Continual Learning [41.371439825328544]
  Continual learning must absorb new tasks without erasing old ones, and replay---mixing a small buffer of past examples into current training---is among the most effective remedies for catastrophic forgetting.<n>We develop a layer-wise information-theoretic framework that separates these effects at every depth.<n>Controlled and benchmark experiments confirm the predicted memory scaling, the interior funnel, and the alignment signal's link to forgetting.
  arXiv  Detail & Related papers  (2026-08-12T06:01:49Z)
• Beyond Decision Boundaries: Relational Geometry Attacks on Contrastive Embedding Manifolds [16.306226464987592]
  We introduce a geometry-aware adversarial attack framework that reformulates attacks on contrastive systems as manifold-level relational corruption.<n>The proposed framework systematically distorts similarity organization within the embedding manifold by pushing positive pairs apart.<n>We show that the proposed framework reduces accuracy from 95.4% to 38.6% while completely reversing the positive-negative similarity structure.
  arXiv  Detail & Related papers  (2026-08-10T21:16:03Z)
• CDS: Counterfactual Directionality Score for Structured Interventions in Spatial Graphs [0.0]
  We introduce a framework for structured counterfactual interventions in graph-based models to estimate directional influence between node types.<n>Our approach trains a Neighbor Influence Model (NIM) to predict node states from local neighborhoods.<n>We define the Counterfactual Directionality Score (CDS), which measures the change in predicted node state induced by targeted perturbations.
  arXiv  Detail & Related papers  (2026-07-15T07:02:33Z)
• The Representational Limit of Scalar Interactions: An Interventional Decomposition [9.850981754860754]
  We prove that signed pairwise interaction scores fundamentally conflate uniqueness (U), redundancy (R), and synergy (S)<n>We introduce Hi-Fi, a post-hoc, retraining-free predictability decomposition that estimates per-feature U/R/S profiles by interventional masked inference.
  arXiv  Detail & Related papers  (2026-06-17T15:21:28Z)
• Repurposing Adversarial Perturbations for Continual Learning: From Defense to Active Alignment [20.37950128370149]
  AdvCL combines three plug-in modules: Intra-Smooth promotes local smoothness via small adversarial perturbations; Proto-Clip uses similarity clipping to prevent excessive alignment to current task prototype; and Inter-Align applies directional alignment toward previous task prototype to reduce representational gaps.<n>Experiments show consistent gains in both standard performance and robustness, with lower forgetting and stronger transfer.
  arXiv  Detail & Related papers  (2026-06-01T14:35:59Z)
• On the Structural Non-Preservation of Epistemic Behaviour under Policy Transformation [51.56484100374058]
  We formalise such information-conditioned interaction patterns as behavioural dependency.<n>This induces a probe-relative notion of $$-behavioural equivalence and a within-policy behavioural distance.<n>Results identify structural conditions under which probe-conditioned behavioural separation is not preserved under common policy transformations.
  arXiv  Detail & Related papers  (2026-02-24T22:55:21Z)
• Generalizing GNNs with Tokenized Mixture of Experts [75.8310720413187]
  We show that improving stability requires reducing reliance on shift-sensitive features, leaving an irreducible worst-case generalization floor.<n>We propose STEM-GNN, a pretrain-then-finetune framework with a mixture-of-experts encoder for diverse computation paths.<n>Across nine node, link, and graph benchmarks, STEM-GNN achieves a stronger three-way balance, improving robustness to degree/homophily shifts and to feature/edge corruptions while remaining competitive on clean graphs.
  arXiv  Detail & Related papers  (2026-02-09T22:48:30Z)
• Geometric-Disentangelment Unlearning [106.99160454669902]
  gradient ascent on forget samples often harms retained knowledge.<n>We propose the Geometric-disment Unlearning (GU) that decomposes any candidate forget gradient update into tangential and normal components to retain space and executes only the normal component.<n>Our method is plug-and-play and can be attached to existing gradient-based unlearning procedures to mitigate side effects.
  arXiv  Detail & Related papers  (2025-11-21T09:58:25Z)
• The Butterfly Effect: Neural Network Training Trajectories Are Highly Sensitive to Initial Conditions [51.68215326304272]
  We show that even small perturbations reliably cause otherwise identical training trajectories to diverge-an effect that diminishes rapidly over training time.<n>Our findings provide insights into neural network training stability, with practical implications for fine-tuning, model merging, and diversity of model ensembles.
  arXiv  Detail & Related papers  (2025-06-16T08:35:16Z)
• Towards Continual Learning Desiderata via HSIC-Bottleneck Orthogonalization and Equiangular Embedding [55.107555305760954]
  We propose a conceptually simple yet effective method that attributes forgetting to layer-wise parameter overwriting and the resulting decision boundary distortion. Our method achieves competitive accuracy performance, even with absolute superiority of zero exemplar buffer and 1.02x the base model.
  arXiv  Detail & Related papers  (2024-01-17T09:01:29Z)
• Improving Neural Additive Models with Bayesian Principles [54.29602161803093]
  Neural additive models (NAMs) enhance the transparency of deep neural networks by handling calibrated input features in separate additive sub-networks. We develop Laplace-approximated NAMs (LA-NAMs) which show improved empirical performance on datasets and challenging real-world medical tasks.
  arXiv  Detail & Related papers  (2023-05-26T13:19:15Z)
• GAMI-Net: An Explainable Neural Network based on Generalized Additive Models with Structured Interactions [5.8010446129208155]
  An explainable neural network based on generalized additive models with structured interactions (GAMI-Net) is proposed to pursue a good balance between prediction accuracy and model interpretability. GAMI-Net is a disentangled feedforward network with multiple additiveworks. Numerical experiments on both synthetic functions and real-world datasets show that the proposed model enjoys superior interpretability.
  arXiv  Detail & Related papers  (2020-03-16T11:51:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.