Region Seeding via Pre-Activation Regularization: A Geometric View of Piecewise Affine Neural Networks

• URL: http://arxiv.org/abs/2605.06300v2
• Date: Sat, 09 May 2026 16:04:30 GMT
• Title: Region Seeding via Pre-Activation Regularization: A Geometric View of Piecewise Affine Neural Networks
• Abstract summary: In practice, standard training realizes far fewer region refinements in data-visited neighborhoods than the architecture could in principle support.<n>Our theory provides a sufficient condition under which bringing neuron switching surfaces sufficiently close to data points ensures their intersection with local neighborhoods.<n>We propose a plug-and-play region-seeding regularizer that encourages early partitioning while allowing task-driven refinement to dominate later in training.
• Score: 12.476368894025233
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Deep networks with continuous piecewise affine activations induce polyhedral partitions of the input space, making the number of realized affine regions a natural measure of expressive capacity and a key determinant of how well the model can approximate nonlinear target functions. In practice, standard training realizes far fewer region refinements in data-visited neighborhoods than the architecture could in principle support, while existing region-count theory is primarily architectural and offers little guidance on how optimization shapes the realized partition near the data. Our theory provides a sufficient condition under which bringing neuron switching surfaces sufficiently close to data points ensures their intersection with local neighborhoods, which in turn implies a strict increase in the local affine-region count, yielding a principled training-time handle for seeding data-relevant partitions early in optimization. Guided by these results, we propose a plug-and-play region-seeding regularizer that encourages early partitioning while allowing task-driven refinement to dominate later in training. Experiments show that the regularizer increases the number of realized affine regions via exact enumeration and improves overall performance on toy datasets, while also improving early-stage accuracy and achieving comparable (or slightly improved) final accuracy on ImageNet-1k for classical models.

Related papers

• Curvature-Aware Radius Shrinkage for Adaptive Nearest Neighbor Classification [52.452902154360565]
  CARSANN is a geometry-driven framework that adapts the spatial support of each neighborhood according to local geometric complexity.<n>Experiments on more than 70 real-world OpenML datasets show that CARSANN consistently improves upon standard $k$-NN.
  arXiv  Detail & Related papers  (2026-08-27T19:19:07Z)
• Expressivity Saturation: Reduced Affine Region Usage Under Increasing Task Complexity [5.366707531676941]
  Piecewise-affine neural networks implement continuous piecewise-affine maps.<n>We study the gap between theoretical region capacity and the affine regions realized after training.
  arXiv  Detail & Related papers  (2026-06-19T18:55:46Z)
• Complexity of Linear Regions in Self-supervised Deep ReLU Networks [0.0]
  This study investigates the local distribution of linear regions produced by Self-Supervised Learning (SSL) models.<n>We use SplineCam to extract two-dimensional polytopes near the data distribution.<n>The study compares supervised, contrastive, and self-distillation methods over two standard benchmark datasets.
  arXiv  Detail & Related papers  (2026-04-27T12:21:25Z)
• Learning to Guide Local Search for MPE Inference in Probabilistic Graphical Models [7.287294240824019]
  Most Probable Explanation (MPE) inference in Probabilistic Graphical Models (PGMs) is a fundamental yet computationally challenging problem.<n>We propose a neural amortization framework for improving local search in repeated-Query regime.<n>We provide theoretical intuition linking distance-reducing move selection to improved promise during neighbor selection.
  arXiv  Detail & Related papers  (2026-02-01T22:43:28Z)
• What Makes Local Updates Effective: The Role of Data Heterogeneity and Smoothness [5.357435119431715]
  The thesis contributes to a self-contained guide for analyzing Local SGD in heterogeneous environments.<n>The thesis also extends to online learning, providing fundamental bounds under both first-order and bandit feedback.
  arXiv  Detail & Related papers  (2025-06-30T19:06:02Z)
• Unbiased Region-Language Alignment for Open-Vocabulary Dense Prediction [80.67150791183126]
  Pre-trained vision-language models (VLMs) have demonstrated impressive zero-shot recognition capability, but still underperform in dense prediction tasks.<n>We propose DenseVLM, a framework designed to learn unbiased region-language alignment from powerful pre-trained VLM representations.<n>We show that DenseVLM can directly replace the original VLM in open-vocabulary object detection and image segmentation methods.
  arXiv  Detail & Related papers  (2024-12-09T06:34:23Z)
• Semantic Segmentation by Early Region Proxy [53.594035639400616]
  We present a novel and efficient modeling that starts from interpreting the image as a tessellation of learnable regions. To model region-wise context, we exploit Transformer to encode regions in a sequence-to-sequence manner. Semantic segmentation is now carried out as per-region prediction on top of the encoded region embeddings.
  arXiv  Detail & Related papers  (2022-03-26T10:48:32Z)
• Point-Level Region Contrast for Object Detection Pre-Training [147.47349344401806]
  We present point-level region contrast, a self-supervised pre-training approach for the task of object detection. Our approach performs contrastive learning by directly sampling individual point pairs from different regions. Compared to an aggregated representation per region, our approach is more robust to the change in input region quality.
  arXiv  Detail & Related papers  (2022-02-09T18:56:41Z)
• Clustered Federated Learning via Generalized Total Variation Minimization [83.26141667853057]
  We study optimization methods to train local (or personalized) models for local datasets with a decentralized network structure. Our main conceptual contribution is to formulate federated learning as total variation minimization (GTV) Our main algorithmic contribution is a fully decentralized federated learning algorithm.
  arXiv  Detail & Related papers  (2021-05-26T18:07:19Z)
• Sign-Agnostic CONet: Learning Implicit Surface Reconstructions by Sign-Agnostic Optimization of Convolutional Occupancy Networks [39.65056638604885]
  We learn implicit surface reconstruction by sign-agnostic optimization of convolutional occupancy networks. We show this goal can be effectively achieved by a simple yet effective design.
  arXiv  Detail & Related papers  (2021-05-08T03:35:32Z)
• Making Affine Correspondences Work in Camera Geometry Computation [62.7633180470428]
  Local features provide region-to-region rather than point-to-point correspondences. We propose guidelines for effective use of region-to-region matches in the course of a full model estimation pipeline. Experiments show that affine solvers can achieve accuracy comparable to point-based solvers at faster run-times.
  arXiv  Detail & Related papers  (2020-07-20T12:07:48Z)
• Real-Time Regression with Dividing Local Gaussian Processes [62.01822866877782]
  Local Gaussian processes are a novel, computationally efficient modeling approach based on Gaussian process regression. Due to an iterative, data-driven division of the input space, they achieve a sublinear computational complexity in the total number of training points in practice. A numerical evaluation on real-world data sets shows their advantages over other state-of-the-art methods in terms of accuracy as well as prediction and update speed.
  arXiv  Detail & Related papers  (2020-06-16T18:43:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.