SONIC: Spectral Oriented Neural Invariant Convolutions

• URL: http://arxiv.org/abs/2601.19884v1
• Date: Tue, 27 Jan 2026 18:51:11 GMT
• Title: SONIC: Spectral Oriented Neural Invariant Convolutions
• Authors: Gijs Joppe Moens, Regina Beets-Tan, Eduardo H. P. Pooch,
• Abstract summary: Convolutional Neural Networks (CNNs) rely on fixed-size kernels scanning local patches.<n>ViTs provide global connectivity but lack spatial inductive bias, depend on explicit positional encodings, and remain tied to the initial patch size.<n>We introduce SONIC, a continuous spectral parameterisation that models convolutional operators using a small set of shared, orientation-selective components.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Convolutional Neural Networks (CNNs) rely on fixed-size kernels scanning local patches, which limits their ability to capture global context or long-range dependencies without very deep architectures. Vision Transformers (ViTs), in turn, provide global connectivity but lack spatial inductive bias, depend on explicit positional encodings, and remain tied to the initial patch size. Bridging these limitations requires a representation that is both structured and global. We introduce SONIC (Spectral Oriented Neural Invariant Convolutions), a continuous spectral parameterisation that models convolutional operators using a small set of shared, orientation-selective components. These components define smooth responses across the full frequency domain, yielding global receptive fields and filters that adapt naturally across resolutions. Across synthetic benchmarks, large-scale image classification, and 3D medical datasets, SONIC shows improved robustness to geometric transformations, noise, and resolution shifts, and matches or exceeds convolutional, attention-based, and prior spectral architectures with an order of magnitude fewer parameters. These results demonstrate that continuous, orientation-aware spectral parameterisations provide a principled and scalable alternative to conventional spatial and spectral operators.

Related papers

• Plain Transformers are Surprisingly Powerful Link Predictors [57.01966734467712]
  Link prediction is a core challenge in graph machine learning, demanding models that capture rich and complex topological dependencies.<n>While Graph Neural Networks (GNNs) are the standard solution, state-of-the-art pipelines often rely on explicit structurals or memory-intensive node embeddings.<n>We present PENCIL, an encoder-only plain Transformer that replaces hand-crafted priors with attention over sampled local subgraphs.
  arXiv  Detail & Related papers  (2026-02-02T02:45:52Z)
• Renormalization Group Guided Tensor Network Structure Search [58.0378300612202]
  Network structure search (TN-SS) aims to automatically discover optimal network topologies and rank robustness for efficient tensor decomposition in high-dimensional data representation.<n>We propose RGTN (Renormalization Group guided Network search), a physics-inspired framework transforming TN-SS via multi-scale renormalization group flows.
  arXiv  Detail & Related papers  (2025-12-31T06:31:43Z)
• Any-Optical-Model: A Universal Foundation Model for Optical Remote Sensing [24.03278912134978]
  We propose Any Optical Model (AOM) to accommodate arbitrary band compositions, sensor types, and resolution scales.<n>AOM consistently achieves state-of-the-art (SOTA) performance under challenging conditions such as band missing, cross sensor, and cross resolution settings.
  arXiv  Detail & Related papers  (2025-12-19T04:21:01Z)
• Geometrically Constrained and Token-Based Probabilistic Spatial Transformers [5.437226012505534]
  We revisit Spatial Transformer Networks (STNs) as a canonicalization tool for transformer-based vision pipelines.<n>We propose a probabilistic, component-wise extension that improves robustness.<n> Experiments on challenging moth classification benchmarks demonstrate that our method consistently improves robustness compared to other STNs.
  arXiv  Detail & Related papers  (2025-09-14T11:30:53Z)
• Sheaf Graph Neural Networks via PAC-Bayes Spectral Optimization [13.021238902084647]
  Over-smoothing in Graph Neural Networks (GNNs) causes collapse in distinct node features.<n>We introduce SGPC (Sheaf GNNs with PAC-Bayes), a unified architecture that combines cellular-sheaf message passing with several mechanisms.<n> Experiments on nine homophilic and heterophilic benchmarks show that SGPC outperforms state-of-the-art spectral and sheaf-based GNNs.
  arXiv  Detail & Related papers  (2025-08-01T06:39:28Z)
• SDTN and TRN: Adaptive Spectral-Spatial Feature Extraction for Hyperspectral Image Classification [1.2871580250533408]
  Hyperspectral image classification plays a pivotal role in precision agriculture, providing accurate insights into crop health monitoring, disease detection, and soil analysis.<n>Traditional methods struggle with high-dimensional data, spectral-spatial redundancy, and the scarcity of labeled samples, often leading to suboptimal performance.<n>To address these challenges, we propose the Self-Adaptive- Regularized Network (SDTN), which combines tensor decomposition with regularization mechanisms to dynamically adjust tensor ranks.<n>This approach not only maintains high classification accuracy but also significantly reduces computational complexity, making the framework highly suitable for real-time deployment in resource-constrained environments.
  arXiv  Detail & Related papers  (2025-07-13T04:53:33Z)
• Geometric Neural Process Fields [58.77241763774756]
  Geometric Neural Process Fields (G-NPF) is a probabilistic framework for neural radiance fields that explicitly captures uncertainty.<n>Building on these bases, we design a hierarchical latent variable model, allowing G-NPF to integrate structural information across multiple spatial levels.<n> Experiments on novel-view synthesis for 3D scenes, as well as 2D image and 1D signal regression, demonstrate the effectiveness of our method.
  arXiv  Detail & Related papers  (2025-02-04T14:17:18Z)
• Point Cloud Denoising With Fine-Granularity Dynamic Graph Convolutional Networks [58.050130177241186]
  Noise perturbations often corrupt 3-D point clouds, hindering downstream tasks such as surface reconstruction, rendering, and further processing. This paper introduces finegranularity dynamic graph convolutional networks called GDGCN, a novel approach to denoising in 3-D point clouds.
  arXiv  Detail & Related papers  (2024-11-21T14:19:32Z)
• Cross-Scan Mamba with Masked Training for Robust Spectral Imaging [51.557804095896174]
  We propose the Cross-Scanning Mamba, named CS-Mamba, that employs a Spatial-Spectral SSM for global-local balanced context encoding.<n>Experiment results show that our CS-Mamba achieves state-of-the-art performance and the masked training method can better reconstruct smooth features to improve the visual quality.
  arXiv  Detail & Related papers  (2024-08-01T15:14:10Z)
• ASWT-SGNN: Adaptive Spectral Wavelet Transform-based Self-Supervised Graph Neural Network [20.924559944655392]
  This paper proposes an Adaptive Spectral Wavelet Transform-based Self-Supervised Graph Neural Network (ASWT-SGNN) ASWT-SGNN accurately approximates the filter function in high-density spectral regions, avoiding costly eigen-decomposition. It achieves comparable performance to state-of-the-art models in node classification tasks.
  arXiv  Detail & Related papers  (2023-12-10T03:07:42Z)
• Locality-Aware Generalizable Implicit Neural Representation [54.93702310461174]
  Generalizable implicit neural representation (INR) enables a single continuous function to represent multiple data instances. We propose a novel framework for generalizable INR that combines a transformer encoder with a locality-aware INR decoder. Our framework significantly outperforms previous generalizable INRs and validates the usefulness of the locality-aware latents for downstream tasks.
  arXiv  Detail & Related papers  (2023-10-09T11:26:58Z)
• CSformer: Bridging Convolution and Transformer for Compressive Sensing [65.22377493627687]
  This paper proposes a hybrid framework that integrates the advantages of leveraging detailed spatial information from CNN and the global context provided by transformer for enhanced representation learning. The proposed approach is an end-to-end compressive image sensing method, composed of adaptive sampling and recovery. The experimental results demonstrate the effectiveness of the dedicated transformer-based architecture for compressive sensing.
  arXiv  Detail & Related papers  (2021-12-31T04:37:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.