Beyond Convolution: A Taxonomy of Structured Operators for Learning-Based Image Processing

• URL: http://arxiv.org/abs/2603.12067v2
• Date: Fri, 13 Mar 2026 13:53:42 GMT
• Title: Beyond Convolution: A Taxonomy of Structured Operators for Learning-Based Image Processing
• Abstract summary: This paper presents a systematic taxonomy of operators that extend or replace the standard convolution in learning-based image processing pipelines.<n>For each family, we provide a formal definition, a discussion of its structural properties with respect to the convolution, and a critical analysis of the tasks for which the operator is most appropriate.
• Score: 3.934224774675743
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The convolution operator is the fundamental building block of modern convolutional neural networks (CNNs), owing to its simplicity, translational equivariance, and efficient implementation. However, its structure as a fixed, linear, locally-averaging operator limits its ability to capture structured signal properties such as low-rank decompositions, adaptive basis representations, and non-uniform spatial dependencies. This paper presents a systematic taxonomy of operators that extend or replace the standard convolution in learning-based image processing pipelines. We organise the landscape of alternative operators into five families: (i) decomposition-based operators, which separate structural and noise components through singular value or tensor decompositions; (ii) adaptive weighted operators, which modulate kernel contributions as a function of spatial position or signal content; (iii) basis-adaptive operators, which optimise the analysis bases together with the network weights; (iv) integral and kernel operators, which generalise the convolution to position-dependent and non-linear kernels; and (v) attention-based operators, which relax the locality assumption entirely. For each family, we provide a formal definition, a discussion of its structural properties with respect to the convolution, and a critical analysis of the tasks for which the operator is most appropriate. We further provide a comparative analysis of all families across relevant dimensions -- linearity, locality, equivariance, computational cost, and suitability for image-to-image and image-to-label tasks -- and outline the open challenges and future directions of this research area.

Related papers

• Where to Compute and How to Interact: Operator-Readable Adaptation with Gauge-Aware Transport [17.892151567058367]
  We propose a physics-informed adaptive allocation system for partial differential equations (PDEs)<n>The Gauge-Aware Adaptive Mesh Neural Operator (GA-AMNO) can account for and test why computation is allocated to particular locations.<n> Experiments on five PDE benchmarks demonstrate improved predictive accuracy, while controlled interventions and geometric-mismatch analyses verify the roles of allocation and interaction.
  arXiv  Detail & Related papers  (2026-09-14T14:13:30Z)
• Operator Calculus for Population-Based Optimization: A Mean-Field Convergence Theory [0.2770822269241973]
  Population-based and distributional optimization methods are widely used for nonreactional problems.<n>We provide a toolkit for convergence of composite composite algorithms.
  arXiv  Detail & Related papers  (2026-06-12T09:16:58Z)
• Topological Neural Operators [22.267195753988418]
  We introduce Topological Neural Operators (TNOs), a principled framework for operator learning on cell complexes.<n>TNOs represent data as features defined on cells of varying dimension and model their interactions through Discrete Exterior Calculus.<n>We propose Hierarchical TNOs (HTNOs), which incorporate learned coarse complexes to propagate long-range and topology-dependent information.
  arXiv  Detail & Related papers  (2026-06-08T17:54:33Z)
• Weight-Informed Self-Explaining Clustering for Mixed-Type Tabular Data [63.62853416081748]
  WISE is a framework that unifies representation, feature weighting, clustering, and interpretation.<n>It produces faithful, human-interpretable explanations grounded in the same primitives that drive clustering.
  arXiv  Detail & Related papers  (2026-04-07T13:18:31Z)
• RKHS Representation of Algebraic Convolutional Filters with Integral Operators [111.57971404925486]
  In this paper, we develop a theory showing that the range of integral operators naturally induces RKHS convolutional signal models.<n>We show that filtering with integral operators corresponds to iterated box products, giving rise to a unital kernel algebra.<n>Our results establish precise connections between eigendecompositions and RKHS representations in graphon signal processing, extend naturally to directed graphons, and enable novel spatial-spectral localization results.
  arXiv  Detail & Related papers  (2026-02-22T08:28:34Z)
• The Coverage Principle: A Framework for Understanding Compositional Generalization [31.762330857169914]
  We show that models relying primarily on pattern matching for compositional tasks cannot reliably generalize beyond substituting fragments that yield identical results when used in the same contexts.<n>We demonstrate that this framework has a strong predictive power for the generalization capabilities of Transformers.
  arXiv  Detail & Related papers  (2025-05-26T17:55:15Z)
• Learning dynamically inspired invariant subspaces for Koopman and transfer operator approximation [0.0]
  We use general operator and representational learning to approximate transfer and Koopman operators.<n>Specifically, we machine-learn orthonormal basis functions that are dynamically tailored to the system.<n>We illustrate our approach with examples that showcase the retrieval of spectral properties from the estimated operator.
  arXiv  Detail & Related papers  (2025-05-08T09:32:39Z)
• Partial Transportability for Domain Generalization [56.37032680901525]
  Building on the theory of partial identification and transportability, this paper introduces new results for bounding the value of a functional of the target distribution.<n>Our contribution is to provide the first general estimation technique for transportability problems.<n>We propose a gradient-based optimization scheme for making scalable inferences in practice.
  arXiv  Detail & Related papers  (2025-03-30T22:06:37Z)
• Mathematical Foundation of Interpretable Equivariant Surrogate Models [4.433915375867081]
  This paper introduces a rigorous mathematical framework for neural network explainability.<n>The central concept involves quantifying the distance between GEOs by measuring the non-commutativity of specific diagrams.<n>We show how it can be applied in classical machine learning scenarios, like image classification with convolutional neural networks.
  arXiv  Detail & Related papers  (2025-03-03T15:06:43Z)
• Stable Nonconvex-Nonconcave Training via Linear Interpolation [51.668052890249726]
  This paper presents a theoretical analysis of linearahead as a principled method for stabilizing (large-scale) neural network training. We argue that instabilities in the optimization process are often caused by the nonmonotonicity of the loss landscape and show how linear can help by leveraging the theory of nonexpansive operators.
  arXiv  Detail & Related papers  (2023-10-20T12:45:12Z)
• Tuning the Geometry of Graph Neural Networks [0.7614628596146599]
  spatial graph convolution operators have been heralded as key to the success of Graph Neural Networks (GNNs) We show that this aggregation operator is in fact tunable, and explicit regimes in which certain choices of operators -- and therefore, embedding geometries -- might be more appropriate.
  arXiv  Detail & Related papers  (2022-07-12T23:28:03Z)
• X-volution: On the unification of convolution and self-attention [52.80459687846842]
  We propose a multi-branch elementary module composed of both convolution and self-attention operation. The proposed X-volution achieves highly competitive visual understanding improvements.
  arXiv  Detail & Related papers  (2021-06-04T04:32:02Z)
• A Trainable Optimal Transport Embedding for Feature Aggregation and its Relationship to Attention [96.77554122595578]
  We introduce a parametrized representation of fixed size, which embeds and then aggregates elements from a given input set according to the optimal transport plan between the set and a trainable reference. Our approach scales to large datasets and allows end-to-end training of the reference, while also providing a simple unsupervised learning mechanism with small computational cost.
  arXiv  Detail & Related papers  (2020-06-22T08:35:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.