Improving Set Function Approximation with Quasi-Arithmetic Neural Networks

• URL: http://arxiv.org/abs/2602.04941v1
• Date: Wed, 04 Feb 2026 18:36:31 GMT
• Title: Improving Set Function Approximation with Quasi-Arithmetic Neural Networks
• Authors: Tomas Tokar, Scott Sanner,
• Abstract summary: We propose quasi-arithmetic neural networks (QUANNs)<n>QUANNs are universal approximators for a broad class of common set-function decompositions.<n>We provide a theoretical analysis showing that, QUANNs are universal approximators for a broad class of common set-function decompositions.
• Score: 23.73257235603082
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Sets represent a fundamental abstraction across many types of data. To handle the unordered nature of set-structured data, models such as DeepSets and PointNet rely on fixed, non-learnable pooling operations (e.g., sum or max) -- a design choice that can hinder the transferability of learned embeddings and limits model expressivity. More recently, learnable aggregation functions have been proposed as more expressive alternatives. In this work, we advance this line of research by introducing the Neuralized Kolmogorov Mean (NKM) -- a novel, trainable framework for learning a generalized measure of central tendency through an invertible neural function. We further propose quasi-arithmetic neural networks (QUANNs), which incorporate the NKM as a learnable aggregation function. We provide a theoretical analysis showing that, QUANNs are universal approximators for a broad class of common set-function decompositions and, thanks to their invertible neural components, learn more structured latent representations. Empirically, QUANNs outperform state-of-the-art baselines across diverse benchmarks, while learning embeddings that transfer effectively even to tasks that do not involve sets.

Related papers

• Enhancing Time Series Classification with Diversity-Driven Neural Network Ensembles [0.776514389034479]
  We introduce a diversity-driven ensemble learning framework that explicitly encourages feature diversity among neural network ensemble members.<n>We evaluate our framework on 128 datasets from the UCR archive and show that it achieves SOTA performance with fewer models.
  arXiv  Detail & Related papers  (2026-02-07T15:05:04Z)
• Weights initialization of neural networks for function approximation [0.9099663022952497]
  Neural network-based function approximation plays a pivotal role in the advancement of scientific computing and machine learning.<n>We propose a reusable framework based on on basis function pretraining.<n>In this approach, basis neural networks are first trained to approximate families of structural correspondences on a reference domain.<n>Their learned parameters are then used to initialize networks for more complex target functions.
  arXiv  Detail & Related papers  (2025-10-09T19:56:26Z)
• Enhancing Neural Subset Selection: Integrating Background Information into Set Representations [53.15923939406772]
  We show that when the target value is conditioned on both the input set and subset, it is essential to incorporate an textitinvariant sufficient statistic of the superset into the subset of interest. This ensures that the output value remains invariant to permutations of the subset and its corresponding superset, enabling identification of the specific superset from which the subset originated.
  arXiv  Detail & Related papers  (2024-02-05T16:09:35Z)
• Equivariance with Learned Canonicalization Functions [77.32483958400282]
  We show that learning a small neural network to perform canonicalization is better than using predefineds. Our experiments show that learning the canonicalization function is competitive with existing techniques for learning equivariant functions across many tasks.
  arXiv  Detail & Related papers  (2022-11-11T21:58:15Z)
• Neural Attentive Circuits [93.95502541529115]
  We introduce a general purpose, yet modular neural architecture called Neural Attentive Circuits (NACs) NACs learn the parameterization and a sparse connectivity of neural modules without using domain knowledge. NACs achieve an 8x speedup at inference time while losing less than 3% performance.
  arXiv  Detail & Related papers  (2022-10-14T18:00:07Z)
• Equivariant Transduction through Invariant Alignment [71.45263447328374]
  We introduce a novel group-equivariant architecture that incorporates a group-in hard alignment mechanism. We find that our network's structure allows it to develop stronger equivariant properties than existing group-equivariant approaches. We additionally find that it outperforms previous group-equivariant networks empirically on the SCAN task.
  arXiv  Detail & Related papers  (2022-09-22T11:19:45Z)
• Random Graph-Based Neuromorphic Learning with a Layer-Weaken Structure [4.477401614534202]
  We transform the random graph theory into an NN model with practical meaning and based on clarifying the input-output relationship of each neuron. Under the usage of this low-operation cost approach, neurons are assigned to several groups of which connection relationships can be regarded as uniform representations of random graphs they belong to. We develop a joint classification mechanism involving information interaction between multiple RGNNs and realize significant performance improvements in supervised learning for three benchmark tasks.
  arXiv  Detail & Related papers  (2021-11-17T03:37:06Z)
• Dynamic Inference with Neural Interpreters [72.90231306252007]
  We present Neural Interpreters, an architecture that factorizes inference in a self-attention network as a system of modules. inputs to the model are routed through a sequence of functions in a way that is end-to-end learned. We show that Neural Interpreters perform on par with the vision transformer using fewer parameters, while being transferrable to a new task in a sample efficient manner.
  arXiv  Detail & Related papers  (2021-10-12T23:22:45Z)
• Tensor-based framework for training flexible neural networks [9.176056742068813]
  We propose a new learning algorithm which solves a constrained coupled matrix-tensor factorization (CMTF) problem. The proposed algorithm can handle different bases decomposition. The goal of this method is to compress large pretrained NN models, by replacing tensorworks, em i.e., one or multiple layers of the original network, by a new flexible layer.
  arXiv  Detail & Related papers  (2021-06-25T10:26:48Z)
• PredRNN: A Recurrent Neural Network for Spatiotemporal Predictive Learning [109.84770951839289]
  We present PredRNN, a new recurrent network for learning visual dynamics from historical context. We show that our approach obtains highly competitive results on three standard datasets.
  arXiv  Detail & Related papers  (2021-03-17T08:28:30Z)
• Sparsely ensembled convolutional neural network classifiers via reinforcement learning [0.0]
  We consider convolutional neural network (CNN) ensemble learning with the objective function inspired by least action principle. We teach an agent to perceive images through the set of pre-trained classifiers and want the resulting dynamically configured system to unfold the computational graph. Our experimental results prove, that if the agent exploits the dynamic (and context-dependent) structure of computations, it outperforms conventional ensemble learning.
  arXiv  Detail & Related papers  (2021-02-07T21:26:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.