Learning in Function Spaces: An Unified Functional Analytic View of Supervised and Unsupervised Learning

• URL: http://arxiv.org/abs/2603.14272v1
• Date: Sun, 15 Mar 2026 08:06:31 GMT
• Title: Learning in Function Spaces: An Unified Functional Analytic View of Supervised and Unsupervised Learning
• Abstract summary: Many machine learning algorithms can be interpreted as procedures for estimating functions defined on the data distribution.<n>We present a conceptual framework that formulates a wide range of learning problems as variational optimization over function spaces induced by the data distribution.
• Score: 1.4637008301481798
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Many machine learning algorithms can be interpreted as procedures for estimating functions defined on the data distribution. In this paper we present a conceptual framework that formulates a wide range of learning problems as variational optimization over function spaces induced by the data distribution. Within this framework the data distribution defines operators that capture structural properties of the data, such as similarity relations or statistical dependencies. Learning algorithms can then be viewed as estimating functions expressed in bases determined by these operators. This perspective provides a unified way to interpret several learning paradigms. In supervised learning the objective functional is defined using labeled data and typically corresponds to minimizing prediction risk, whereas unsupervised learning relies on structural properties of the input distribution and leads to objectives based on similarity or smoothness constraints. From this viewpoint, the distinction between learning paradigms arises primarily from the choice of the functional being optimized rather than from the underlying function space. We illustrate this framework by discussing connections with kernel methods, spectral clustering, and manifold learning, highlighting how operators induced by data distributions naturally define function representations used by learning algorithms. The goal of this work is not to introduce a new algorithm but to provide a conceptual framework that clarifies the role of function spaces and operators in modern machine learning.

Related papers

• Self-explainable Operator Learning for Discovering Spatial Patterns in Functional Data [0.24554686192257422]
  Operator learning has emerged as a powerful tool for modeling complex physical systems in functional spaces.<n>We introduce a self-explainable operator learning framework that overcomes this challenge by reformulating operator learning as a linear combination of generalized functional linear models.<n>We demonstrate the framework on function-to-scalar and function-to-function mappings in fluid flow problems involving blood flow and unsteady aerodynamics.
  arXiv  Detail & Related papers  (2026-07-02T14:11:20Z)
• Operator learning meets inverse problems: A probabilistic perspective [5.311997112925023]
  Operator learning offers a robust framework for approximating mappings between infinite-dimensional function spaces.<n>This chapter surveys developments at the intersection of operator learning and inverse problems.
  arXiv  Detail & Related papers  (2025-08-27T18:34:40Z)
• Neural Feature Learning in Function Space [5.807950618412389]
  We present a novel framework for learning system design with neural feature extractors. We introduce the feature geometry, which unifies statistical dependence and feature representations in a function space equipped with inner products. We propose a nesting technique, which provides systematic algorithm designs for learning the optimal features from data samples.
  arXiv  Detail & Related papers  (2023-09-18T20:39:12Z)
• An Entropy-Based Model for Hierarchical Learning [3.1473798197405944]
  A common feature among real-world datasets is that data domains are multiscale. We propose a learning model that exploits this multiscale data structure. The hierarchical learning model is inspired by the logical and progressive easy-to-hard learning mechanism of human beings.
  arXiv  Detail & Related papers  (2022-12-30T13:14:46Z)
• Offline Reinforcement Learning with Differentiable Function Approximation is Provably Efficient [65.08966446962845]
  offline reinforcement learning, which aims at optimizing decision-making strategies with historical data, has been extensively applied in real-life applications. We take a step by considering offline reinforcement learning with differentiable function class approximation (DFA) Most importantly, we show offline differentiable function approximation is provably efficient by analyzing the pessimistic fitted Q-learning algorithm.
  arXiv  Detail & Related papers  (2022-10-03T07:59:42Z)
• Invariant Causal Mechanisms through Distribution Matching [86.07327840293894]
  In this work we provide a causal perspective and a new algorithm for learning invariant representations. Empirically we show that this algorithm works well on a diverse set of tasks and in particular we observe state-of-the-art performance on domain generalization.
  arXiv  Detail & Related papers  (2022-06-23T12:06:54Z)
• Stabilizing Q-learning with Linear Architectures for Provably Efficient Learning [53.17258888552998]
  This work proposes an exploration variant of the basic $Q$-learning protocol with linear function approximation. We show that the performance of the algorithm degrades very gracefully under a novel and more permissive notion of approximation error.
  arXiv  Detail & Related papers  (2022-06-01T23:26:51Z)
• Learning Operators with Coupled Attention [9.715465024071333]
  We propose a novel operator learning method, LOCA, motivated from the recent success of the attention mechanism. In our architecture the input functions are mapped to a finite set of features which are then averaged with attention weights that depend on the output query locations. By coupling these attention weights together with an integral transform, LOCA is able to explicitly learn correlations in the target output functions.
  arXiv  Detail & Related papers  (2022-01-04T08:22:03Z)
• A geometric perspective on functional outlier detection [0.0]
  We develop a conceptualization of functional outlier detection that is more widely applicable and realistic than previously proposed. We show that simple manifold learning methods can be used to reliably infer and visualize the geometric structure of functional data sets. Our experiments on synthetic and real data sets demonstrate that this approach leads to outlier detection performances at least on par with existing functional data-specific methods.
  arXiv  Detail & Related papers  (2021-09-14T17:42:57Z)
• Learning outside the Black-Box: The pursuit of interpretable models [78.32475359554395]
  This paper proposes an algorithm that produces a continuous global interpretation of any given continuous black-box function. Our interpretation represents a leap forward from the previous state of the art.
  arXiv  Detail & Related papers  (2020-11-17T12:39:44Z)
• Estimating Structural Target Functions using Machine Learning and Influence Functions [103.47897241856603]
  We propose a new framework for statistical machine learning of target functions arising as identifiable functionals from statistical models. This framework is problem- and model-agnostic and can be used to estimate a broad variety of target parameters of interest in applied statistics. We put particular focus on so-called coarsening at random/doubly robust problems with partially unobserved information.
  arXiv  Detail & Related papers  (2020-08-14T16:48:29Z)
• Explaining Black Box Predictions and Unveiling Data Artifacts through Influence Functions [55.660255727031725]
  Influence functions explain the decisions of a model by identifying influential training examples. We conduct a comparison between influence functions and common word-saliency methods on representative tasks. We develop a new measure based on influence functions that can reveal artifacts in training data.
  arXiv  Detail & Related papers  (2020-05-14T00:45:23Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.