Function-Counting Theory for Low-Dimensional Data Structures

• URL: http://arxiv.org/abs/2607.01010v1
• Date: Wed, 01 Jul 2026 14:47:36 GMT
• Title: Function-Counting Theory for Low-Dimensional Data Structures
• Abstract summary: This work attempts to provide a mathematical framework for binary classification on low-dimensional data.<n>Cover's theory relies on a general position assumption that blinds it to the underlying data structure.<n>We refine this assumption to account for the low-dimensionality of the data and derive counts that reflect the data structure.
• Score: 1.2318267573115804
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The success of deep learning models in classification and regression is widely attributed to the low-dimensional structure that real-world data tend to exhibit, despite their high-dimensional representation. This work attempts to provide a mathematical framework for binary classification on low-dimensional data, building on Cover's (1965) function-counting theory. With our framework, we aim to address the question of how the low-dimensional structure of the data affects the classification capabilities of learning models. Cover's theory relies on a general position assumption that blinds it to the underlying data structure. We refine this assumption to account for the low-dimensionality of the data and derive dichotomy counts that reflect the data structure. We further extend Cover's separation capacity and problem of generalization to the low-dimensional setting, enabling the impact of the underlying data structure on both to be analyzed.

Related papers

• Diffusion Model for Manifold Data: Score Decomposition, Curvature, and Statistical Complexity [93.52679474816728]
  Diffusion models have become a leading framework in generative modeling.<n>This paper investigates how diffusion models learn structured data, focusing on two key aspects: statistical complexity and influence of data geometric properties.<n>We also highlight the interplay of manifold curvature with the structures in the score function.
  arXiv  Detail & Related papers  (2026-03-21T04:40:19Z)
• Learning with Hidden Factorial Structure [2.474908349649168]
  Recent advances suggest that text and image data contain such hidden structures, which help mitigate the curse of dimensionality.<n>We present a controlled experimental framework to test whether neural networks can indeed exploit such "hidden factorial structures"
  arXiv  Detail & Related papers  (2024-11-02T22:32:53Z)
• TopoFR: A Closer Look at Topology Alignment on Face Recognition [58.45515807380505]
  We propose TopoFR, a novel FR model that leverages a topological structure alignment strategy called PTSA and a hard sample mining strategy named SDE.<n> PTSA uses persistent homology to align the topological structures of the input and latent spaces, effectively preserving the structure information and improving the generalization performance of FR model.<n> Experimental results on popular face benchmarks demonstrate the superiority of our TopoFR over the state-of-the-art methods.
  arXiv  Detail & Related papers  (2024-10-14T14:58:30Z)
• Dissecting embedding method: learning higher-order structures from data [0.0]
  Geometric deep learning methods for data learning often include set of assumptions on the geometry of the feature space. These assumptions together with data being discrete and finite can cause some generalisations, which are likely to create wrong interpretations of the data and models outputs.
  arXiv  Detail & Related papers  (2024-10-14T08:19:39Z)
• Distributional Reduction: Unifying Dimensionality Reduction and Clustering with Gromov-Wasserstein [56.62376364594194]
  Unsupervised learning aims to capture the underlying structure of potentially large and high-dimensional datasets.<n>In this work, we revisit these approaches under the lens of optimal transport and exhibit relationships with the Gromov-Wasserstein problem.<n>This unveils a new general framework, called distributional reduction, that recovers DR and clustering as special cases and allows addressing them jointly within a single optimization problem.
  arXiv  Detail & Related papers  (2024-02-03T19:00:19Z)
• Hierarchical clustering with dot products recovers hidden tree structure [53.68551192799585]
  In this paper we offer a new perspective on the well established agglomerative clustering algorithm, focusing on recovery of hierarchical structure. We recommend a simple variant of the standard algorithm, in which clusters are merged by maximum average dot product and not, for example, by minimum distance or within-cluster variance. We demonstrate that the tree output by this algorithm provides a bona fide estimate of generative hierarchical structure in data, under a generic probabilistic graphical model.
  arXiv  Detail & Related papers  (2023-05-24T11:05:12Z)
• Towards a mathematical understanding of learning from few examples with nonlinear feature maps [68.8204255655161]
  We consider the problem of data classification where the training set consists of just a few data points. We reveal key relationships between the geometry of an AI model's feature space, the structure of the underlying data distributions, and the model's generalisation capabilities.
  arXiv  Detail & Related papers  (2022-11-07T14:52:58Z)
• A geometric framework for outlier detection in high-dimensional data [0.0]
  Outlier or anomaly detection is an important task in data analysis. We provide a framework that exploits the metric structure of a data set. We show that exploiting this structure significantly improves the detection of outlying observations in high-dimensional data.
  arXiv  Detail & Related papers  (2022-07-01T12:07:51Z)
• Learning from few examples with nonlinear feature maps [68.8204255655161]
  We explore the phenomenon and reveal key relationships between dimensionality of AI model's feature space, non-degeneracy of data distributions, and the model's generalisation capabilities. The main thrust of our present analysis is on the influence of nonlinear feature transformations mapping original data into higher- and possibly infinite-dimensional spaces on the resulting model's generalisation capabilities.
  arXiv  Detail & Related papers  (2022-03-31T10:36:50Z)
• Visualizing the Finer Cluster Structure of Large-Scale and High-Dimensional Data [7.400745342582259]
  We propose using a generalized sigmoid function to model the distance similarity in both high- and low-dimensional spaces. Using both simulated and real-world data sets, we show that our proposed method can generate visualization results comparable to those of uniform manifold approximation and projection.
  arXiv  Detail & Related papers  (2020-07-17T01:36:45Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.