Simplifying complex machine learning by linearly separable network embedding spaces

• URL: http://arxiv.org/abs/2410.01865v1
• Date: Wed, 2 Oct 2024 11:41:17 GMT
• Title: Simplifying complex machine learning by linearly separable network embedding spaces
• Authors: Alexandros Xenos, Noel-Malod Dognin, Natasa Przulj,
• Abstract summary: Low-dimensional embeddings are a cornerstone in the modelling and analysis of complex networks. We show that there are structural properties of network data that yields this linearity. We introduce novel graphlet-based methods enabling embedding of networks into more linearly separable spaces.
• Score: 45.62331048595689
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Low-dimensional embeddings are a cornerstone in the modelling and analysis of complex networks. However, most existing approaches for mining network embedding spaces rely on computationally intensive machine learning systems to facilitate downstream tasks. In the field of NLP, word embedding spaces capture semantic relationships \textit{linearly}, allowing for information retrieval using \textit{simple linear operations} on word embedding vectors. Here, we demonstrate that there are structural properties of network data that yields this linearity. We show that the more homophilic the network representation, the more linearly separable the corresponding network embedding space, yielding better downstream analysis results. Hence, we introduce novel graphlet-based methods enabling embedding of networks into more linearly separable spaces, allowing for their better mining. Our fundamental insights into the structure of network data that enable their \textit{\textbf{linear}} mining and exploitation enable the ML community to build upon, towards efficiently and explainably mining of the complex network data.

Related papers

• How Low Can You Go? Searching for the Intrinsic Dimensionality of Complex Networks using Metric Node Embeddings [1.7061868168035932]
  Low-dimensional embeddings are essential for machine learning tasks involving graphs. We prove that lower dimensional embeddings are possible when using Euclidean metric embeddings. For the first time, we demonstrate that even large-scale networks can be effectively embedded in very low-dimensional spaces.
  arXiv  Detail & Related papers  (2025-03-03T16:37:38Z)
• On the Local Complexity of Linear Regions in Deep ReLU Networks [15.335716956682203]
  We show theoretically that ReLU networks that learn low-dimensional feature representations have a lower local complexity. In particular, we show that the local complexity serves as an upper bound on the total variation of the function over the input data distribution.
  arXiv  Detail & Related papers  (2024-12-24T08:42:39Z)
• SpaceMesh: A Continuous Representation for Learning Manifold Surface Meshes [61.110517195874074]
  We present a scheme to directly generate manifold, polygonal meshes of complex connectivity as the output of a neural network. Our key innovation is to define a continuous latent connectivity space at each mesh, which implies the discrete mesh. In applications, this approach not only yields high-quality outputs from generative models, but also enables directly learning challenging geometry processing tasks such as mesh repair.
  arXiv  Detail & Related papers  (2024-09-30T17:59:03Z)
• Defining Neural Network Architecture through Polytope Structures of Dataset [53.512432492636236]
  This paper defines upper and lower bounds for neural network widths, which are informed by the polytope structure of the dataset in question. We develop an algorithm to investigate a converse situation where the polytope structure of a dataset can be inferred from its corresponding trained neural networks. It is established that popular datasets such as MNIST, Fashion-MNIST, and CIFAR10 can be efficiently encapsulated using no more than two polytopes with a small number of faces.
  arXiv  Detail & Related papers  (2024-02-04T08:57:42Z)
• Linear Mode Connectivity in Sparse Neural Networks [1.30536490219656]
  We study how neural network pruning with synthetic data leads to sparse networks with unique training properties. We find that these properties lead to syntheticworks matching the performance of traditional IMP with up to 150x less training points in settings where distilled data applies.
  arXiv  Detail & Related papers  (2023-10-28T17:51:39Z)
• Exploring explicit coarse-grained structure in artificial neural networks [0.0]
  We propose to employ the hierarchical coarse-grained structure in the artificial neural networks explicitly to improve the interpretability without degrading performance. One is a neural network called TaylorNet, which aims to approximate the general mapping from input data to output result in terms of Taylor series directly. The other is a new setup for data distillation, which can perform multi-level abstraction of the input dataset and generate new data.
  arXiv  Detail & Related papers  (2022-11-03T13:06:37Z)
• Globally Gated Deep Linear Networks [3.04585143845864]
  We introduce Globally Gated Deep Linear Networks (GGDLNs) where gating units are shared among all processing units in each layer. We derive exact equations for the generalization properties in these networks in the finite-width thermodynamic limit. Our work is the first exact theoretical solution of learning in a family of nonlinear networks with finite width.
  arXiv  Detail & Related papers  (2022-10-31T16:21:56Z)
• Inducing Gaussian Process Networks [80.40892394020797]
  We propose inducing Gaussian process networks (IGN), a simple framework for simultaneously learning the feature space as well as the inducing points. The inducing points, in particular, are learned directly in the feature space, enabling a seamless representation of complex structured domains. We report on experimental results for real-world data sets showing that IGNs provide significant advances over state-of-the-art methods.
  arXiv  Detail & Related papers  (2022-04-21T05:27:09Z)
• Dive into Layers: Neural Network Capacity Bounding using Algebraic Geometry [55.57953219617467]
  We show that the learnability of a neural network is directly related to its size. We use Betti numbers to measure the topological geometric complexity of input data and the neural network. We perform the experiments on a real-world dataset MNIST and the results verify our analysis and conclusion.
  arXiv  Detail & Related papers  (2021-09-03T11:45:51Z)
• Parallel Machine Learning for Forecasting the Dynamics of Complex Networks [0.0]
  We present a machine learning scheme for forecasting the dynamics of large complex networks. We use a parallel architecture that mimics the topology of the network of interest.
  arXiv  Detail & Related papers  (2021-08-27T06:06:41Z)
• Learning Connectivity of Neural Networks from a Topological Perspective [80.35103711638548]
  We propose a topological perspective to represent a network into a complete graph for analysis. By assigning learnable parameters to the edges which reflect the magnitude of connections, the learning process can be performed in a differentiable manner. This learning process is compatible with existing networks and owns adaptability to larger search spaces and different tasks.
  arXiv  Detail & Related papers  (2020-08-19T04:53:31Z)
• Neural networks adapting to datasets: learning network size and topology [77.34726150561087]
  We introduce a flexible setup allowing for a neural network to learn both its size and topology during the course of a gradient-based training. The resulting network has the structure of a graph tailored to the particular learning task and dataset.
  arXiv  Detail & Related papers  (2020-06-22T12:46:44Z)
• EPINE: Enhanced Proximity Information Network Embedding [2.257737378757467]
  In this work, we devote to mining valuable information in adjacency matrices at a deeper level. Under the same objective, many NE methods calculate high-order proximity by the powers of adjacency matrices. We propose to redefine high-order proximity in a more intuitive manner.
  arXiv  Detail & Related papers  (2020-03-04T15:57:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.