Representer Theorems for Metric and Preference Learning: A Geometric Perspective

• URL: http://arxiv.org/abs/2304.03720v1
• Date: Fri, 7 Apr 2023 16:34:25 GMT
• Title: Representer Theorems for Metric and Preference Learning: A Geometric Perspective
• Authors: Peyman Morteza
• Abstract summary: We obtain a novel representer theorem for the simultaneous task of metric and preference learning. We show that our framework can be applied to the task of metric learning from triplet comparisons. In the case of Reproducing Kernel Hilbert Spaces, we demonstrate that the solution to the learning problem can be expressed using kernel terms.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We explore the metric and preference learning problem in Hilbert spaces. We obtain a novel representer theorem for the simultaneous task of metric and preference learning. Our key observation is that the representer theorem can be formulated with respect to the norm induced by the inner product inherent in the problem structure. Additionally, we demonstrate how our framework can be applied to the task of metric learning from triplet comparisons and show that it leads to a simple and self-contained representer theorem for this task. In the case of Reproducing Kernel Hilbert Spaces (RKHS), we demonstrate that the solution to the learning problem can be expressed using kernel terms, akin to classical representer theorems.

Related papers

• Machine Learning of the Prime Distribution [49.84018914962972]
  We provide a theoretical argument explaining the experimental observations of Yang-Hui He about the learnability of primes. We also posit that the ErdHos-Kac law would very unlikely be discovered by current machine learning techniques.
  arXiv  Detail & Related papers  (2024-03-19T09:47:54Z)
• Understanding Probe Behaviors through Variational Bounds of Mutual Information [53.520525292756005]
  We provide guidelines for linear probing by constructing a novel mathematical framework leveraging information theory. First, we connect probing with the variational bounds of mutual information (MI) to relax the probe design, equating linear probing with fine-tuning. We show that the intermediate representations can have the biggest MI estimate because of the tradeoff between better separability and decreasing MI.
  arXiv  Detail & Related papers  (2023-12-15T18:38:18Z)
• Connecting classical finite exchangeability to quantum theory [69.62715388742298]
  Exchangeability is a fundamental concept in probability theory and statistics. We show how a de Finetti-like representation theorem for finitely exchangeable sequences requires a mathematical representation which is formally equivalent to quantum theory.
  arXiv  Detail & Related papers  (2023-06-06T17:15:19Z)
• Simplicial quantum contextuality [0.0]
  We introduce a new framework for contextuality based on simplicial sets, models of topological spaces that play a prominent role in homotopy theory. Our approach extends measurement scenarios to consist of spaces (rather than sets) of measurements and outcomes. We present a topologically inspired new proof of Fine's theorem for characterizing noncontextuality in Bell scenarios.
  arXiv  Detail & Related papers  (2022-04-13T22:03:28Z)
• A singular Riemannian geometry approach to Deep Neural Networks I. Theoretical foundations [77.86290991564829]
  Deep Neural Networks are widely used for solving complex problems in several scientific areas, such as speech recognition, machine translation, image analysis. We study a particular sequence of maps between manifold, with the last manifold of the sequence equipped with a Riemannian metric. We investigate the theoretical properties of the maps of such sequence, eventually we focus on the case of maps between implementing neural networks of practical interest.
  arXiv  Detail & Related papers  (2021-12-17T11:43:30Z)
• Learning primal-dual sparse kernel machines [10.230121160034674]
  Traditionally, kernel methods rely on the representer theorem which states that the solution to a learning problem is obtained as a linear combination of the data mapped into the reproducing kernel Hilbert space (RKHS) We propose to search for a solution in RKHS that has a pre-image decomposition in the original data space, where the elements don't necessarily correspond to the elements in the training set. Our gradient-based method then hinges on optimisation over possibly sparse elements in the input space, and enables us to obtain a kernel-based model with both primal and dual sparsity.
  arXiv  Detail & Related papers  (2021-08-27T09:38:53Z)
• Williamson theorem in classical, quantum, and statistical physics [0.0]
  We show that applying the Williamson theorem reveals the normal-mode coordinates and frequencies of the system in the Hamiltonian scenario. A more advanced topic concerning uncertainty relations is developed to show once more its utility in a distinct and modern perspective.
  arXiv  Detail & Related papers  (2021-06-22T17:59:59Z)
• Learning to Guide a Saturation-Based Theorem Prover [9.228237801323042]
  TRAIL is a deep learning-based approach to theorem proving that characterizes core elements of saturation-based theorem proving within a neural framework. To the best of our knowledge, TRAIL is the first reinforcement learning-based approach to exceed the performance of a state-of-the-art traditional theorem prover.
  arXiv  Detail & Related papers  (2021-06-07T18:35:57Z)
• A Unifying and Canonical Description of Measure-Preserving Diffusions [60.59592461429012]
  A complete recipe of measure-preserving diffusions in Euclidean space was recently derived unifying several MCMC algorithms into a single framework. We develop a geometric theory that improves and generalises this construction to any manifold.
  arXiv  Detail & Related papers  (2021-05-06T17:36:55Z)
• INT: An Inequality Benchmark for Evaluating Generalization in Theorem Proving [36.194330645092634]
  In this paper, we introduce an INequality Theorem proving benchmark, specifically designed to test agents' generalization ability. Int is based on a procedure for generating theorems and proofs; this procedure's knobs allow us to measure 6 different types of generalization. We then evaluate the same agents augmented with Monte Carlo Tree Search (MCTS) at test time, and show that MCTS can help to prove new theorems.
  arXiv  Detail & Related papers  (2020-07-06T17:55:33Z)
• Learning to Prove Theorems by Learning to Generate Theorems [71.46963489866596]
  We learn a neural generator that automatically synthesizes theorems and proofs for the purpose of training a theorem prover. Experiments on real-world tasks demonstrate that synthetic data from our approach improves the theorem prover.
  arXiv  Detail & Related papers  (2020-02-17T16:06:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.