Sparse Representations, Inference and Learning

• URL: http://arxiv.org/abs/2306.16097v1
• Date: Wed, 28 Jun 2023 10:58:27 GMT
• Title: Sparse Representations, Inference and Learning
• Authors: Clarissa Lauditi, Emanuele Troiani and Marc M\'ezard
• Abstract summary: We will present a general framework that can be used in a large variety of problems with weak long-range interactions. We shall see how these problems can be studied at the replica symmetric level, using developments of the cavity methods.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In recent years statistical physics has proven to be a valuable tool to probe into large dimensional inference problems such as the ones occurring in machine learning. Statistical physics provides analytical tools to study fundamental limitations in their solutions and proposes algorithms to solve individual instances. In these notes, based on the lectures by Marc M\'ezard in 2022 at the summer school in Les Houches, we will present a general framework that can be used in a large variety of problems with weak long-range interactions, including the compressed sensing problem, or the problem of learning in a perceptron. We shall see how these problems can be studied at the replica symmetric level, using developments of the cavity methods, both as a theoretical tool and as an algorithm.

Related papers

• Physics and Deep Learning in Computational Wave Imaging [24.99422165859396]
  Computational wave imaging (CWI) extracts hidden structure and physical properties of a volume of material. Current approaches for solving CWI problems can be divided into categories: those rooted in traditional physics, and those based on deep learning. Machine learning-based computational methods have emerged, offering a different perspective to address these challenges.
  arXiv  Detail & Related papers  (2024-10-10T19:32:17Z)
• Multi-qubit state visualizations to support problem solving $-$ a pilot study [1.8879980022743639]
  We compare students' performance, time taken, and cognitive load when solving problems using the mathematical-symbolic Dirac notation alone with using it accompanied by the circle notation or the dimensional circle notation in single- and multi-qubit systems. Although little overall differences in students' performance can be detected depending on the presented representations, we observe that problem-solving performance is student- and context-dependent.
  arXiv  Detail & Related papers  (2024-06-24T11:46:35Z)
• G-LLaVA: Solving Geometric Problem with Multi-Modal Large Language Model [124.68242155098189]
  Large language models (LLMs) have shown remarkable proficiency in human-level reasoning and generation capabilities. G-LLaVA demonstrates exceptional performance in solving geometric problems, significantly outperforming GPT-4-V on the MathVista benchmark with only 7B parameters.
  arXiv  Detail & Related papers  (2023-12-18T17:36:20Z)
• Models and algorithms for simple disjunctive temporal problems [0.8793721044482611]
  We focus on the case where events may have an arbitrarily large number of release and due dates. We provide three mathematical models to describe this problem using constraint programming and linear programming. We implement algorithms from the literature and provide the first in-depth empirical study comparing methods to solve simple disjunctive temporal problems.
  arXiv  Detail & Related papers  (2023-02-06T09:40:24Z)
• Features for the 0-1 knapsack problem based on inclusionwise maximal solutions [0.7734726150561086]
  We formulate several new computationally challenging problems related to the IMSs of arbitrary 0-1 knapsack problem instances. We derive a set of 14 computationally expensive features, which we calculate for two large datasets on a supercomputer in approximately 540 CPU-hours. We show that the proposed features contain important information related to the empirical hardness of a problem instance.
  arXiv  Detail & Related papers  (2022-11-16T12:48:35Z)
• Continuous-time Analysis for Variational Inequalities: An Overview and Desiderata [87.77379512999818]
  We provide an overview of recent progress in the use of continuous-time perspectives in the analysis and design of methods targeting the broad VI problem class. Our presentation draws parallels between single-objective problems and multi-objective problems, highlighting the challenges of the latter. We also formulate various desiderata for algorithms that apply to general VIs and we argue that achieving these desiderata may profit from an understanding of the associated continuous-time dynamics.
  arXiv  Detail & Related papers  (2022-07-14T17:58:02Z)
• Provable Reinforcement Learning with a Short-Term Memory [68.00677878812908]
  We study a new subclass of POMDPs, whose latent states can be decoded by the most recent history of a short length $m$. In particular, in the rich-observation setting, we develop new algorithms using a novel "moment matching" approach with a sample complexity that scales exponentially. Our results show that a short-term memory suffices for reinforcement learning in these environments.
  arXiv  Detail & Related papers  (2022-02-08T16:39:57Z)
• Unsupervised Statistical Learning for Die Analysis in Ancient Numismatics [1.1470070927586016]
  We propose a model for unsupervised computational die analysis, which can reduce the time investment necessary for large-scale die studies by several orders of magnitude. The efficacy of our method is demonstrated through an analysis of 1135 Roman silver coins struck between 64-66 C.E.
  arXiv  Detail & Related papers  (2021-12-01T06:02:07Z)
• Distributed Methods with Compressed Communication for Solving Variational Inequalities, with Theoretical Guarantees [115.08148491584997]
  We present the first theoretically grounded distributed methods for solving variational inequalities and saddle point problems using compressed communication: MASHA1 and MASHA2. New algorithms support bidirectional compressions, and also can be modified for setting with batches and for federated learning with partial participation of clients.
  arXiv  Detail & Related papers  (2021-10-07T10:04:32Z)
• Decentralized Personalized Federated Learning for Min-Max Problems [79.61785798152529]
  This paper is the first to study PFL for saddle point problems encompassing a broader range of optimization problems. We propose new algorithms to address this problem and provide a theoretical analysis of the smooth (strongly) convex-(strongly) concave saddle point problems. Numerical experiments for bilinear problems and neural networks with adversarial noise demonstrate the effectiveness of the proposed methods.
  arXiv  Detail & Related papers  (2021-06-14T10:36:25Z)
• Heterogeneous Representation Learning: A Review [66.12816399765296]
  Heterogeneous Representation Learning (HRL) brings some unique challenges. We present a unified learning framework which is able to model most existing learning settings with the heterogeneous inputs. We highlight the challenges that are less-touched in HRL and present future research directions.
  arXiv  Detail & Related papers  (2020-04-28T05:12:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.