On Maximum Entropy Linear Feature Inversion

• URL: http://arxiv.org/abs/2407.14166v1
• Date: Fri, 19 Jul 2024 09:52:18 GMT
• Title: On Maximum Entropy Linear Feature Inversion
• Authors: Paul M Baggenstoss,
• Abstract summary: We revisit the classical problem of inverting dimension-reducing linear mappings using the maximum entropy criterion. We propose a new unified approach that not only specializes to the existing approaches, but offers solutions to new cases.
• Score: 7.1795069620810805
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We revisit the classical problem of inverting dimension-reducing linear mappings using the maximum entropy (MaxEnt) criterion. In the literature, solutions are problem-dependent, inconsistent, and use different entropy measures. We propose a new unified approach that not only specializes to the existing approaches, but offers solutions to new cases, such as when data values are constrained to [0, 1], which has new applications in machine learning.

Related papers

• Non-convex entropic mean-field optimization via Best Response flow [0.0]
  We discuss the problem of minimizing non- functionals on the space probability measures, regularized by the relative entropy (KL) with respect to a fixed reference measure.<n>We show how to choose the regularizer, given the non functional, so that the Best Response becomes a contraction with respect to the $L1$Wasserstein distance.
  arXiv  Detail & Related papers  (2025-05-28T18:22:08Z)
• Reduced-Space Iteratively Reweighted Second-Order Methods for Nonconvex Sparse Regularization [11.56128809794923]
  This paper explores a specific type of non sparsity-promoting regularization problems, namely those involving $ell_p-$ iterations of local property convergence.
  arXiv  Detail & Related papers  (2024-07-24T12:15:59Z)
• Randomized algorithms and PAC bounds for inverse reinforcement learning in continuous spaces [47.907236421762626]
  This work studies discrete-time discounted Markov decision processes with continuous state and action spaces. We first consider the case in which we have access to the entire expert policy and characterize the set of solutions to the inverse problem.
  arXiv  Detail & Related papers  (2024-05-24T12:53:07Z)
• Feature selection in linear SVMs via a hard cardinality constraint: a scalable SDP decomposition approach [3.7876216422538485]
  We study the embedded feature selection problem in linear Support Vector Machines (SVMs) in which a cardinality constraint is employed. The problem is NP-hard due to the presence of the cardinality constraint, even though the original linear SVM amounts to a solvable problem in time. To handle the hard problem, we first introduce two mixed-integer formulations for which novel semidefinite relaxations are proposed.
  arXiv  Detail & Related papers  (2024-04-15T19:15:32Z)
• ODE Discovery for Longitudinal Heterogeneous Treatment Effects Inference [69.24516189971929]
  In this paper, we introduce a new type of solution in the longitudinal setting: a closed-form ordinary differential equation (ODE) While we still rely on continuous optimization to learn an ODE, the resulting inference machine is no longer a neural network.
  arXiv  Detail & Related papers  (2024-03-16T02:07:45Z)
• OKRidge: Scalable Optimal k-Sparse Ridge Regression [21.17964202317435]
  We propose a fast algorithm, OKRidge, for sparse ridge regression. We also propose a method to warm-start our solver, which leverages a beam search.
  arXiv  Detail & Related papers  (2023-04-13T17:34:44Z)
• Learning Proximal Operators to Discover Multiple Optima [66.98045013486794]
  We present an end-to-end method to learn the proximal operator across non-family problems. We show that for weakly-ized objectives and under mild conditions, the method converges globally.
  arXiv  Detail & Related papers  (2022-01-28T05:53:28Z)
• Simplifying deflation for non-convex optimization with applications in Bayesian inference and topology optimization [0.0]
  Non-local optimization problems are commonly found in applications. One of the methods recently explore multiple optimal minimum constraint limitations to local applications. The proposed methodology in the approximate inference is presented.
  arXiv  Detail & Related papers  (2022-01-28T04:20:07Z)
• Sequential- and Parallel- Constrained Max-value Entropy Search via Information Lower Bound [9.09466320810472]
  We focus on an information-theoretic approach called Max-value Entropy Search (MES) We propose a novel constrained BO method called Constrained Max-value Entropy Search via Information lower BOund (CMES-IBO)
  arXiv  Detail & Related papers  (2021-02-19T08:10:51Z)
• Dimension Free Generalization Bounds for Non Linear Metric Learning [61.193693608166114]
  We provide uniform generalization bounds for two regimes -- the sparse regime, and a non-sparse regime. We show that by relying on a different, new property of the solutions, it is still possible to provide dimension free generalization guarantees.
  arXiv  Detail & Related papers  (2021-02-07T14:47:00Z)
• Efficient Methods for Structured Nonconvex-Nonconcave Min-Max Optimization [98.0595480384208]
  We propose a generalization extraient spaces which converges to a stationary point. The algorithm applies not only to general $p$-normed spaces, but also to general $p$-dimensional vector spaces.
  arXiv  Detail & Related papers  (2020-10-31T21:35:42Z)
• Conditional gradient methods for stochastically constrained convex minimization [54.53786593679331]
  We propose two novel conditional gradient-based methods for solving structured convex optimization problems. The most important feature of our framework is that only a subset of the constraints is processed at each iteration. Our algorithms rely on variance reduction and smoothing used in conjunction with conditional gradient steps, and are accompanied by rigorous convergence guarantees.
  arXiv  Detail & Related papers  (2020-07-07T21:26:35Z)
• Gradient-Free Methods for Saddle-Point Problem [125.99533416395765]
  We generalize the approach Gasnikov et. al., 2017, which allows to solve (stochastic) convex optimization problems with an inexact gradient-free oracle. Our approach reduces $fracnlog n$ times the required number of oracle calls. In the second part of the paper, we analyze the case when such an assumption cannot be made, we propose a general approach on how to modernize the method to solve this problem.
  arXiv  Detail & Related papers  (2020-05-12T16:44:27Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.