Convex Compositional Reasoning Models

• URL: http://arxiv.org/abs/2605.23395v2
• Date: Mon, 25 May 2026 08:06:16 GMT
• Title: Convex Compositional Reasoning Models
• Abstract summary: We show that compositional energy-based models can generalize to larger reasoning problems by a learned factor energy across many local constraints.<n>Because convexity is preserved under the global summation, the global projected first-order optimization is possible.
• Score: 8.508815294591221
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Compositional energy-based models can generalize to larger combinatorial reasoning problems by reusing a learned factor energy across many local constraints. In our paper, we show that a key bottleneck in compositional reasoning is not composition itself, but the non-convex geometry of the learned energy landscape. To solve this problem, we introduce Convex Compositional Energy Minimization (CCEM), a framework that parameterizes each factor with an input-convex neural network and optimizes the composed energy over a tight convex relaxation of the feasible set. Because convexity is preserved under summation, the global relaxed objective remains convex, enabling deterministic projected first-order optimization. CCEM is trained in two stages: factor-level contrastive learning to shape local energy basins, followed by end-to-end refinement through an unrolled projected solver. Our experiments show that our models trained on small subproblems or a single problem size transfer to larger instances without retraining.

Related papers

• Energy-Structured Low-Rank Adaptation for Continual Learning [78.5390284258307]
  We propose textbfEnergy-Concentrated and textbfEnergy-Ordered textbfLow-textbfRank textbfAdaptation (E$2$-LoRA)<n>By explicitly ordering and concentrating knowledge into leading ranks, E$2$-LoRA frees capacity for subsequent tasks.
  arXiv  Detail & Related papers  (2026-05-26T12:55:02Z)
• Diagonalizing the Softmax: Hadamard Initialization for Tractable Cross-Entropy Dynamics [29.85277126753054]
  Cross-entropy (CE) loss dominates deep learning, yet existing theory often relies on simplifications.<n>We provide an in-depth characterization of a canonical network with standard neural-basis vectors.
  arXiv  Detail & Related papers  (2025-12-03T17:45:09Z)
• A Saddle Point Remedy: Power of Variable Elimination in Non-convex Optimization [37.51825281790747]
  The proliferation of saddle points, rather than poor local minima, is an obstacle in large-scale non- optimization for machine learning.<n>We show that variable elimination fundamentally reshapes critical maximassian in the reduced landscape.
  arXiv  Detail & Related papers  (2025-11-03T05:19:43Z)
• Generalizable Reasoning through Compositional Energy Minimization [91.76056742068813]
  Generalization is a key challenge in machine learning, specifically in reasoning tasks.<n>We propose a novel approach to reasoning generalization by learning energy landscapes over the solution spaces of smaller, more tractable subproblems.<n>Our method outperforms existing state-of-the-art methods, demonstrating its ability to generalize to larger and more complex problems.
  arXiv  Detail & Related papers  (2025-10-23T14:38:36Z)
• MAP Image Recovery with Guarantees using Locally Convex Multi-Scale Energy (LC-MUSE) Model [12.218356507147583]
  We propose a multi-scale deep energy model that is strongly convex in the local neighbourhood around the data manifold.<n>We use the learned energy model in image-based inverse problems, where the formulation offers several desirable properties.<n>In the context of parallel Magnetic Resonance (MR) image reconstruction, we show that the proposed method performs better than the state-of-the-art convex regularizers.
  arXiv  Detail & Related papers  (2025-02-05T16:00:55Z)
• Stochastic Compositional Minimax Optimization with Provable Convergence Guarantees [14.301500851291989]
  compositional minimax problems are in machine learning, yet there are only limited established on the convergence of this class of problems. In this paper we propose a formal definition of the minimax problem which involves optimizing a minimax loss with a compositional structure.
  arXiv  Detail & Related papers  (2024-08-22T16:00:31Z)
• Compositional Diffusion-Based Continuous Constraint Solvers [98.1702285470628]
  This paper introduces an approach for learning to solve continuous constraint satisfaction problems (CCSP) in robotic reasoning and planning. By contrast, our model, the compositional diffusion continuous constraint solver (Diffusion-CCSP), derives global solutions to CCSPs.
  arXiv  Detail & Related papers  (2023-09-02T15:20:36Z)
• Sample Complexity for Quadratic Bandits: Hessian Dependent Bounds and Optimal Algorithms [64.10576998630981]
  We show the first tight characterization of the optimal Hessian-dependent sample complexity. A Hessian-independent algorithm universally achieves the optimal sample complexities for all Hessian instances. The optimal sample complexities achieved by our algorithm remain valid for heavy-tailed noise distributions.
  arXiv  Detail & Related papers  (2023-06-21T17:03:22Z)
• The star-shaped space of solutions of the spherical negative perceptron [4.511197686627054]
  We show that low-energy configurations are often found in complex connected structures. We identify a subset of atypical high-margin connected with most other solutions.
  arXiv  Detail & Related papers  (2023-05-18T00:21:04Z)
• D4FT: A Deep Learning Approach to Kohn-Sham Density Functional Theory [79.50644650795012]
  We propose a deep learning approach to solve Kohn-Sham Density Functional Theory (KS-DFT) We prove that such an approach has the same expressivity as the SCF method, yet reduces the computational complexity. In addition, we show that our approach enables us to explore more complex neural-based wave functions.
  arXiv  Detail & Related papers  (2023-03-01T10:38:10Z)
• Optimization Induced Equilibrium Networks [76.05825996887573]
  Implicit equilibrium models, i.e., deep neural networks (DNNs) defined by implicit equations, have been becoming more and more attractive recently. We show that deep OptEq outperforms previous implicit models even with fewer parameters.
  arXiv  Detail & Related papers  (2021-05-27T15:17:41Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.