Advancing Optimal Subset Oracle via Learning Relaxation of Neural Set Functions

• URL: http://arxiv.org/abs/2607.11555v1
• Date: Mon, 13 Jul 2026 13:37:47 GMT
• Title: Advancing Optimal Subset Oracle via Learning Relaxation of Neural Set Functions
• Abstract summary: Learning neural set functions is pivotal to a wide range of important applications, including compound selection in AI-driven drug discovery and product recommendation.<n>Recent work has introduced optimal subsets to implicitly learn set functions under practical weakly supervised settings.<n>These frameworks rely on Monte Carlo sampling to estimate gradients of the evidence lower bound when updating the variational distribution.<n>In this work, we reinterpret the evidence lower bound as a continuous relaxation of the set function and learn a surrogate objective that replaces sampling-based ELBO estimation during variational optimization.
• Score: 41.373161204333435
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Learning neural set functions is pivotal to a wide range of important applications, including compound selection in AI-driven drug discovery and product recommendation. Recent work has introduced optimal subset oracles to implicitly learn set functions under practical weakly supervised settings, where model parameters are optimized through mean-field variational inference. However, these frameworks rely on Monte Carlo sampling to estimate gradients of the evidence lower bound when updating the variational distribution. Repeated sampling across iterations incurs substantial computational overhead, while the resulting stochasticity can destabilize the optimization trajectory. In this work, we reinterpret the evidence lower bound as a continuous relaxation of the set function and learn a surrogate objective that replaces sampling-based ELBO gradient estimation during variational optimization. The learned surrogate provides stable and efficient gradients throughout the continuous domain, thereby reducing computational overhead and accelerating inference. Furthermore, we establish an approximation guarantee for the proposed framework under submodular maximization and characterize its connection to variational free energy. Experiments on a variety of real-world tasks demonstrate consistent improvements over existing baselines.

Related papers

• Learning Perturbations to Extrapolate Your LLM [12.34345212532974]
  We propose a framework where token prefixes are perturbed by a learnable transformation of a continuous latent vector within an embedding space.<n>To overcome the challenge of an intractable marginal likelihood, we derive estimating equations for model parameters and optimize them via gradient descent.
  arXiv  Detail & Related papers  (2026-05-13T10:00:58Z)
• Gradient-Free Noise Optimization for Reward Alignment in Generative Models [52.42087778653147]
  ZeNO is a gradient-free framework that formulates noise optimization as a path-integral control problem.<n>It enables effective inference-time scaling and demonstrates strong performance across diverse generators and reward functions.
  arXiv  Detail & Related papers  (2026-05-12T00:05:36Z)
• Fine-tuning Factor Augmented Neural Lasso for Heterogeneous Environments [9.523993463071834]
  This paper introduces the fine-tuning factor augmented neural Lasso (FAN-Lasso)<n>FAN-Lasso is a transfer learning framework for high-dimensional nonparametric regression with variable selection.<n>It consistently outperforms standard baselines and achieves near-oracle performance even under severe target sample size constraints.
  arXiv  Detail & Related papers  (2026-04-14T05:01:18Z)
• Natural Hypergradient Descent: Algorithm Design, Convergence Analysis, and Parallel Implementation [5.754044493040163]
  Natural Hypergradient Descent (NHGD) is a new method for solving bilevel optimization problems.<n>Our main theoretical contribution establishes high-probability error bounds and sample complexity guarantees for NHGD.<n> Empirical evaluations on representative bilevel learning tasks demonstrate the practical advantages of NHGD.
  arXiv  Detail & Related papers  (2026-02-11T14:31:33Z)
• ODELoRA: Training Low-Rank Adaptation by Solving Ordinary Differential Equations [54.886931928255564]
  Low-rank adaptation (LoRA) has emerged as a widely adopted parameter-efficient fine-tuning method in deep transfer learning.<n>We propose a novel continuous-time optimization dynamic for LoRA factor matrices in the form of an ordinary differential equation (ODE)<n>We show that ODELoRA achieves stable feature learning, a property that is crucial for training deep neural networks at different scales of problem dimensionality.
  arXiv  Detail & Related papers  (2026-02-07T10:19:36Z)
• Not All Preferences Are Created Equal: Stability-Aware and Gradient-Efficient Alignment for Reasoning Models [52.48582333951919]
  We propose a dynamic framework designed to enhance alignment reliability by maximizing the Signal-to-Noise Ratio of policy updates.<n>SAGE (Stability-Aware Gradient Efficiency) integrates a coarse-grained curriculum mechanism that refreshes candidate pools based on model competence.<n> Experiments on multiple mathematical reasoning benchmarks demonstrate that SAGE significantly accelerates convergence and outperforms static baselines.
  arXiv  Detail & Related papers  (2026-02-01T12:56:10Z)
• Efficient Group Lasso Regularized Rank Regression with Data-Driven Parameter Determination [2.847099287022546]
  High-dimensional regression often suffers from heavy-tailed noise and outliers, which can severely undermine the reliability of least-squares based methods.<n>To improve robustness, we adopt a non-smooth Wilcoxon score based rank objective and incorporate structured group sparsity regularization.<n>We also introduce a data-driven, simulation-based tuning rule and further establish a finite-sample error bound for the resulting estimator.
  arXiv  Detail & Related papers  (2025-10-13T15:45:58Z)
• Stochastic Optimization with Optimal Importance Sampling [49.484190237840714]
  We propose an iterative-based algorithm that jointly updates the decision and the IS distribution without requiring time-scale separation between the two.<n>Our method achieves the lowest possible variable variance and guarantees global convergence under convexity of the objective and mild assumptions on the IS distribution family.
  arXiv  Detail & Related papers  (2025-04-04T16:10:18Z)
• Momentum Accelerates the Convergence of Stochastic AUPRC Maximization [80.8226518642952]
  We study optimization of areas under precision-recall curves (AUPRC), which is widely used for imbalanced tasks. We develop novel momentum methods with a better iteration of $O (1/epsilon4)$ for finding an $epsilon$stationary solution. We also design a novel family of adaptive methods with the same complexity of $O (1/epsilon4)$, which enjoy faster convergence in practice.
  arXiv  Detail & Related papers  (2021-07-02T16:21:52Z)
• Real-Time Optimization Meets Bayesian Optimization and Derivative-Free Optimization: A Tale of Modifier Adaptation [0.0]
  This paper investigates a new class of modifier-adaptation schemes to overcome plant-model mismatch in real-time optimization of uncertain processes. The proposed schemes embed a physical model and rely on trust-region ideas to minimize risk during the exploration. The benefits of using an acquisition function, knowing the process noise level, or specifying a nominal process model are illustrated.
  arXiv  Detail & Related papers  (2020-09-18T12:57:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.