Nonsmooth Set-Gradient Ascent to the Pareto Front via Layered Hypervolume and Magnitude Indicators

• URL: http://arxiv.org/abs/2605.13468v1
• Date: Wed, 13 May 2026 12:55:22 GMT
• Title: Nonsmooth Set-Gradient Ascent to the Pareto Front via Layered Hypervolume and Magnitude Indicators
• Abstract summary: A nonsmooth set-gradient ascent method is developed for moving finite approximation sets toward the Pareto front.<n>Two base indicators are treated: the hypervolume indicator and the magnitude indicator of the dominated set.<n>For finite-$$ surrogates, the main nonsmoothness mechanisms are isolated and chamberwise Lipschitz continuity on bounded sets is proved.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: A nonsmooth set-gradient ascent method is developed for moving finite approximation sets toward the Pareto front in multiobjective optimization. The method optimizes layered set indicators: a base indicator is evaluated on successive nondomination layers, and the layer values are combined with rapidly decreasing weights. This gives ascent directions to nondominated and dominated points while preventing deeper layers from compensating for deterioration of the first front. Two base indicators are treated: the hypervolume indicator and the magnitude indicator of the dominated set, whose expansion over coordinate projections contains extent, projected-area, and volume terms. The scalar objectives are nonsmooth because nondomination layers change combinatorially and the active orthogonal-union geometry changes piecewise. On fixed strata, where layer assignments and active geometry remain unchanged, the indicators are piecewise smooth and chamberwise continuous. For the magnitude indicator, an exact gradient formula is derived as a linear combination of hypervolume gradients of projected shadow sets. Thus, for fixed objective dimension, magnitude gradients have the same asymptotic time complexity as hypervolume gradients. Lexicographic layer aggregation is related to a unary infinitesimal encoding. For finite-$ε$ surrogates, the main nonsmoothness mechanisms are isolated and chamberwise Lipschitz continuity on bounded sets is proved; a two-point counterexample shows that hard-layer scalarization is not globally continuous across layer switches. The theory motivates a projected finite-difference implementation with repulsion and recovery from stagnation. Numerical examples and reproducible code cover two- and three-objective settings, including objective-space tests, curved fronts, a supersphere benchmark, and traces comparing layered magnitude and hypervolume ascent.

Related papers

• Across the Loss Landscape with Progressive Growth [51.366966420881646]
  We show how grow-and-optimize strategies to train toward flatter regions by viewing growth as progressive relaxation.<n>We empirically validate these predictions in controlled toy basins and in a realistic ResNet/CI-100 setting.<n>Although progressive subspace growth reliably produces flatter solutions, curvature reductions do not universally translate into improved test performance, highlighting subtleties in the flatness-generalization connection.
  arXiv  Detail & Related papers  (2026-08-25T13:50:48Z)
• Hierarchical Grading in Large Language Models [0.0]
  We introduce Graded Large Language Models (GLLMs)<n>GLLMs are a framework that equips the representation space of a transformer with a grading.<n>The construction extends the theory of graded neural networks and graded transformers to autoregressive language models.
  arXiv  Detail & Related papers  (2026-07-23T20:14:48Z)
• Stability Annealing Selects the Implicit Bias of Smoothed Sign Descent: A Rate-Indexed Barrier Path on Separable Data [3.7437735326653883]
  This paper studies the rate-controlled middle case for full-batch linear classification on separable data.<n>For memoryless stability-annealed smoothed-sign descent with weighted exponential loss, we prove that the normalized iterates converge to the minimizer of a convex Burg-type barrier over a margin slice.
  arXiv  Detail & Related papers  (2026-07-07T08:55:35Z)
• When Both Layers Learn: Training Dynamics of Representing Linear Models via ReLU Networks [22.81761236732655]
  We study the gradient descent dynamics for jointly training both layers of a one-hidden-layer ReLU network to fit a linear target function.<n>Our analysis tracks the trajectory through three phases: an alignment phase in which hidden weights progressively align with the planted direction while the output weights maintain the correct sign pattern.<n>We establish novel uniform concentration results that hold along the entire trajectory, and are essential for obtaining order-wise optimal sample complexity.
  arXiv  Detail & Related papers  (2026-06-03T05:44:30Z)
• The Geometric Wall: Manifold Structure Predicts Layerwise Sparse Autoencoder Scaling Laws [13.054357482525505]
  We conduct the first cross-layer SAE scaling study, fitting and regressing on 68 layers of Gemma 2 2B and 9B.<n>We find that manifold geometry predicts the per-layer width exponent in both models.
  arXiv  Detail & Related papers  (2026-05-11T02:24:46Z)
• Flatness and Gradient Alignment Are Both Necessary: Spectral-Aware Gradient-Aligned Exploration for Multi-Distribution Learning [7.794885131732119]
  We show that flatness and gradient alignment should be considered in multi-distribution learning settings.<n>We propose SAGE (Spectral-Aware Gradient-Aligned Exploration) that targets both terms.<n> Experiments on five domain-generalization and two multi-task learning benchmarks show that the proposed method establishes a new state-of-the-art.
  arXiv  Detail & Related papers  (2026-05-08T15:52:34Z)
• The Magnitude of Dominated Sets: A Pareto Compliant Indicator Grounded in Metric Geometry [0.0]
  Magnitude is a notion of size or point content for compact metric spaces.<n>For dominated regions in the (ell_1) box setting, magnitude is close to hypervolume.<n>Unlike hypervolume, magnitude assigns positive value to boundary points sharing one or more coordinates with the anchor point.
  arXiv  Detail & Related papers  (2026-04-20T12:10:12Z)
• AEGIS: Anchor-Enforced Gradient Isolation for Knowledge-Preserving Vision-Language-Action Fine-Tuning [0.0]
  Adapting pre-trained vision-language models (VLMs) for robotic control requires injecting high-magnitude continuous gradients from a flow-matching action expert into a backbone trained exclusively with cross-entropy.<n>This cross-modal gradient asymmetry causes rapid, severe erosion of the VLM's visual-question-answering (VQA) capability.<n>Industry-standard defences either sever the gradient pathway entirely via stop gradient, discarding the rich continuous supervision, or restrict parameter capacity through low-rank adapters (LoRA) that constrain the rank of updates but not their direction, and thus still overwrite the pre-trained manifold
  arXiv  Detail & Related papers  (2026-04-17T13:49:57Z)
• The Devil Is in Gradient Entanglement: Energy-Aware Gradient Coordinator for Robust Generalized Category Discovery [76.35318747478522]
  Energy-Aware Gradient Coordinator (EAGC) is a plug-and-play gradient module that explicitly regulates the optimization process.<n>EAGC comprises two components: Anchor-based Gradient Alignment (AGA) and Energy-aware Elastic Projection (EEP)
  arXiv  Detail & Related papers  (2026-03-25T14:58:42Z)
• SPINAL -- Scaling-law and Preference Integration in Neural Alignment Layers [16.976750197698063]
  We introduce SPINAL, a diagnostic that measures how alignment reshapes representations across depth.<n>Across model families, DPO produces a layerwise calibration effect concentrated in the final decoder blocks.<n>Aligned checkpoints show a late-layer ramp-up in contraction and a smooth reduction in transport, consistent with tightened and stabilized policy mass.
  arXiv  Detail & Related papers  (2026-01-08T17:47:12Z)
• Implicit Bias in Leaky ReLU Networks Trained on High-Dimensional Data [63.34506218832164]
  In this work, we investigate the implicit bias of gradient flow and gradient descent in two-layer fully-connected neural networks with ReLU activations. For gradient flow, we leverage recent work on the implicit bias for homogeneous neural networks to show that leakyally, gradient flow produces a neural network with rank at most two. For gradient descent, provided the random variance is small enough, we show that a single step of gradient descent suffices to drastically reduce the rank of the network, and that the rank remains small throughout training.
  arXiv  Detail & Related papers  (2022-10-13T15:09:54Z)
• On optimization of coherent and incoherent controls for two-level quantum systems [77.34726150561087]
  This article considers some control problems for closed and open two-level quantum systems. The closed system's dynamics is governed by the Schr"odinger equation with coherent control. The open system's dynamics is governed by the Gorini-Kossakowski-Sudarshan-Lindblad master equation.
  arXiv  Detail & Related papers  (2022-05-05T09:08:03Z)
• Deriving Differential Target Propagation from Iterating Approximate Inverses [91.3755431537592]
  We show that a particular form of target propagation, relying on learned inverses of each layer, which is differential, gives rise to an update rule which corresponds to an approximate Gauss-Newton gradient-based optimization. We consider several iterative calculations based on local auto-encoders at each layer in order to achieve more precise inversions for more accurate target propagation.
  arXiv  Detail & Related papers  (2020-07-29T22:34:45Z)
• Cogradient Descent for Bilinear Optimization [124.45816011848096]
  We introduce a Cogradient Descent algorithm (CoGD) to address the bilinear problem. We solve one variable by considering its coupling relationship with the other, leading to a synchronous gradient descent. Our algorithm is applied to solve problems with one variable under the sparsity constraint.
  arXiv  Detail & Related papers  (2020-06-16T13:41:54Z)
• Analytic Signal Phase in $N-D$ by Linear Symmetry Tensor--fingerprint modeling [69.35569554213679]
  We show that the Analytic Signal phase, and its gradient have a hitherto unstudied discontinuity in $2-D $ and higher dimensions. This shortcoming can result in severe artifacts whereas the problem does not exist in $1-D $ signals. We suggest the use of Linear Symmetry phase, relying on more than one set of Gabor filters, but with a negligible computational add-on.
  arXiv  Detail & Related papers  (2020-05-16T21:17:26Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.