Symmetric Behavior Regularization via Taylor Expansion of Symmetry

• URL: http://arxiv.org/abs/2508.04225v2
• Date: Thu, 07 Aug 2025 02:09:06 GMT
• Title: Symmetric Behavior Regularization via Taylor Expansion of Symmetry
• Authors: Lingwei Zhu, Zheng Chen, Han Wang, Yukie Nagai,
• Abstract summary: We show that symmetric divergences do not permit an analytic policy as regularization and can incur numerical issues as loss.<n>We propose Symmetric $f$ Actor-Critic (S$f$-AC), the first practical BRPO algorithm with symmetric divergences.
• Score: 8.032060509915821
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: This paper introduces symmetric divergences to behavior regularization policy optimization (BRPO) to establish a novel offline RL framework. Existing methods focus on asymmetric divergences such as KL to obtain analytic regularized policies and a practical minimization objective. We show that symmetric divergences do not permit an analytic policy as regularization and can incur numerical issues as loss. We tackle these challenges by the Taylor series of $f$-divergence. Specifically, we prove that an analytic policy can be obtained with a finite series. For loss, we observe that symmetric divergences can be decomposed into an asymmetry and a conditional symmetry term, Taylor-expanding the latter alleviates numerical issues. Summing together, we propose Symmetric $f$ Actor-Critic (S$f$-AC), the first practical BRPO algorithm with symmetric divergences. Experimental results on distribution approximation and MuJoCo verify that S$f$-AC performs competitively.

Related papers

• Joint Asymmetric Loss for Learning with Noisy Labels [95.14298444251044]
  symmetric losses usually suffer from the underfitting issue due to the overly strict constraint.<n>Within APL, symmetric losses have been successfully extended, yielding advanced robust loss functions.<n>We introduce a novel robust loss framework termed Joint Asymmetric Loss (JAL)
  arXiv  Detail & Related papers  (2025-07-23T16:57:43Z)
• Entropic Mirror Descent for Linear Systems: Polyak's Stepsize and Implicit Bias [55.72269695392027]
  This paper focuses on applying entropic mirror descent to solve linear systems.<n>The main challenge for the convergence analysis stems from the unboundedness of the domain.<n>To overcome this without imposing restrictive assumptions, we introduce a variant of Polyak-type stepsizes.
  arXiv  Detail & Related papers  (2025-05-05T12:33:18Z)
• Predicting symmetries of quantum dynamics with optimal samples [41.42817348756889]
  Identifying symmetries in quantum dynamics is a crucial challenge with profound implications for quantum technologies.<n>We introduce a unified framework combining group representation theory and subgroup hypothesis testing to predict these symmetries with optimal efficiency.<n>We prove that parallel strategies achieve the same performance as adaptive or indefinite-causal-order protocols.
  arXiv  Detail & Related papers  (2025-02-03T15:57:50Z)
• Variational Inference Failures Under Model Symmetries: Permutation Invariant Posteriors for Bayesian Neural Networks [43.88179780450706]
  We investigate the impact of weight space permutation symmetries on variational inference. We devise a symmetric symmetrization mechanism for constructing permutation invariant variational posteriors. We show that the symmetrized distribution has a strictly better fit to the true posterior, and that it can be trained using the original ELBO objective.
  arXiv  Detail & Related papers  (2024-08-10T09:06:34Z)
• Multivariate root-n-consistent smoothing parameter free matching estimators and estimators of inverse density weighted expectations [51.000851088730684]
  We develop novel modifications of nearest-neighbor and matching estimators which converge at the parametric $sqrt n $-rate.<n>We stress that our estimators do not involve nonparametric function estimators and in particular do not rely on sample-size dependent parameters smoothing.
  arXiv  Detail & Related papers  (2024-07-11T13:28:34Z)
• Global optimality under amenable symmetry constraints [0.5656581242851759]
  We show the interplay between convexity, the group, and the underlying vector space, which is typically infinite-dimensional. We apply this toolkit to the invariant optimality problem. It yields new results on invariant kernel mean embeddings and risk-optimal invariant couplings.
  arXiv  Detail & Related papers  (2024-02-12T12:38:20Z)
• Equivariant Symmetry Breaking Sets [0.6475999521931204]
  Equivariant neural networks (ENNs) have been shown to be extremely effective in applications involving underlying symmetries. We propose a novel symmetry breaking framework that is fully equivariant and is the first which fully addresses spontaneous symmetry breaking.
  arXiv  Detail & Related papers  (2024-02-05T02:35:11Z)
• Asymmetric Norms to Approximate the Minimum Action Distance [9.040428950629153]
  This paper presents a state representation for reward-free Markov decision processes. We show how this representation can be leveraged to learn goal-conditioned policies.
  arXiv  Detail & Related papers  (2023-12-16T00:50:17Z)
• Symmetry-resolved Entanglement Entropy, Spectra & Boundary Conformal Field Theory [0.0]
  We perform a comprehensive analysis of the symmetry-resolved entanglement entropy (EE) for one single interval in the ground state of a $1+1$D conformal field theory (CFT) We utilize the boundary CFT approach to study the total EE, which enables us to find the universal leading order behavior of the SREE. We derive the symmetry-resolved entanglement spectra for a CFT invariant under a finite symmetry group.
  arXiv  Detail & Related papers  (2023-09-06T18:03:14Z)
• Kernel-based off-policy estimation without overlap: Instance optimality beyond semiparametric efficiency [53.90687548731265]
  We study optimal procedures for estimating a linear functional based on observational data. For any convex and symmetric function class $mathcalF$, we derive a non-asymptotic local minimax bound on the mean-squared error.
  arXiv  Detail & Related papers  (2023-01-16T02:57:37Z)
• When Random Tensors meet Random Matrices [50.568841545067144]
  This paper studies asymmetric order-$d$ spiked tensor models with Gaussian noise. We show that the analysis of the considered model boils down to the analysis of an equivalent spiked symmetric textitblock-wise random matrix.
  arXiv  Detail & Related papers  (2021-12-23T04:05:01Z)
• Symmetry Breaking in Symmetric Tensor Decomposition [44.181747424363245]
  We consider the nonsymmetry problem associated with computing the points rank decomposition of symmetric tensors. We show that critical points the loss function is detected by standard methods.
  arXiv  Detail & Related papers  (2021-03-10T18:11:22Z)
• On Linear Stochastic Approximation: Fine-grained Polyak-Ruppert and Non-Asymptotic Concentration [115.1954841020189]
  We study the inequality and non-asymptotic properties of approximation procedures with Polyak-Ruppert averaging. We prove a central limit theorem (CLT) for the averaged iterates with fixed step size and number of iterations going to infinity.
  arXiv  Detail & Related papers  (2020-04-09T17:54:18Z)
• The quantum marginal problem for symmetric states: applications to variational optimization, nonlocality and self-testing [0.0]
  We present a method to solve the quantum marginal problem for symmetric $d$-level systems. We illustrate the applicability of the method in central quantum information problems with several exemplary case studies.
  arXiv  Detail & Related papers  (2020-01-13T18:20:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.