Refined Gradient-Based Temperature Optimization for the Replica-Exchange Monte-Carlo Method

• URL: http://arxiv.org/abs/2601.13542v1
• Date: Tue, 20 Jan 2026 03:03:32 GMT
• Title: Refined Gradient-Based Temperature Optimization for the Replica-Exchange Monte-Carlo Method
• Authors: Tatsuya Miyata, Shunta Arai, Satoshi Takabe,
• Abstract summary: The replica-exchange Monte-Carlo (RXMC) method is a powerful Markov-chain Monte-Carlo algorithm for sampling from multi-modal distributions.<n>We propose a refined online temperature selection method by extending the gradient-based optimization framework.<n>Our results show that the method successfully achieves uniform acceptance rates and reduces round-trip times across the temperature space.
• Score: 1.5293427903448018
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The replica-exchange Monte-Carlo (RXMC) method is a powerful Markov-chain Monte-Carlo algorithm for sampling from multi-modal distributions, which are challenging for conventional methods. The sampling efficiency of the RXMC method depends highly on the selection of the temperatures, and finding optimal temperatures remains a challenge. In this study, we propose a refined online temperature selection method by extending the gradient-based optimization framework proposed previously. Building upon the existing temperature update approach, we introduce a reparameterization technique to strictly enforce physical constraints, such as the monotonic ordering of inverse temperatures, which were not explicitly addressed in the original formulation. The proposed method defines the variance of acceptance rates between adjacent replicas as a loss function, estimates its gradient using differential information from the sampling process, and optimizes the temperatures via gradient descent. We demonstrate the effectiveness of our method through experiments on benchmark spin systems, including the two-dimensional ferromagnetic Ising model, the two-dimensional ferromagnetic XY model, and the three-dimensional Edwards-Anderson model. Our results show that the method successfully achieves uniform acceptance rates and reduces round-trip times across the temperature space. Furthermore, our proposed method offers a significant advantage over recently proposed policy gradient method that require careful hyperparameter tuning, while simultaneously preventing the constraint violations that destabilize optimization.

Related papers

• HeatGen: A Guided Diffusion Framework for Multiphysics Heat Sink Design Optimization [0.0]
  Geometries are represented using boundary representations of multiple fins.<n>We train a denoising diffusion probabilistic model to generate heat sinks with characteristics consistent with those observed in the data.<n>We train two different residual neural networks to predict the pressure drop and surface temperature for each geometry.
  arXiv  Detail & Related papers  (2025-11-12T08:07:46Z)
• Low-Rank Curvature for Zeroth-Order Optimization in LLM Fine-Tuning [8.349781300731225]
  We introduce LOREN, a curvature-aware zeroth-order (ZO) optimization method for fine-tuning large language models (LLMs)<n>Existing ZO methods, which estimate gradients via finite differences using random perturbations, often suffer from high variance and suboptimal search directions.<n>Our approach addresses these challenges by: (i) adaptively estimating an anisotropic perturbation distribution for gradient estimation, (ii) capturing curvature through a low-rank block diagonal preconditioner, and (iii) applying a REINFORCE leave-one-out (RLOO) gradient estimator to reduce variance.
  arXiv  Detail & Related papers  (2025-11-11T08:34:09Z)
• Inference-Time Scaling of Discrete Diffusion Models via Importance Weighting and Optimal Proposal Design [17.7006862812979]
  We propose a Sequential Monte Carlo framework that enables scalable inference-time control of discrete diffusion models.<n>Specifically, our approach derives tractable importance weights for a range of intermediate targets and characterises the optimal proposal.<n> Empirical results across synthetic tasks, language modelling, biology design, and text-to-image generation demonstrate that our framework enhances controllability and sample quality.
  arXiv  Detail & Related papers  (2025-05-28T16:12:03Z)
• Feynman-Kac Correctors in Diffusion: Annealing, Guidance, and Product of Experts [64.34482582690927]
  We provide an efficient and principled method for sampling from a sequence of annealed, geometric-averaged, or product distributions derived from pretrained score-based models.<n>We propose Sequential Monte Carlo (SMC) resampling algorithms that leverage inference-time scaling to improve sampling quality.
  arXiv  Detail & Related papers  (2025-03-04T17:46:51Z)
• Monte Carlo Temperature: a robust sampling strategy for LLM's uncertainty quantification methods [1.3892342684177872]
  We propose a robust sampling strategy that eliminates the need for temperature calibration.<n>MCT provides more robust uncertainty estimates across a wide range of temperatures.<n>MCT achieves statistical parity with oracle temperatures, which represent the ideal outcome of a well-tuned but computationally expensive HPO process.
  arXiv  Detail & Related papers  (2025-02-25T17:33:20Z)
• Policy Gradients for Optimal Parallel Tempering MCMC [0.276240219662896]
  Parallel tempering is a meta-algorithm for Markov Chain Monte Carlo that uses multiple chains to sample from tempered versions of the target distribution.<n>We present an adaptive temperature selection algorithm that dynamically adjusts temperatures during sampling using a policy gradient approach.
  arXiv  Detail & Related papers  (2024-09-03T03:12:45Z)
• Total Uncertainty Quantification in Inverse PDE Solutions Obtained with Reduced-Order Deep Learning Surrogate Models [50.90868087591973]
  We propose an approximate Bayesian method for quantifying the total uncertainty in inverse PDE solutions obtained with machine learning surrogate models. We test the proposed framework by comparing it with the iterative ensemble smoother and deep ensembling methods for a non-linear diffusion equation.
  arXiv  Detail & Related papers  (2024-08-20T19:06:02Z)
• EM Distillation for One-step Diffusion Models [65.57766773137068]
  We propose a maximum likelihood-based approach that distills a diffusion model to a one-step generator model with minimal loss of quality.<n>We develop a reparametrized sampling scheme and a noise cancellation technique that together stabilizes the distillation process.
  arXiv  Detail & Related papers  (2024-05-27T05:55:22Z)
• Convergence Acceleration of Markov Chain Monte Carlo-based Gradient Descent by Deep Unfolding [5.584060970507506]
  This study proposes a trainable sampling-based solver for optimization problems (COPs) using a deep-learning technique called deep unfolding. The proposed solver is based on the Ohzeki method that combines Markov-chain Monte-Carlo (MCMC) and gradient descent. The numerical results for a few COPs demonstrated that the proposed solver significantly accelerated the convergence speed compared with the original Ohzeki method.
  arXiv  Detail & Related papers  (2024-02-21T08:21:48Z)
• Optimization of Discrete Parameters Using the Adaptive Gradient Method and Directed Evolution [49.1574468325115]
  The search for an optimal solution is carried out by a population of individuals. Unadapted individuals die, and optimal ones interbreed, the result directed evolutionary dynamics.
  arXiv  Detail & Related papers  (2024-01-12T15:45:56Z)
• Model-Based Reparameterization Policy Gradient Methods: Theory and Practical Algorithms [88.74308282658133]
  Reization (RP) Policy Gradient Methods (PGMs) have been widely adopted for continuous control tasks in robotics and computer graphics. Recent studies have revealed that, when applied to long-term reinforcement learning problems, model-based RP PGMs may experience chaotic and non-smooth optimization landscapes. We propose a spectral normalization method to mitigate the exploding variance issue caused by long model unrolls.
  arXiv  Detail & Related papers  (2023-10-30T18:43:21Z)
• Sampling with Mollified Interaction Energy Descent [57.00583139477843]
  We present a new optimization-based method for sampling called mollified interaction energy descent (MIED) MIED minimizes a new class of energies on probability measures called mollified interaction energies (MIEs) We show experimentally that for unconstrained sampling problems our algorithm performs on par with existing particle-based algorithms like SVGD.
  arXiv  Detail & Related papers  (2022-10-24T16:54:18Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.