Path-dependent Discrete Amortized Inference

• URL: http://arxiv.org/abs/2608.08644v1
• Date: Sun, 09 Aug 2026 11:35:05 GMT
• Title: Path-dependent Discrete Amortized Inference
• Abstract summary: We consider the problem of sampling compositional and discrete objects from a given unnormalized posterior distribution.<n>We show that the Markovian assumption can both hamper signal propagation during training and catastrophically reduce the learned sampler's expressivity.<n>We propose lifting the MDP with a learnable latent dynamical system that allows the underlying policy to depend on the entire past trajectory.
• Score: 7.20046890493201
• License: http://creativecommons.org/publicdomain/zero/1.0/
• Abstract: We consider the problem of sampling compositional and discrete objects from a given unnormalized posterior distribution. Notably, recent studies have shown that this problem can be efficiently solved by learning a deterministic Markov Decision Process (MDP) that progressively builds each object in proportion to the posterior. In this work, however, we demonstrate that the Markovian assumption can both hamper signal propagation during training and catastrophically reduce the learned sampler's expressivity due to state aliasing. To address these issues, we propose lifting the MDP with a learnable latent dynamical system that allows the underlying policy to depend on the entire past trajectory---and not only on the current state. In view of this, we refer to the resulting method as path-dependent discrete amortized inference. Importantly, we provably extend existing learning algorithms for discrete amortized samplers to our setting. In experiments on standard benchmark problems, we also show that our approach often leads to faster learning convergence and improved state space exploration relatively to prior techniques.

Related papers

• Causal Algorithmic Recourse: Foundations and Methods [57.73269033551628]
  Key feature of AI systems is the ability to provide recommendations for how an individual may reverse a negative decision.<n>Existing approaches treat recourse outcomes as counterfactuals of a fixed unit.<n>We develop a causal framework that models recourse as a process over pre- and post-intervention outcomes.
  arXiv  Detail & Related papers  (2026-05-12T00:55:19Z)
• The Benefits of Temporal Correlations: SGD Learns k-Juntas from Random Walks Efficiently [17.896041616302657]
  temporal correlations in the data can make certain sparse learning problems efficiently learnable by gradient-based methods.<n>We show that for large-batch gradient methods using standard convex pointwise losses, temporal correlations do not provide the same advantage.
  arXiv  Detail & Related papers  (2026-05-11T09:11:20Z)
• Reinforcement learning based data assimilation for unknown state model [3.032674692886751]
  We propose a novel method that integrates reinforcement learning with ensemble-based Bayesian ffltering methods.<n>The proposed framework accommodates a wide range of observation scenarios, including nonlinear and partially observed measurement models.<n>A few numerical examples demonstrate that the proposed method achieves superior accuracy and robustness in high-dimensional settings.
  arXiv  Detail & Related papers  (2025-11-04T05:58:37Z)
• Amortized Posterior Sampling with Diffusion Prior Distillation [55.03585818289934]
  Amortized Posterior Sampling is a novel variational inference approach for efficient posterior sampling in inverse problems.<n>Our method trains a conditional flow model to minimize the divergence between the variational distribution and the posterior distribution implicitly defined by the diffusion model.<n>Unlike existing methods, our approach is unsupervised, requires no paired training data, and is applicable to both Euclidean and non-Euclidean domains.
  arXiv  Detail & Related papers  (2024-07-25T09:53:12Z)
• Observation-Guided Diffusion Probabilistic Models [41.749374023639156]
  We propose a novel diffusion-based image generation method called the observation-guided diffusion probabilistic model (OGDM) Our approach reestablishes the training objective by integrating the guidance of the observation process with the Markov chain. We demonstrate the effectiveness of our training algorithm using diverse inference techniques on strong diffusion model baselines.
  arXiv  Detail & Related papers  (2023-10-06T06:29:06Z)
• Primal Dual Continual Learning: Balancing Stability and Plasticity through Adaptive Memory Allocation [86.8475564814154]
  We show that it is both possible and beneficial to undertake the constrained optimization problem directly. We focus on memory-based methods, where a small subset of samples from previous tasks can be stored in a replay buffer. We show that dual variables indicate the sensitivity of the optimal value of the continual learning problem with respect to constraint perturbations.
  arXiv  Detail & Related papers  (2023-09-29T21:23:27Z)
• Score-based Data Assimilation [7.215767098253208]
  We introduce score-based data assimilation for trajectory inference. We learn a score-based generative model of state trajectories based on the key insight that the score of an arbitrarily long trajectory can be decomposed into a series of scores over short segments.
  arXiv  Detail & Related papers  (2023-06-18T14:22:03Z)
• Reflected Diffusion Models [93.26107023470979]
  We present Reflected Diffusion Models, which reverse a reflected differential equation evolving on the support of the data. Our approach learns the score function through a generalized score matching loss and extends key components of standard diffusion models.
  arXiv  Detail & Related papers  (2023-04-10T17:54:38Z)
• Posterior sampling with CNN-based, Plug-and-Play regularization with applications to Post-Stack Seismic Inversion [0.0]
  Uncertainty quantification is crucial to inverse problems, as it could provide valuable information about the inversion results. We present a framework that performs posterior inference by implicitly regularizing the Kullback-Leibler divergence loss with a CNN-based denoiser. We call this algorithm new Plug-and-Play Stein Vari-SVGD and demonstrate its ability in producing high-resolution, trustworthy samples.
  arXiv  Detail & Related papers  (2022-12-30T08:20:49Z)
• Reinforcement Learning with a Terminator [80.34572413850186]
  We learn the parameters of the TerMDP and leverage the structure of the estimation problem to provide state-wise confidence bounds. We use these to construct a provably-efficient algorithm, which accounts for termination, and bound its regret.
  arXiv  Detail & Related papers  (2022-05-30T18:40:28Z)
• Provable RL with Exogenous Distractors via Multistep Inverse Dynamics [85.52408288789164]
  Real-world applications of reinforcement learning (RL) require the agent to deal with high-dimensional observations such as those generated from a megapixel camera. Prior work has addressed such problems with representation learning, through which the agent can provably extract endogenous, latent state information from raw observations. However, such approaches can fail in the presence of temporally correlated noise in the observations.
  arXiv  Detail & Related papers  (2021-10-17T15:21:27Z)
• A Kernel-Based Approach to Non-Stationary Reinforcement Learning in Metric Spaces [53.47210316424326]
  KeRNS is an algorithm for episodic reinforcement learning in non-stationary Markov Decision Processes. We prove a regret bound that scales with the covering dimension of the state-action space and the total variation of the MDP with time.
  arXiv  Detail & Related papers  (2020-07-09T21:37:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.