Maximum entropy optimal density control of discrete-time linear systems and Schr\"odinger bridges

• URL: http://arxiv.org/abs/2204.05263v2
• Date: Sun, 13 Aug 2023 05:13:11 GMT
• Title: Maximum entropy optimal density control of discrete-time linear systems and Schr\"odinger bridges
• Authors: Kaito Ito, Kenji Kashima
• Abstract summary: We consider an entropy-regularized version of optimal density control of deterministic discrete-time linear systems. Despite the merits, high-entropy control policies induced by the regularization introduce probabilistic uncertainty into systems. We reveal that the MaxEnt optimal density control gives the so-called Schr"odinger bridge associated to a discrete-time linear system.
• Score: 0.087024326813104
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We consider an entropy-regularized version of optimal density control of deterministic discrete-time linear systems. Entropy regularization, or a maximum entropy (MaxEnt) method for optimal control has attracted much attention especially in reinforcement learning due to its many advantages such as a natural exploration strategy. Despite the merits, high-entropy control policies induced by the regularization introduce probabilistic uncertainty into systems, which severely limits the applicability of MaxEnt optimal control to safety-critical systems. To remedy this situation, we impose a Gaussian density constraint at a specified time on the MaxEnt optimal control to directly control state uncertainty. Specifically, we derive the explicit form of the MaxEnt optimal density control. In addition, we also consider the case where density constraints are replaced by fixed point constraints. Then, we characterize the associated state process as a pinned process, which is a generalization of the Brownian bridge to linear systems. Finally, we reveal that the MaxEnt optimal density control gives the so-called Schr\"odinger bridge associated to a discrete-time linear system.

Related papers

• Krotov Type Optimization of Coherent and Incoherent Controls for Open Two-Qubit Systems [77.34726150561087]
  This work considers two-qubit open quantum systems driven by coherent and incoherent controls. Incoherent control induces time-dependent decoherence rates via time-dependent spectral density of the environment. The system evolves according to the Gorini-Kossakowski-Sudarshan-Lindblad master equation with time-dependent coefficients.
  arXiv  Detail & Related papers  (2023-08-11T13:17:19Z)
• Optimal State Manipulation for a Two-Qubit System Driven by Coherent and Incoherent Controls [77.34726150561087]
  State preparation is important for optimal control of two-qubit quantum systems. We exploit two physically different coherent control and optimize the Hilbert-Schmidt target density matrices.
  arXiv  Detail & Related papers  (2023-04-03T10:22:35Z)
• Optimal control for state preparation in two-qubit open quantum systems driven by coherent and incoherent controls via GRAPE approach [77.34726150561087]
  We consider a model of two qubits driven by coherent and incoherent time-dependent controls. The dynamics of the system is governed by a Gorini-Kossakowski-Sudarshan-Lindblad master equation. We study evolution of the von Neumann entropy, purity, and one-qubit reduced density matrices under optimized controls.
  arXiv  Detail & Related papers  (2022-11-04T15:20:18Z)
• A Physics-informed Deep Learning Approach for Minimum Effort Stochastic Control of Colloidal Self-Assembly [9.791617215182598]
  The control objective is formulated in terms of steering the state PDFs from a prescribed initial probability measure towards a prescribed terminal probability measure with minimum control effort. We derive the conditions of optimality for the associated optimal control problem. The performance of the proposed solution is demonstrated via numerical simulations on a benchmark colloidal self-assembly problem.
  arXiv  Detail & Related papers  (2022-08-19T07:01:57Z)
• On optimization of coherent and incoherent controls for two-level quantum systems [77.34726150561087]
  This article considers some control problems for closed and open two-level quantum systems. The closed system's dynamics is governed by the Schr"odinger equation with coherent control. The open system's dynamics is governed by the Gorini-Kossakowski-Sudarshan-Lindblad master equation.
  arXiv  Detail & Related papers  (2022-05-05T09:08:03Z)
• Application of Pontryagin's Maximum Principle to Quantum Metrology in Dissipative Systems [8.920103626492315]
  We look for the optimal control that maximizes quantum Fisher information for "twist and turn" problem. We find that the optimal control is singular without dissipation but can become unbounded once the quantum decoherence is introduced.
  arXiv  Detail & Related papers  (2022-04-30T00:02:57Z)
• A Quantum Optimal Control Problem with State Constrained Preserving Coherence [68.8204255655161]
  We consider a three-level $Lambda$-type atom subjected to Markovian decoherence characterized by non-unital decoherence channels. We formulate the quantum optimal control problem with state constraints where the decoherence level remains within a pre-defined bound.
  arXiv  Detail & Related papers  (2022-03-24T21:31:34Z)
• Density Constrained Reinforcement Learning [9.23225507471139]
  We study constrained reinforcement learning from a novel perspective by setting constraints directly on state density functions. We leverage the duality between density functions and Q functions to develop an effective algorithm to solve the density constrained RL problem optimally. We prove that the proposed algorithm converges to a near-optimal solution with a bounded error even when the policy update is imperfect.
  arXiv  Detail & Related papers  (2021-06-24T04:22:03Z)
• Numerical estimation of reachable and controllability sets for a two-level open quantum system driven by coherent and incoherent controls [77.34726150561087]
  The article considers a two-level open quantum system governed by the Gorini--Kossakowski--Lindblad--Sudarshan master equation. The system is analyzed using Bloch parametrization of the system's density matrix.
  arXiv  Detail & Related papers  (2021-06-18T14:23:29Z)
• Stochastic optimal control formalism for an open quantum system [15.076862040458124]
  A procedure is developed which allows one to express Pontryagin's maximum principle for dissipative quantum system. Time-optimal computing controls can be efficiently computed without the density matrix.
  arXiv  Detail & Related papers  (2020-11-06T15:37:51Z)
• Time-optimal control of a dissipative qubit [9.035958018596155]
  A formalism based on Pontryagin's maximum principle is applied to determine the time-optimal protocol. Dissipation typically drives the system to the maximally mixed state. For some specific dissipation channel, however, the optimal control can keep the system from the maximum entropy state for infinitely long.
  arXiv  Detail & Related papers  (2020-02-18T15:43:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.