From Open Loop to Closed Loop: A Test-Time Iterative Optimization Framework for Reference-Consistent Image Generation

• URL: http://arxiv.org/abs/2607.04691v1
• Date: Mon, 06 Jul 2026 05:36:30 GMT
• Title: From Open Loop to Closed Loop: A Test-Time Iterative Optimization Framework for Reference-Consistent Image Generation
• Abstract summary: controllable image generation has made significant strides by incorporating visual reference conditions.<n>Existing methods predominantly operate as open-loop systems.<n>We propose a novel test-time iterative optimization framework that reformulates reference-consistent generation as a closed-loop dynamic tracking problem.
• Score: 108.02425846269234
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: While controllable image generation has made significant strides by incorporating visual reference conditions, existing methods predominantly operate as open-loop systems. They inject control signals in a strictly feed-forward manner, failing to guarantee strict fidelity to the reference due to the absence of active feedback and error correction mechanisms. To address this fundamental limitation, we propose a novel test-time iterative optimization framework that reformulates reference-consistent generation as a closed-loop dynamic tracking problem. By treating the pre-trained generative model as a control plant, our framework employs a sensor-controller architecture driven by a modified Proportional-Integral-Derivative (PID) algorithm. This mechanism iteratively optimizes the latent control signals at test time based on the sensed discrepancy between the generated output and the reference target. Notably, this approach is entirely training-free, model-agnostic, and integrates seamlessly around existing diffusion pipelines. Extensive evaluations across ID-preserving, pose-controlled, and depth-controlled generation tasks validate the universality of our method. Empirical results demonstrate improvements over computation-matched open-loop baselines, achieving relative performance gains of up to 25.36\% for facial similarity, alongside spatial error reductions of up to 27.71\% for pose alignment and 28.50\% for depth consistency. More broadly, this work offers a new conceptual perspective: it demonstrates that controllable generation can be effectively managed as a dynamic feedback system, bringing the rigorous principles of classical control theory into the optimization of generative models. Code is available at https://github.com/zzdrill/From-Open-Loop-to-Closed-Loop.

Related papers

• REFINE: Trajectory Representation Learning via Closed-Loop Transcription -- Extended Version [22.872723951282282]
  REFINE is a simple yet effective lEarning Framework vIa closed-loop trajector rEfinement for trajectory data.<n>We provide a control-theoretic analysis that establishes guarantees for the proposed closed-loop optimization.<n>Experiments on four real-world datasets demonstrate that REFINE consistently outperforms state-of-the-art methods.
  arXiv  Detail & Related papers  (2026-09-07T08:26:49Z)
• LCPNet: Latent Consistent Proximal Unfolding Network for Infrared Small Target Detection [50.3892176822492]
  Infrared small target detection (IRSTD) aims to identify long distance small targets from complex infrared backgrounds.<n>Deep learning methods have improved IRSTD by learning discriminative image-to-mask mappings.<n>Deep unfolding methods partially address this issue by embedding model-driven iterations into neural networks.<n>We propose Latent Consistent Proximal unfolding network (LCPNet) to address these limitations.
  arXiv  Detail & Related papers  (2026-07-06T02:15:09Z)
• ParetoPilot: Zero-Surrogate Offline Multi-Objective Optimization via Infer-Perturb-Guide Diffusion [28.29207284864795]
  We propose a novel zero-surrogate diffusion framework for offline MOO.<n>Perturb-Guide engine is seamlessly interleaved within the unconditional denoising steps of the reverse generation process.<n>Our approach preserves data privacy while achieving hypervolume improvement.
  arXiv  Detail & Related papers  (2026-06-03T05:27:21Z)
• Diffusion Controller: Framework, Algorithms and Parameterization [54.82539154511621]
  We present a unified control-theoretic view that casts reverse diffusion sampling as state-only control within (generalized) linearly-solvable Markov Decision Processes.<n>Under this framework, control acts by reweighting the pretrained reverse-time transition kernels, balancing terminal objectives against an $f$-divergence cost.<n>Experiments on Stable Diffusion v1.4 show consistent gains in preference-alignment win rates and improved quality-efficiency trade-offs.
  arXiv  Detail & Related papers  (2026-03-07T01:49:59Z)
• Score-Guided Proximal Projection: A Unified Geometric Framework for Rectified Flow Editing [1.0312968200748118]
  Rectified Flow models achieve state-of-the-art generation quality, yet controlling them for precise tasks remains a challenge.<n>Current approaches bifurcate into inversion-based guidance, which suffers from "geometric locking"<n>We propose Score-Guided Proximal Projection, a unified framework that bridges the gap between deterministic optimization and contraction sampling.
  arXiv  Detail & Related papers  (2026-03-05T23:44:45Z)
• Closing the Loop: A Control-Theoretic Framework for Provably Stable Time Series Forecasting with LLMs [22.486083545585984]
  Large Language Models (LLMs) have recently shown exceptional potential in time series forecasting.<n>Existing approaches typically employ a naive autoregressive generation strategy.<n>We propose textbfF-LLM, a novel closed-loop framework.
  arXiv  Detail & Related papers  (2026-02-13T09:35:12Z)
• Why Steering Works: Toward a Unified View of Language Model Parameter Dynamics [81.80010043113445]
  Local weight fine-tuning, LoRA-based adaptation, and activation-based interventions are studied in isolation.<n>We present a unified view that frames these interventions as dynamic weight updates induced by a control signal.<n>Across methods, we observe a consistent trade-off between preference and utility: stronger control increases preference while predictably reducing utility.
  arXiv  Detail & Related papers  (2026-02-02T17:04:36Z)
• ScaleWeaver: Weaving Efficient Controllable T2I Generation with Multi-Scale Reference Attention [86.93601565563954]
  ScaleWeaver is a framework designed to achieve high-fidelity, controllable generation upon advanced visual autoregressive( VAR) models.<n>The proposed Reference Attention module discards the unnecessary attention from image$rightarrow$condition, reducing computational cost.<n>Experiments show that ScaleWeaver delivers high-quality generation and precise control while attaining superior efficiency over diffusion-based methods.
  arXiv  Detail & Related papers  (2025-10-16T17:00:59Z)
• Fully Spiking Neural Networks for Unified Frame-Event Object Tracking [17.626181371045575]
  We propose the first fully Spiking Frame-Event Tracking framework called SpikeFET.<n>This network achieves synergistic integration of convolutional local feature extraction and Transformer-based global modeling within the spiking paradigm.<n>We show that proposed framework achieves superior tracking accuracy over existing methods while significantly reducing power consumption.
  arXiv  Detail & Related papers  (2025-05-27T07:53:50Z)
• Adaptive Control and Regret Minimization in Linear Quadratic Gaussian (LQG) Setting [91.43582419264763]
  We propose LqgOpt, a novel reinforcement learning algorithm based on the principle of optimism in the face of uncertainty. LqgOpt efficiently explores the system dynamics, estimates the model parameters up to their confidence interval, and deploys the controller of the most optimistic model.
  arXiv  Detail & Related papers  (2020-03-12T19:56:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.