FireFlow: Fast Inversion of Rectified Flow for Image Semantic Editing

• URL: http://arxiv.org/abs/2412.07517v1
• Date: Tue, 10 Dec 2024 13:56:26 GMT
• Title: FireFlow: Fast Inversion of Rectified Flow for Image Semantic Editing
• Authors: Yingying Deng, Xiangyu He, Changwang Mei, Peisong Wang, Fan Tang,
• Abstract summary: FireFlow is a simple yet effective zero-shot approach that inherits the startling capacity of ReFlow-based models. We first demonstrate that a carefully designed numerical solver is pivotal for ReFlow inversion. This solver achieves a $3times$ runtime speedup compared to state-of-the-art ReFlow inversion and editing techniques.
• Score: 27.57630797294312
• License:
• Abstract: Though Rectified Flows (ReFlows) with distillation offers a promising way for fast sampling, its fast inversion transforms images back to structured noise for recovery and following editing remains unsolved. This paper introduces FireFlow, a simple yet effective zero-shot approach that inherits the startling capacity of ReFlow-based models (such as FLUX) in generation while extending its capabilities to accurate inversion and editing in $8$ steps. We first demonstrate that a carefully designed numerical solver is pivotal for ReFlow inversion, enabling accurate inversion and reconstruction with the precision of a second-order solver while maintaining the practical efficiency of a first-order Euler method. This solver achieves a $3\times$ runtime speedup compared to state-of-the-art ReFlow inversion and editing techniques, while delivering smaller reconstruction errors and superior editing results in a training-free mode. The code is available at $\href{https://github.com/HolmesShuan/FireFlow}{this URL}$.

Related papers

• Taming Rectified Flow for Inversion and Editing [57.3742655030493]
  Rectified-flow-based diffusion transformers like FLUX and OpenSora have demonstrated outstanding performance in the field of image and video generation. Despite their robust generative capabilities, these models often struggle with inaccuracies. We propose RF-r, a training-free sampler that effectively enhances inversion precision by mitigating the errors in the inversion process of rectified flow.
  arXiv  Detail & Related papers  (2024-11-07T14:29:02Z)
• Simple ReFlow: Improved Techniques for Fast Flow Models [68.32300636049008]
  Diffusion and flow-matching models achieve remarkable generative performance but at the cost of many sampling steps. We propose seven improvements for training dynamics, learning and inference. We achieve state-of-the-art FID scores (without / with guidance, resp.) for fast generation via neural ODEs.
  arXiv  Detail & Related papers  (2024-10-10T11:00:55Z)
• Rectified Diffusion: Straightness Is Not Your Need in Rectified Flow [65.51671121528858]
  Diffusion models have greatly improved visual generation but are hindered by slow generation speed due to the computationally intensive nature of solving generative ODEs. Rectified flow, a widely recognized solution, improves generation speed by straightening the ODE path. We propose Rectified Diffusion, which generalizes the design space and application scope of rectification to encompass the broader category of diffusion models.
  arXiv  Detail & Related papers  (2024-10-09T17:43:38Z)
• Improving the Training of Rectified Flows [14.652876697052156]
  Diffusion models have shown great promise for image and video generation, but sampling from state-of-the-art models requires expensive numerical integration of a generative ODE. One approach for tackling this problem is rectified flows, which iteratively learn smooth ODE paths that are less susceptible to truncation error. We propose improved techniques for training rectified flows, allowing them to compete with emphknowledge distillation methods even in the low NFE setting. Our improved rectified flow outperforms the state-of-the-art distillation methods such as consistency distillation and progressive distillation in both one-step and two
  arXiv  Detail & Related papers  (2024-05-30T17:56:04Z)
• Lightning-Fast Image Inversion and Editing for Text-to-Image Diffusion Models [46.729930784279645]
  We formulate the problem by finding the roots of an implicit equation and devlop a method to solve it efficiently. Our solution is based on Newton-Raphson (NR), a well-known technique in numerical analysis. We show improved results in image and generation of rare objects.
  arXiv  Detail & Related papers  (2023-12-19T19:19:19Z)
• InstaFlow: One Step is Enough for High-Quality Diffusion-Based Text-to-Image Generation [33.70116170511312]
  We propose a novel text-conditioned pipeline to turn Stable Diffusion (SD) into an ultra-fast one-step model. We create the first one-step diffusion-based text-to-image generator with SD-level image quality, achieving an FID of $23.3$ on MS COCO 2017-5k.
  arXiv  Detail & Related papers  (2023-09-12T16:42:09Z)
• GMFlow: Learning Optical Flow via Global Matching [124.57850500778277]
  We propose a GMFlow framework for learning optical flow estimation. It consists of three main components: a customized Transformer for feature enhancement, a correlation and softmax layer for global feature matching, and a self-attention layer for flow propagation. Our new framework outperforms 32-iteration RAFT's performance on the challenging Sintel benchmark.
  arXiv  Detail & Related papers  (2021-11-26T18:59:56Z)
• DeFlow: Learning Complex Image Degradations from Unpaired Data with Conditional Flows [145.83812019515818]
  We propose DeFlow, a method for learning image degradations from unpaired data. We model the degradation process in the latent space of a shared flow-decoder network. We validate our DeFlow formulation on the task of joint image restoration and super-resolution.
  arXiv  Detail & Related papers  (2021-01-14T18:58:01Z)
• LiteFlowNet3: Resolving Correspondence Ambiguity for More Accurate Optical Flow Estimation [99.19322851246972]
  We introduce LiteFlowNet3, a deep network consisting of two specialized modules to address the problem of optical flow estimation. LiteFlowNet3 not only achieves promising results on public benchmarks but also has a small model size and a fast runtime.
  arXiv  Detail & Related papers  (2020-07-18T03:30:39Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.