VibeFlow: Versatile Video Chroma-Lux Editing through Self-Supervised Learning

• URL: http://arxiv.org/abs/2604.13425v1
• Date: Wed, 15 Apr 2026 02:51:26 GMT
• Title: VibeFlow: Versatile Video Chroma-Lux Editing through Self-Supervised Learning
• Abstract summary: This paper proposes VibeFlow, a novel self-supervised framework for chroma-lux editing.<n>We introduce a disentangled data pipeline that enforces the model to adaptively recombine structure from source videos and color-illumination cues from reference images.<n>Our framework generalizes in a zero-shot manner to diverse applications, including video relighting, recoloring, low-light enhancement, day-night translation, and object-specific color editing.
• Score: 31.16852887085847
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Video chroma-lux editing, which aims to modify illumination and color while preserving structural and temporal fidelity, remains a significant challenge. Existing methods typically rely on expensive supervised training with synthetic paired data. This paper proposes VibeFlow, a novel self-supervised framework that unleashes the intrinsic physical understanding of pre-trained video generation models. Instead of learning color and light transitions from scratch, we introduce a disentangled data perturbation pipeline that enforces the model to adaptively recombine structure from source videos and color-illumination cues from reference images, enabling robust disentanglement in a self-supervised manner. Furthermore, to rectify discretization errors inherent in flow-based models, we introduce Residual Velocity Fields alongside a Structural Distortion Consistency Regularization, ensuring rigorous structural preservation and temporal coherence. Our framework eliminates the need for costly training resources and generalizes in a zero-shot manner to diverse applications, including video relighting, recoloring, low-light enhancement, day-night translation, and object-specific color editing. Extensive experiments demonstrate that VibeFlow achieves impressive visual quality with significantly reduced computational overhead. Our project is publicly available at https://lyf1212.github.io/VibeFlow-webpage.

Related papers

• LiveLight: Real-time Streaming Video Relighting with Interactive Control [50.08584152649119]
  We present LiveLight, the first diffusion-based framework for real-time streaming video relighting with interactive 3D lighting control.<n>We overcome three critical challenges: effectively injecting dynamic 3D lighting into a diffusion model, maintaining high-fidelity generation under a low NFE budget, and facilitating continuous streaming for interactive control.<n>Experiments on real-world and synthetic benchmarks demonstrate that LiveLight achieves state-of-the-art relighting quality while running at real-time speed.
  arXiv  Detail & Related papers  (2026-08-03T06:45:00Z)
• LightCrafter: PBR-Conditioned Video Diffusion Refinement for Controllable and Consistent Relighting [40.05860097816268]
  LightCrafter is a hybrid pipeline that reforms video relighting as video translation of a proxy video.<n>Rather than translating the input video directly to the target, we translate a rendering of the input under the target illumination to the final target.<n>We show PBR renders alone already outperform some prior art but struggle with effects like global illumination.
  arXiv  Detail & Related papers  (2026-07-09T00:44:43Z)
• Relit-LiVE: Relight Video by Jointly Learning Environment Video [17.623352754423287]
  We present Relit-LiVE, a novel video relighting framework that produces physically consistent, temporally stable results without requiring prior knowledge of camera pose.<n>Our key insight is to explicitly introduce raw reference images into the rendering process, enabling the model to recover critical scene cues that are inevitably lost or corrupted in intrinsic representations.<n>Our framework naturally supports a wide range of downstream applications, including scene-level rendering, material editing, object insertion, and streaming video relighting.
  arXiv  Detail & Related papers  (2026-05-07T17:58:15Z)
• SAMA: Factorized Semantic Anchoring and Motion Alignment for Instruction-Guided Video Editing [76.349958946335]
  We present SAMA (factorized Semantic Anchoring and Motion Alignment), a framework that factorizes video editing into semantic anchoring and motion modeling.<n>First, we introduce Semantic Anchoring, which establishes a reliable visual anchor by jointly predicting semantic tokens and video latents at sparse anchor frames.<n>Second, Motion Alignment pre-trains the same backbone on motion-centric video restoration pretext tasks.
  arXiv  Detail & Related papers  (2026-03-19T17:59:51Z)
• Kiwi-Edit: Versatile Video Editing via Instruction and Reference Guidance [55.32799307123252]
  We introduce a scalable data generation pipeline that transforms existing video editing pairs into high-fidelity training quadruplets.<n>We propose a unified editing architecture, Kiwi-Edit, that synergizes learnable queries and latent visual features for reference semantic guidance.
  arXiv  Detail & Related papers  (2026-03-02T18:46:28Z)
• FlowPortal: Residual-Corrected Flow for Training-Free Video Relighting and Background Replacement [9.038689270820258]
  FlowPortal is a training-free flow-based video relighting framework.<n>Our core innovation is a Residual-Corrected Flow mechanism that transforms a standard flow-based model into an editing model.<n> Experiments demonstrate that FlowPortal achieves superior performance in temporal coherence, structural preservation, and lighting realism.
  arXiv  Detail & Related papers  (2025-11-23T08:45:17Z)
• UniLumos: Fast and Unified Image and Video Relighting with Physics-Plausible Feedback [31.03901228901908]
  We present UniLumos, a unified relighting framework for both images and videos.<n>We explicitly align lighting effects with the scene structure, enhancing physical plausibility.<n>Experiments demonstrate that UniLumos achieves state-of-the-art relighting with significantly improved physical consistency.
  arXiv  Detail & Related papers  (2025-11-03T15:41:41Z)
• Lumen: Consistent Video Relighting and Harmonious Background Replacement with Video Generative Models [18.008901495139717]
  We propose Lumen, an end-to-end video relighting framework developed on large-scale video generative models.<n>For the synthetic domain, we leverage advanced 3D rendering engine to curate video pairs in diverse environments.<n>For the realistic domain, we adapt a HDR-based lighting simulation to complement the lack of paired in-the-wild videos.
  arXiv  Detail & Related papers  (2025-08-18T14:21:22Z)
• TC-Light: Temporally Coherent Generative Rendering for Realistic World Transfer [47.22201704648345]
  Illumination and texture editing are critical dimensions for world-to-world transfer.<n>Existing techniques generatively re-render the input video to realize the transfer, such as video relighting models and conditioned world generation models.<n>We propose TC-Light, a novel generative computation to overcome these problems.
  arXiv  Detail & Related papers  (2025-06-23T17:59:58Z)
• VanGogh: A Unified Multimodal Diffusion-based Framework for Video Colorization [53.35016574938809]
  Video colorization aims to transform grayscale videos into vivid color representations while maintaining temporal consistency and structural integrity.<n>Existing video colorization methods often suffer from color bleeding and lack comprehensive control.<n>We introduce VanGogh, a unified multimodal diffusion-based framework for video colorization.
  arXiv  Detail & Related papers  (2025-01-16T12:20:40Z)
• Video Decomposition Prior: A Methodology to Decompose Videos into Layers [74.36790196133505]
  This paper introduces a novel video decomposition prior VDP' framework which derives inspiration from professional video editing practices.<n>VDP framework decomposes a video sequence into a set of multiple RGB layers and associated opacity levels.<n>We address tasks such as video object segmentation, dehazing, and relighting.
  arXiv  Detail & Related papers  (2024-12-06T10:35:45Z)
• LightningDrag: Lightning Fast and Accurate Drag-based Image Editing Emerging from Videos [101.59710862476041]
  We present LightningDrag, a rapid approach enabling high quality drag-based image editing in 1 second. Unlike most previous methods, we redefine drag-based editing as a conditional generation task. Our approach can significantly outperform previous methods in terms of accuracy and consistency.
  arXiv  Detail & Related papers  (2024-05-22T15:14:00Z)
• Temporally Consistent Video Colorization with Deep Feature Propagation and Self-regularization Learning [90.38674162878496]
  We propose a novel temporally consistent video colorization framework (TCVC) TCVC effectively propagates frame-level deep features in a bidirectional way to enhance the temporal consistency of colorization. Experiments demonstrate that our method can not only obtain visually pleasing colorized video, but also achieve clearly better temporal consistency than state-of-the-art methods.
  arXiv  Detail & Related papers  (2021-10-09T13:00:14Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.