Alpha as an Efficiency Signal: Visibility-Routed RGBA Image-to-Video Generation

• URL: http://arxiv.org/abs/2608.09355v1
• Date: Mon, 10 Aug 2026 09:35:36 GMT
• Title: Alpha as an Efficiency Signal: Visibility-Routed RGBA Image-to-Video Generation
• Abstract summary: RGBA videos combine RGB appearance with an alpha channel, enabling animated assets to be applied across arbitrary backgrounds.<n>Most existing RGBA video datasets are dominated by photorealistic content.<n>Traditional generate-then-matte pipelines estimate alpha only after RGB synthesis.
• Score: 10.796062926739511
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: RGBA videos combine RGB appearance with an alpha channel, enabling animated assets to be applied across arbitrary backgrounds, which are heavily used in gaming industry. However, generating high-quality RGBA animations for games remains challenging for two reasons. First, most existing RGBA video datasets are dominated by photorealistic content, with limited coverage of game assets. Second, the traditional generate-then-matte pipelines estimate alpha only after RGB synthesis, so semi-transparent regions are often blurred by background, resulting in unstable matting outputs. More recently, many methods have begun to model RGB and alpha jointly, but existing approaches are mostly text-conditioned, and still have unresolved issues in efficiency and quality. To address these challenges, we introduce GameAlpha-2.4K, a 2.4K-clip game-style RGBA video dataset built with matte-friendly synthesis, multi-hypothesis alpha recovery, and compositing-based quality gates. Using this dataset, we train a reference-conditioned RGBA video generator that jointly produces RGB frames and alpha mattes in a single pass. To improve efficiency, we propose a visibility router that identifies transparent tokens in an early stage and bypasses their later DiT updates, while x_0-lock guides them along the original flow-matching schedule toward self-predicted endpoints. Our model obtains lower FVD than traditional two-stage pipelines, and the visibility router skips 35% of token evaluations in the final two DiT denoising steps, providing a 1.2x backbone speedup with negligible quality degradation compared to dense inference.

Related papers

• TransText: Alpha-as-RGB Representation for Transparent Text Animation [35.1650602838868]
  We introduce the first method, for adapting image-to-video models to layer-aware text (glyph) animation.<n>We propose TransText, a framework based on a novel Alpha-as-RGB paradigm to jointly model appearance and transparency.<n>Our experiments demonstrate that TransText significantly outperforms baselines, generating coherent, high-fidelity transparent animations.
  arXiv  Detail & Related papers  (2026-03-18T17:16:40Z)
• Wan-Alpha: High-Quality Text-to-Video Generation with Alpha Channel [14.361698701397545]
  Wan-Alpha is a new framework that generates transparent videos by learning both RGB and alpha channels jointly.<n>Compared with state-of-the-art methods, our model demonstrates superior performance in visual quality, motion realism, and transparency rendering.
  arXiv  Detail & Related papers  (2025-09-29T16:08:21Z)
• AlphaVAE: Unified End-to-End RGBA Image Reconstruction and Generation with Alpha-Aware Representation Learning [32.798523698352916]
  We propose ALPHA, the first comprehensive RGBA benchmark that adapts standard RGB metrics to four-channel images via alpha blending over canonical backgrounds.<n>We further introduce ALPHAVAE, a unified end-to-end RGBA VAE that extends a pretrained RGB VAE by incorporating a dedicated alpha channel.<n>Our RGBA VAE, trained on only 8K images in contrast to 1M used by prior methods, achieves a +4.9 dB improvement in PSNR and a +3.2% increase in SSIM over LayerDiffuse in reconstruction.
  arXiv  Detail & Related papers  (2025-07-12T14:53:42Z)
• Human Activity Recognition using RGB-Event based Sensors: A Multi-modal Heat Conduction Model and A Benchmark Dataset [65.76480665062363]
  Human Activity Recognition primarily relied on traditional RGB cameras to achieve high-performance activity recognition.<n>Challenges in real-world scenarios, such as insufficient lighting and rapid movements, inevitably degrade the performance of RGB cameras.<n>In this work, we rethink human activity recognition by combining the RGB and event cameras.
  arXiv  Detail & Related papers  (2025-04-08T09:14:24Z)
• TransAnimate: Taming Layer Diffusion to Generate RGBA Video [3.7031943280491997]
  TransAnimate is an innovative framework that integrates RGBA image generation techniques with video generation modules.<n>We introduce an interactive motion-guided control mechanism, where directional arrows define movement and colors adjust scaling.<n>We have developed a pipeline for creating an RGBA video dataset, incorporating high-quality game effect videos, extracted foreground objects, and synthetic transparent videos.
  arXiv  Detail & Related papers  (2025-03-23T04:27:46Z)
• TransPixeler: Advancing Text-to-Video Generation with Transparency [43.6546902960154]
  We introduce TransPixeler, a method to extend pretrained video models for RGBA generation while retaining the original RGB capabilities.<n>Our approach effectively generates diverse and consistent RGBA videos, advancing the possibilities for VFX and interactive content creation.
  arXiv  Detail & Related papers  (2025-01-06T13:32:16Z)
• Meta 3D AssetGen: Text-to-Mesh Generation with High-Quality Geometry, Texture, and PBR Materials [58.178540282148475]
  AssetGen is a significant advancement in text-to-3D generation. It produces faithful, high-quality meshes with texture and material control.
  arXiv  Detail & Related papers  (2024-07-02T17:21:47Z)
• Lumina-Next: Making Lumina-T2X Stronger and Faster with Next-DiT [120.39362661689333]
  We present an improved version of Lumina-T2X, showcasing stronger generation performance with increased training and inference efficiency. Thanks to these improvements, Lumina-Next not only improves the quality and efficiency of basic text-to-image generation but also demonstrates superior resolution extrapolation capabilities.
  arXiv  Detail & Related papers  (2024-06-05T17:53:26Z)
• Dual Swin-Transformer based Mutual Interactive Network for RGB-D Salient Object Detection [67.33924278729903]
  In this work, we propose Dual Swin-Transformer based Mutual Interactive Network. We adopt Swin-Transformer as the feature extractor for both RGB and depth modality to model the long-range dependencies in visual inputs. Comprehensive experiments on five standard RGB-D SOD benchmark datasets demonstrate the superiority of the proposed DTMINet method.
  arXiv  Detail & Related papers  (2022-06-07T08:35:41Z)
• Bi-directional Cross-Modality Feature Propagation with Separation-and-Aggregation Gate for RGB-D Semantic Segmentation [59.94819184452694]
  Depth information has proven to be a useful cue in the semantic segmentation of RGBD images for providing a geometric counterpart to the RGB representation. Most existing works simply assume that depth measurements are accurate and well-aligned with the RGB pixels and models the problem as a cross-modal feature fusion. In this paper, we propose a unified and efficient Crossmodality Guided to not only effectively recalibrate RGB feature responses, but also to distill accurate depth information via multiple stages and aggregate the two recalibrated representations alternatively.
  arXiv  Detail & Related papers  (2020-07-17T18:35:24Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.